darktable ep 139 - Why do my RAW files look like crap?

#darktable #tutorial #processing
In this episode, we look at just why newly-imported RAW files look great in the lighttable view, but look all washed-out in the darkroom view.
As always, thanks to my tier 1 Patreon supporters:
Leona
Jon Hoskins
Alun Spencer
Rohn Shearer
Martin Vines
Matthias Liebich
Michael Rasmussen
Martin
Andy McGarty
Preben Rasmussen
Ignacio Mauleón
Ulrich Gesing
Bart Bartels
Daniel
Robert Stites
Emil Ohlsson
Steve Ansell
Fernando Pérez
Frank Jäger
Ferg
Rob van Esseveld
Erik Oksanen
David Staves
Gitte
Jürgen Bauer
Rudolfs
chris8142
Wolfgang Schimeck
Kevin McKibben
Alexander Haak
K&C
Carsten Wünsch
Hitesh Tulsani
mike s
If you'd like to support my work, get access to these videos before everyone else, AND get access to additional content, please head over to patreon.com/understandingdarktable and pick a tier that is comfortable for you.

[@Eigil_Skovgaard]

I remember long ago when I had exactly that problem with Darktable, comparing the initial presentation of a raw file with what I was use to from Adobe Camera Raw. Today my first thought was - Why the h... is Bruce wasting time on a problem that everybody knows the answer to? There you see, arrogance up front. We forget that it was not that obvious to know about the hidden adjustments in ACR, Lightroom, Capture One, ON1, etc. just a few years ago. So, with this episode you opened the door for new users of Darktable and removed one of the reasons for failure even before Darktable would have a chance to shine. Well done, Bruce.

[@audio2u]

Thanks for the kind words! I do try! :)

[@xperience-evolution]

First time I shot raw with my Alpha around 4 years ago I was "shocked". It looked so good on Camera and so flat and lifeless in DT😅 But then I learned - mostly thanks to your Videos - what the advantage of raw is and how to make them look great again.

[@audio2u]

Thanks for the kind words. Glad to hear you are getting great results with darktable.

[@bernym4047]

Another issue related to raw files but not related directly to dt that many photographers don't realise is that if you use the histogram to check for clipping, that also uses a JPEG image which being a compressed version does not have as wide a dynamic range as the raw file and thus may indicate clipping where in fact there is none as the raw file is capturing the full range of the image but you will only be able to check that after importing into dt. New glass looks very tasty. 😀

[@Ichijoe2112]

Definitely been guilty of that. Your outside, it's a bright sunny day, it's hard to chimp the screen. Never checked the Histogram, and 96% of the work was overexposed to clipping white.

[@timroach5898]

Thats why I have the thought that if you shoot in neutral you should get a more acurate histogram.

[@audio2u]

Absolutely Berny. I learned that way too late in my journey, but have been aware of it for about 10 years now. As you say, another trap for young players!

[@edwardprior5946]

@@timroach5898 I'm sure you know but if not you can look at uni-white balance..this is a way to get a more accurate histogram until the camera manufacturers decide to include the raw histogram as an option...

[@audio2u]

@edwardprior5946 what is this? Is it some manufacturer-specific setting, because I've never noticed anything by that name in the Sony menus on my cameras.

[@awiseman-so2ke]

The improvements you made to your videos through editing are truly remarkable. I was particularly impressed with the masked video in the bottom right corner; it added a more connected and polished feel to the overall product.

[@audio2u]

Thanks! Not my work, though. I had someone else edit the video.

[@danderth]

I really appreciate your content; I've been looking forward to this all week, thanks. I had to laugh out loud near the end of your video when you said, "no, darktable is not broken," because I've had that same thought.

[@audio2u]

Glad you got value out of this! 😃

[@lxhk3595]

Once you understood it, it becomes counter intuitive not to start your personal edit with the most neutral starting point. Once you know your preference, darktable gives you ways to easily automate your standard steps with presets and styles.

[@audio2u]

Exactly.

[@DalsPhotography]

Are you still on Patreon , right? Thanks for a refresing lesson Bruce! Best from Uruguay.

[@audio2u]

Yes, still on patreon.com slash understandingdarktable

[@spark198rus]

i was guessing the same but you just clarified on that, thanks

[@audio2u]

No problem!

[@joaobrissonlopes1522]

Very true! My first steps into darktable with RAW hit this "wall" of misunderstanding so much so that I was using Canon's Digital Photo Professional at the same time which goes someway along the JPEG way my Canon camera went. I remember that for a while I considered dropping darktable ...

[@audio2u]

Well, hopefully this video will enlighten other users somewhere down the track and save them the doubt! Glad you found your way through! 😃

[@JasonGillmanJr]

Bruce, decided to get back into photography and your videos have been *immensely* helpful picking up darktable (I got my new rig last month, so 4.6 was already out for a minute based on your videos). By the way, if you decide to upgrade or add another body, the A7IV has been a treat! The in-body image stabilization is a great tool to have in the bag.

[@audio2u]

Great to hear my videos have been helpful to you! And my a7iii has IBIS as well. It's awesome. We are Abbott to get a7iv bodies at the radio station, so I'll be keen to play with the iv and see how it differs from my a7iii.

[@JasonGillmanJr]

@@audio2u It blows my mind how things have progressed since I was running around with a Canon 40D. ISO 20000 shots? The profiled denoise module does a good enough job I don't even care. It's wild.

[@audio2u]

It really kicks butt, hey? I love how I can enable that module and not even need to tweak parameters. It's just.... THERE!

[@GaryParris]

Well done Bruce! good to see you address this for new users!

[@audio2u]

Thanks Gary!

[@lxhk3595]

Good to have u back after your break

[@audio2u]

Thanks!

[@Kamil.SZAPER]

OMG I really needed this kind of video. Thank you very much. Now I need many nights to watch all your Darktable videos 😂

[@audio2u]

Glad you found this helpful!

[@victoralcobia3393]

Well done Bruce. Thank you👏

[@audio2u]

No problemo! :)

[@andrewgreig1197]

Hey Bruce, Great topic. I noticed you had RAW+jpg in the darktable file manager. When I upload my files I separate the RAW from the jpgs. I never look at the jpgs as I upload to the RAW folder and then send the jpgs to the jpg folder.I never look at the jpgs, I keep them only as long as it takes to finish processing the RAW files, I export from the RAW files and when those jpgs are safe 'n' sound,I delete all of the out of camera jpgs. They are a form of insurance until the edits are in the bag.

[@audio2u]

Interesting approach. I just couldn't be arsed doing 2 import routines to separate them! :) But I understand your philosophy.

[@emrg777]

You could just transfer both and have DT ignore the JPG on import. Also could use filters... but each to their own

[@mikeremski2102]

Good stuff as expected. Thanks.

[@audio2u]

Any time!

[@somekid2091]

suppose i would like to start at something closer to the in camera jpeg colors while still taking advantage of darktables powerful tools inside of the scene referred workflow? my capture one subscription is up at the end of the year and i like to ditch them if favor of darktable. the problem is im finding it a hell of a lot more work than im used to to get the colors that i like. i really enjoy the colors from my fuji camera thats why i bought it. but i also need to retain the flexibility provided from working with raw. youve been a great help, if you or anyone has an answer to this youd save me a whole lot of headache. thanks!

[@audio2u]

Good luck! I've not found a way to get the RAW files to have the colours/saturation of the in-camera jpegs as a starting point.

[@luna010]

Base curve exists and has camera/maker-based presets for this reason. For tone mapping, I have my last module be base curve and _keep preserve colors set to none_ . This tends to be extremely close to the in-camera jpeg in terms of color and value for me before adding any extra modules. I have display-referred default selected, but my actual workflow and the modules I use more match the intended scene-referred workflow.

[@timroach5898]

Also as a side note, if you want your raw file to look like your jpg view from camera use the free software from your camera manufacturer to read the raw file. It will read the camera settings embedded into the header of the raw file and recreate the exact same jpg view that you seen in the back of the camera. You can then export it as a 16 bit TIFF to use in any software after. Sony Imaging edge and Nikon NX Studio both do it. Those same software will also show yoou the focus point where you had the point set in the image. I check that sometimes if I have a missed focus photo to see if I had the focus point not where I should have.

[@xperience-evolution]

Unfortunately these Programs are not available on Linux. Although some Cameras run Linux and Sony has a lot of other things running the Linux Kernel, they don't care about a 2% market share

[@audio2u]

Great tip, Tim, but as experience-evolution said, that doesn't help the Linux users at all. Admittedly, that's a small share of the users, but it is exclusion no-one the less.

[@JHuffPhoto]

I have my preferences set to not use the embedded .jpg for the thumbnail of a raw file on import. Takes a little more time to render but I will start off having an idea of what the starting point for my image will be. I can usually have a decent image within a few clicks with presets or LUT files. Then I can work on any details of files I want to actually use (print, display or post to social media).

[@audio2u]

Nice one!

[@karstenleinekefotografie]

Thanks Bruce 👍

[@audio2u]

No problem! :)

[@Strandlaeufer77]

As always, a lovely explanation. So we're just waiting for the magic button to activate and set all necessary functions in DT that an equivalent of the jpg falls out :)

[@emrg777]

The base curve would be somewhat comparable and if it's a good one this in conjunction with a style made using DT chart will get you fairly close... but this has roots in the older display referred workflow. Honestly. the base curve has been moved now in the pipeline and so using it and setting the color preservation to none. Maybe use the fusion options and one instance of the tone eq esp if you have HNR and you can get pretty nice results

[@audio2u]

See, almost everyone thinks that the in-camera jpeg is the end goal. But I would argue that is you really understand RAW processing, and you understand the benefits of a scene-referred workflow, you can get better results than the in-camera jpeg most of the time.

[@stefanbuchner8710]

Hi Bruce, thank you for thus video. Is it in some way possible to get the JPEG-status for RAW datas as a starting point for further processing? Often I love the JPEG and just want to improve some things from this point on. Regards from Germany Stefan

[@audio2u]

Not in darktable, that I'm aware of. That in-camera jpeg has all of your camera manufacturer's secret sauce baked into it. So, any "jpeg-like" starting point for a RAW file would only be reverse engineering at best.

[@donaldgray2501]

Can you set darktable to have the jpeg adjustments as the starting point for your raw file in darktable? As I think it would be a less daunting task to edit from a good base point such as the jpeg.

[@emrg777]

The dt equivalent would be to use the base curve and dt chart feature to create a matching style for a raw JPG pair... you would need a color checker... it does a fairly good job but is not following scene referred workflow

[@audio2u]

What he said! The base curve module had a ton of camera presets which will get you part of the way there, but you are following a display-referred workflow.

[@emrg777]

Two comments Bruce...it might have been good to note that you can choose in preferences not to use the embedded jpg...but its would be slower. Also if you google "Develop a raw file by hand" There is a great document that walks through with examples the stages of raw development..its great context for yhe process and those extra modules you demobstrated at the start...

[@audio2u]

Got a link?

[@emrg777]

@@audio2u On my phone I'll dig it up

[@WLarchive]

I've never used any other processing software - how does Lightroom for example handle this? Does it just only ever show you the "RAW unedited image"? Or does the starting point for editing always have some "JPEG processing" applied?

[@emrg777]

LR is strongly processed out of the gate... you have to work to get a vanilla image

[@audio2u]

Honestly, I don't recall! It's been about 8 years since I looked at Lightroom.

[@audio2u]

That's such a bad decision on Adobe's part.

[@WLarchive]

@@emrg777 interesting, thanks

[@andrewwilkin1923]

Hi Bruce, totally correct, the image on the back of the camera is a jpeg. CR2 (Canon) file format, for example, stores 3 jpg's, two different size thumbnails for a file manager, a one quarter size jpg (preview file) that is displayed on the camera back screen, exif information and the raw data. A CR2 is basically a TIFF file. As you're a linux user you can use Exiftool to extract that jpg preview file should you need it. So I only shoot in raw but have set my jpeg mode to neutral which is pretty flat. This gives a more realistic "raw" view on the back of the camera. I assume that most other camera makes work the same.

[@audio2u]

Wow. Nope, Sony does not do that (not that I'm aware of, anyway!).

[@Ichijoe2112]

Cool Lense bro. But, outside of any weight issues, and there are. Why would One choose the 150-500 DI VC G2 when the 150-600 DI VC G2 is a thing? (Ok there I went assuming these were both Canon EF Mount. (i.e. DI, and not in the case of the 150-500 a DIII (i.e. Mirrorless Objective). Understanding that I guess I can spare myself the ignorance of suggesting that you should get the optional Tap, to tune said lense to your body. All that said I recently picked up a mint copy of the 150-600 DI VC G2 for Canon EF, and use it for Birding / BiF, and I am loving it. Given I'm NOT on a nu-type system I still need to tune up my lense with my 5Dmkii to improve upon the focal plane. To be fair though. I'm more or less pinning most of the crappy shots on my lack of skill at being able to pan with the bird in frame. But, I agree the Tamron G2 Line are great lenses for their price point.

[@audio2u]

Yeah, for Sony's E-mount, Tamron only does 150-500. I have a work colleague who just bought herself the Sigma 150-600 in E-mount because she wanted the extra reach, but omfg, the weight difference is phenomenal. Glad I bought the Tamron!😃

[@Ichijoe2112]

@@audio2u Having dug into this before having settled on my choice. Had the Sigma 150-600 Sport actually been available, I'd have gone for that. Given that it has better weather protection than the Contemporary version. The Tamron was IIRC about 400€ more than the Sigma C 150-600, but unlike the Sigma, the Tamron has the better sealing. So that's how my decision fell. This said I think this Lense will force me to step up the deficiencies in my skill level. Mores the potty that it's starts at f6.3 @ 600mm so that makes shooting kind of a challenge on the 5DmkII, between setting the correct ISO, and a more desirable Shutter speed.

[@audio2u]

Oh man, have I got an eye-opening blog post for YOU to read! Seriously, it has changed my thinking on how to shoot BIG TIME. This is a four part essay, written by good friend and occasional Shutters Inc co-host, Joe Edelman. It'll take you about half an hour to read it all, and I SERIOUSLY recommend that you DO read it all! www.joeedelman.com/what-the-exposure-triangle-and-brooklyn-bridge-have-in-common

[@stevesvids]

I knew all of this but still every time I open a raw in darkroom I am disappointed that my Fuji film sim does not replicate.

[@audio2u]

Embrace the challenge! 😃

[@stevesvids]

@@audio2u I do... 😊👊👍 Thanks for all your vids. I find them extremely helpful. 🙂👍👋

[@LuisAFlorit]

No. Cameras do not process RAWs prior to showing it. They all show the JPG thumbnail inside the RAW (that's why that is fast). This small JPG is generated when the image is taken (that's why that is slow).

[@emrg777]

Sensor captures raw data that is only light values. There is no preview unless you apply the necessary steps to give you RGB data and the subsequent image so I'm not quite certain what you mean. In fact I think some cameras embed a full size or near full size preview....

[@sebas11tian]

That is exactly what he meant. Glad you caught on

[@audio2u]

I wasn't suggesting that they process the raw data every time you want to look at your image in-camera. It's done once, at the time of capture, and that jpeg is what you see when you hit the review button.

[@TonySkraba]

No. That small JPG, or thumbnail (or regular JPG if you saved that in camera) HAS TO BE created, and that comes from processing the RAW. So yes they most certainly DO process RAW to display images. - in fact, cameras that dont save RAW (old phones) do that too, they do process RAW data and then save only JPG.

[@Ichijoe2112]

I get that this channel caters to noobies, hell I'm pretty noobish myself. But, I'd have thought this was more or less understood. Hence why it's called a Raw (as in uncooked) File. I mean do any of the other usual suspects treat these Images any better? Or, do the JPEG previews turn to 💩 in Lightroom too?

[@audio2u]

You might be surprised at how many people don't understand these basics. That's why I try not to make assumptions, and cover what might seem mundane and obvious to some viewers. "Leave no-one behind".

[@audio2u]

I can't speak for Lightroom. I haven't looked at it since 2018.

[@Ichijoe2112]

@@audio2u Yeah I know, I'm just a hobbiist at this point Adobe is (to me), a bit of a pipedream, that someday I'd be good enough to warrant it's commercial use. (As to it's proper use in making money, instead of just giving it out. That said (not having used it either), I find it difficult to fathom (Outside of Adobe's AI), that it would produce better results than Darktable upon just opening a Raw File.

[@audio2u]

As someone else said, Adobe Lightroom actually presents your RAW file in an "already-processed-to-look-like-the-camera-jpeg" starting point. I don't like that idea AT ALL.

[@mredben]

You get a little lost with the actual statement and tip in your video. It's all a little too long and it's a little unclear what you're trying to tell us. B.t.w. the Tamron 150-500mm is a top zoom lens.

[@audio2u]

In what way did I get lost? Happy to be corrected if I got something wrong..

[@mredben]

@@audio2u The problem with almost all Darktable tutorials is that a lot of side information is mentioned in addition to the actual tip. Darktable is modular and versatile. At the same time, however, it is also very confusing. Partly illogical from the overall concept. This confuses the new user anyway. - For learning, it's better if you only see and hear the things that belong to the topic and lead to the solution and the goal.

[@audio2u]

Sorry you feel that this was off-topic. I thought it was actually relevant. Each to their own, I guess! :)