Creating camera input profile with ColorChecker Passport
19 October 2016
Lately I tend to spend considerable amount of time trying to get the colours in my photos accurate. It is not unusual that I revisit my white balance temperature and green tint settings multiple times over a few days, until I am satisfied with the result.
Few months back I came across a video describing how to create a camera input profile with ColorChecker Passport Photo target, Darktable, and Argyll CMS. It seemed to have a potential to give me a better starting point for my photo processing. Unfortunately this video has a few issues:
- It is a bit out of date, for Darktable 1.6.3 while today 2.0.6 is available. The interface has changed a bit.
- The resulting profiles are not always well behaved. See more notes on that further down.
- The author used a CIE file that comes with Argyll CMS. I discovered however that there are slight variations in the ColorChecker target pigments (specifically targets manufactured before or after November 2014). I am not quite sure where the Argyll file came from and how accurately it describes my own target.
I would like to offer my procedure to create a camera input profile with Darktable, Argyll CMS, and ColorChecker 24 patch target.
I have tested this procedure with following software versions, all running on 64-bit Linux Mint:
- Darktable 2.0.6
- Argyll CMS 1.8.3
Overview of the procedure
In order to create a camera input profile, you will need to complete a number of steps:
- Take a target shot in raw format.
- Lookup some details in your CIE file.
- Process your target shot into a TIFF file in Darktable.
- Process your TIFF file with Argyll CMS tools scanin and colprof to create your camera input profile.
- Copy your profile into ~/.config/darktable/color/in/ and restart Darktable.
Taking a target shot
When you read articles about camera input profiles, you will notice that they express many different points of view. Some argue that lay people shouldn't even try making their camera profiles. Since I enjoy tinkering I created one general camera profile for my Nikon D7100 using Wolf Faust IT8 target and following this procedure. And for situations when light is difficult and I would like to reproduce accurate colours I carry ColourChecker Passport with me. Imagine you want to photograph flowers under green canopy - your light will have a very specific spectrum that a profile can help correct. When taking the profiling target shot in such situation make sure that:
- You shoot the target square on.
- Get illumination of the target as even as possible.
- Avoid any glare.
- Shoot in raw.
Analyse your CIE file
In order to process your target shot, you will need two extra files: One that describes layout of your target (CHT file), and another one that describes colours of each patch (CIE file). The good news is that both of them ship with Argyll CMS for many common targets, including ColorChecker. You can create your own CIE file if you really want, here is how.
You can find the CHT and CIE files in Argyll CMS installation directory. On my Linux Mint they are located in /usr/share/color/argyll/ref/.
$ sudo updatedb $ locate ColorChecker.cie ColorChecker.cht
We will need to determine a few things from the CIE file. Open it up in a text editor, for example gedit. The file starts with some headers:
IT8.7/2 ORIGINATOR "Graeme Gill, ArgyllCMS from Gretag Macbeth reference" DESCRIPTOR "ColorChecker 24" CREATED "Feb 18, 2008" MANUFACTURER "X-Rite/Gretag Macbeth"
Then there is a definition of data format. In this case simply a sample id (patch name) and its 3 coordinates in Lab colour space:
NUMBER_OF_FIELDS 4 BEGIN_DATA_FORMAT SAMPLE_ID LAB_L LAB_A LAB_B END_DATA_FORMAT
Sample ids are defined in the CHT file. Their layout for ColorChecker target is this:
And last part of the CIE file is the data itself. Here is a sample:
NUMBER_OF_SETS 24 BEGIN_DATA A01 37.99 13.56 14.06 A02 65.71 18.13 17.81 A03 49.93 -4.88 -21.93 ... D01 96.54 -0.43 1.19 D02 81.26 -0.64 -0.34 D03 66.77 -0.73 -0.50 D04 50.87 -0.15 -0.27 D05 35.66 -0.42 -1.23 D06 20.46 -0.08 -0.97 END_DATA
We need to determine three things:
L value for white patch (D01). In the sample above this is: 96.54
L value for black patch (D06). In the sample above this is: 20.46
Which is the most colour neutral patch with L value > 50? This will be a patch with smallest value of A square plus B square. Let's select grey patches with L > 50 and look at them:
Sample Id L A B A² + B² D01 96.54 -0.43 1.19 1.601 D02 81.26 -0.64 -0.34 0.5252 D03 66.77 -0.73 -0.50 0.7829 D04 50.87 -0.15 -0.27 0.0954
Looking at the numbers the obvious most neutral grey patch is D04.
Processing target shot in Darktable
Let's open the target raw file in Darktable. Looking at my default active modules I can see following:
First of all turn off following modules:
- Highlight reconstruction
- Base curve
Next set following colour profiles to Linear Rec709 RGB. It should make your picture look very dull:
- Input color profile
- Output color profile
Suggestion: If you plan to create profiles for various cameras or lighting situations in the future, I would recommend that you create a new style out of your current history stack. It will save you time and simplify the procedure.
Spot white balance on the most neutral grey patch you discovered in your CIE file. In my case it is patch D04. You want to select reasonable area of that patch for better accuracy, not a single point. I tend to select about 50% of the patch area, keeping well away from its edges.
Setup live samples in color picker on patches D01 and D06:
Activate Exposure module. Increase exposure value so that L value of patch D01 (white) matches as closely as possible value in your CIE file. In my case this was 96.54 and I managed to get the actual reading of the white patch to 96.444.
Activate Base Curve module and double click on the curve. This will turn it into a linear curve. Switch scale to logarithmic. Now drag the black point (lower left corner) to the right until your L value of patch D06 (black) gets as close as possible to value in your CIE file. In my case this was 20.46 and I managed to get the actual reading of the black patch to 20.578.
This shouldn't alter the reading for the white patch by much. If it did, you might need to repeat these two steps. Since my target shot was taken on camera auto settings it turned out rather dark in the screenshot above. Once the white and black points are adjusted, it looks much more natural:
Crop your target shot to include only the ColorChecker target (24 patches) and a small margin. If your shot is not straight, you can apply full keystone perspective correction aiming for the corner markers:
Then crop close to the target edges. Occasionally scanin refuses to read your image. In such case come back to this step and try slightly different crop. I use 3:2 aspect ratio:
Switch to Lighttable. We will need to define specific parameters for export. I would recommend that before you do so, first you save your current export parameters in a new preset. This way you can easily switch back to your normal photography workflow later on.
Select file format TIFF, bit depth 16 bit, and Uncompressed. Profile will be set to Image settings (we have already set output profile to Linear Rec709 RGB earlier on), intent Absolute colorimetric, and style None. You might want to save these settings as another preset. Then export your target shot and close Darktable.
Processing TIFF file with Argyll CMS tools
Next steps need to be done in terminal window. First of all cd into a directory with your TIFF file. We will copy in CIE and CHT files too. On my Linux Mint system I would use:
$ cp /usr/share/color/argyll/ref/ColorChecker.cie . $ cp /usr/share/color/argyll/ref/ColorChecker.cht .
Then run scanin to read your TIFF file and produce appropriate TI3 file. In my case I exported my TIFF file as 161002_0009.tif (I name my photos with YYMMDD prefix):
$ scanin -v -p -G 1.0 -dipn 161002_0009.tif ColorChecker.cht ColorChecker.cie
If everything went well, you should end up with two new files in your directory:
- diag.tif ~ Diagnostic file, it should look like this picture.
- 161002_0009.ti3 ~ Calibration target chart information we will use in the next step.
If this step fails, try adjusting crop and re-export from Darktable. You can also try scaling the image down during export. Documentation for scanin suggests that if your image is over 1200 pixels on a side, you should consider down sampling.
Hopefully everything went fine and you have a TI3 file ready. Now we will create another version of it with two extra synthetic patches. To learn more about this step, please refer to section D5 in this article.
Open your TI3 file in a text editor and change two things:
- Increase value in clause NUMBER_OF_SETS from 24 to 26.
- Add two new lines straight after BEGIN_DATA:
00W 96.4200 100.000 82.4910 100.000 100.000 100.000 0.000000 0.000000 0.000000 00B 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.000000 0.000000 0.000000
I have saved my modified TI3 file under new name, 161002_0009_addbw.ti3. Now we are ready to create our ICC profile. You will probably want to adjust some of the arguments to your liking:
- C - Copyright information. I like to make my intentions clear and state here reference to CC0.
- O - Name of the resulting ICC profile file.
- A - Manufacturer name.
- M - Camera model.
- D - Description you will see in Darktable. Since I am creating a profile for a specific shoot, using a target shot filename is a practical choice for me.
- The last argument is filename of your modified TI3 file, but without its extension.
$ colprof -v -am -qh -u \ -C"https://creativecommons.org/publicdomain/zero/1.0/" \ -O"161002_0009.icc" -A"Nikon" -M"D7100" \ -D"161002_0009" 161002_0009_addbw
Using new ICC profile in Darktable
In order to make your new camera input profile available in Darktable, copy it into appropriate location and restart Darktable:
$ cp 161002_0009.icc ~/.config/darktable/color/in/
Now you should see your new profile in Input color profile module. I use this procedure when the light is difficult. Even though it takes time to create the profile, it can save a lot of time trying to produce accurate colours manually.
Notes on well behaved profiles
I was struggling to understand why should my profiles be well behaved. Here are some notes I received from Elle Stone on this topic:
The "well-behaved" part is actually a side-effect of doing a proper white balance and then normalizing the data by setting the black and white points and adding D50 white and black points, and then making a simple linear gamma camera input profile.
If you don't do a proper white balance, your camera profile "compensates" by adding a slight tint to your images to counterbalance the tint inherent in the ti3 data.
If you don't normalize the data, your camera profile "compensates" by adding positive or negative exposure compensation. So the target chart has the right tonality, but any other scene will look brighter or darker than it would if you had normalized the data.
The way to produce a "not well-behaved" simple linear gamma camera input profile is to not do a proper white balance and/or not normalize the data, though I'm not sure why anyone would want to do this. Though in fact this is what most people do, because most people don't use steps similar to the steps in my tutorial.
My takeaway: It seems that if a profile is used as a camera input profile, then making it well behaved will ensure tonal and exposure neutrality. This is probably not as critical when you are creating a profile to be used for one specific photo shoot and lighting situation - as long as you are happy to adjust your exposure and white balance. On the other hand ensuring that a profile is well behaved is not difficult; simply follow the procedure above.
Please let me know if you notice any issue or inaccuracy in this article.
Article Creating camera input profile with ColorChecker Passport by Tomas Sobek is licensed under a Creative Commons Attribution 4.0 International License.