Scanning Negatives in RAW

This is a total rewrite of the original article.

The first approach was to use Mac's Folder Actions, but It's a bit tricky to install and configure, so I decided to change everything to an Automator process.

Updated on November 10th, 2022

I'm working on a hugely improved new Automator scripts and they will be deployed as soon as possible.

Foreword

Scanning negatives can be challenging and very frustrating sometimes, no matter if you’re using a flatbed scanner capable of managing films like the Epson V700 or a dedicated film scanner like the Konica-Minolta Dimage series. Negative films are very annoying to scan and process to obtain a decent final image.

I’ve tried several softwares:

• Minolta DS Scan
• Canon scanning utilities
• Epson scanning utilities
• Silverfast 6 and 8
• Sane
• Vuescan

Some are good at doing some things, bad doing other things, but no software I used before works 100% well with negatives. Vuescan and Silverfast are pretty annoying even having lots of bundled negative film profiles.

Silverfast case:

Silverfast interface is horrible and appears to be made by a 10-year-old programmer. It’s confusing, clumsy and not very practical, not to mention some broken features like the auto framing. It's completely schizophrenic, it NEVER finds frames correctly, no matter if you're using a film holder or not, at least with my Epson V700.

Some other annoying things always drive me crazy, like when you apply the Auto CCR settings and try to copy this adjustment to other frames. It copies the final Auto-CCR calculated adjustments to the other image, and not the command itself. This is extremely stupid and time-consuming.

Even with this, the Negafix feature works reasonably well, but yet very far from perfect. Silverfast scans positive films just in an acceptable way, but the included negative film profiles aren't even close to what I expected from expensive software.

Silverfast is costly and when you buy it, the software license is tied to a specific scanner model. If you have two different scanners, you’ll need to pay for two copies, even if they are plugged to the same computer. This is sick and completely not necessary. I have three scanners and by no way on earth I would pay for three licenses! Wake up Silverfast !!! This is ridiculous...

Vuescan case

Vuescan is competent and but also a very spartan software. It’s not for the beginner and a real pain in the ass to use and do some basic things, but it’s very good with positive films and an excellent choice if you have an old scanner not supported by the maker for newer operational systems. For example, the original Minolta drivers won’t even install on MaxOSX 10.8 and up.

But like I said before, the output from all the above programs are far from perfect, even tweaking the image in a simple way from inside those programs. What the hell, what’s the sense to have to modify parameters like white/black point and R,G,B curves on programs that already have the film profiles ?

The strong point of Vuescan is its universality and compatibility with hundreds of scanners.

Epson and Canon Scan 

Basic but very usable, but they have very outdated user interfaces. Not bad, but at least they are free.

Don’t expect miracles from the scanning software. They promise all sorts of magic tricks like total dust and scratch removal, a fantastic dynamic range and an out of this world mega uber super detail enhancement feature. Even more, to be easy to use. Believe me. This is not the reality. If you want really good scans, be prepared to suffer.

From the scanner makers, for me, the best software is from Canon. It has the best user interface and offers no miracles, besides, maybe Digital Ice, to remove dust and normal scratches. Epson’s software for the V700 is so-so but adequate to produce the raw images we need here.

Remember one IMPORTANT thing: Your scanner will never be able to deliver more than the raw image. Every single thing after this is produced by the software.

My approach:

Use a raw scanner output and process it with external software. BINGO!

Contrary to the popular belief, converting a scanned negative film to a positive image is much more than simply invert the channels. The complicated part is to have the curves adjusted, to compensate for the base film plastic color and to do a decent white balance. Some heavy voodoo is needed.

I tried a very nice shell script made for use with the incredible ImageMagick, called negfix8, from Chris Hunt. Works like a charm, it’s FAST and makes very nice images maybe 90% of the time. Then just post-process the inverted images with the software you prefer.

Please take a look at the Negfix8 project on Github. It's an amazing project. The download link and installation instructions can be found there.

Note that you need both Negfix8 scripts and the ImageMagick image processing software, both free.

Here are some samples:

Original negative 35mm frame scanned in raw

Above: Raw output from the Konica Minolta Dimage Scan Dual IV using Vuescan. This image was not post-processed. Note the strong orange cast and inverted image. Film: Agfa Ultra 50.

Same image converted with Negfix8 script

Above: Default conversion from the raw file with Negfix8. The Negfix8 script can do Autolevels and saturation increasing but I decided to not use on the above example.

Same image processed with Negfix8 and autolevels

Finally, the output from Negfix8 processed for auto levels, auto white balance and +5% saturation. No other software was used.

You can do this directly with ImageMagick or with your preferred software and then tweak later the image to your taste.

But to make use of this, the first thing you will need is to install ImageMagick and be sure that it's running. Unfortunately, there is no simple way to install it and you WILL need to use the terminal and type some commands. Be careful with what you do, be advised.

Important note:

This procedure was tested on Intel Macs and with OSX 10.11 El Capitan, 10.13 High Sierra, 10.14 Mojave and 10.15 Catalina. It should run on any version from 10.9 Mavericks up to 10.15 Catalina, but you have to install the proper XCode version and XCode Command Line Tools on your Mac. You can find both on Apple Developer's site.

It was NOT TESTED on 10.17 and up.

Installing ImageMagick

The less complicated way to do this is to install a port manager like Homebrew or Macports.

To my knowledge, the Homebrew version of ImageMagick is newer (Version 7) than the offered by Macports (Version 6). Note that IM 7 has several advantages over IM 6, including better multicore support.

Depending on your OSX version sometimes Macports may be easyer to install and make IM working.

Here you can find instructions about how to install both. Choose just one.

For Macports, look here.

For Homebrew, look here.

I'll comment here on how to install IM using Macports.

If you succeed to install Macports (it can take a while to download everything needed), the first thing is to check if Macports is working. Open a terminal and type:

port version

followed by the ENTER key.

You should see something like 

Version: 2.6.4

Otherwise, something went wrong and you may need to recheck everything.

If ok, then just type:

port install imagemagick

This WILL take a while. Relax and wait.

Check if IM installation was successful by typing:

convert --version

You should get something like this:

Version: ImageMagick 6.9.11-56 Q16 x86_64 2021-01-13 https://imagemagick.org

Copyright: © 1999-2020 ImageMagick Studio LLC

License: https://imagemagick.org/script/license.php

Features: Cipher DPC Modules 

Delegates (built-in): bzlib djvu fftw fontconfig freetype gslib heic jbig jng jp2 jpeg lcms ltdl lzma openexr png ps raw tiff webp x xml zlib

If you get this, you're almost done.

You will need the following:

Fred's and Chris' scripts

My Automator App for batch conversion

My Automator App for proofing

Please DO NOT modify any of them unless you really know what you're doing.

Unzip the 3 files and put the two Automator Apps wherever you want, for example on the Desktop.

Then put the bash scripts on your HOME directory.

Depending on the OSX version you're using it will be necessary to give permission to execute the scripts. Just allow and you're done. Sometimes you may need to authorize the bash scripts. This is done at the Accessibility and Privacy configurations. You may need to allow the terminal app for full disk access. 

Your computer will need at least 8GB of RAM due to the extensive use of RAMDISK for temporary files. This will speed up a lot the process and avoid excessive use of your HD or SSD. The RAMDISK size is 1GB and should be enough even for medium format scans in 3200 dpi.

IMPORTANT things to remember:

• Scan in RAW, set the scanner mode for POSITIVE TRANSPARENCY or something like.
• No sharpening, no dust removal, nothing !
• Save in TIFF, always. No other format is allowed. Period.
• If using any of the scripts named nf8_xxx , the images MUST HAVE NO BORDERS (unexposed film areas) neither BLACK borders. CROP IT CAREFULLY
• If using Neg2positive, the image crop MUST include some of the unexposed film around the image frame.

If you use Linux or Windows, forget what I said before. You will need to find the IM's binaries for your system and install them by yourself.

My Automator workflows are NOT compatible with Windows or Linux. For Linux experienced users, it should not be difficult to make a decent bash script equivalent to my scripts. For Windows, I don't have a clue about how to do it, sorry.

USE THE SCRIPTS AT YOUR OWN RISK. NO RESPONSABILITY WILL BE ACCEPTED IN ANY CASE OF ACCIDENTAL OR INCIDENTAL DAMAGES TO YOUR FILES CAUSED BY MISUSE OR ANY OTHER CAUSE. BE ADVISED.

Negfix8 and ImageMagick are command-line tools, there is no graphical user interface! You need to open a terminal and execute the commands, for example:

$ ./negfix8 scan.tif result.jpg

Where scan.tif is the input raw file and result.jpg the converted image, in jpg format.

Fast. Good. Cheap.

Fred Weinhaus is what we like to say, an ImageMagick wizard. He wrote tens of very powerful scripts for it, and some of these scripts are perfect to be used along with Negfix8. My picks are:

AUTOCOLOR - automatically color balance and image by shifting the mean value of each channel to the neutral gray value. There are three methods that can be used. The first uses -gamma computed from the neutral gray value and mean value of each of the RGB channels. The second uses -recolor (or -color-matrix) computed from the neutral gray value and mean value of each of the RGB channels. The third skips this step. In addition to correcting the neutral color, the histogram is stretched to full black and white using the clip values.

AUTOWHITE - automatically adjusts the white balance of an image. Two methods are available. Method 1 uses a multiplicative adjustment using -recolor (-color-matrix). Method 2 uses an additive adjustment using -evaluate add. Both methods compute RGB channel averages of a user-specified percentage of pixels closest to white. The channel averages are used in a ratio compared with white in method 1 and as a difference from white in method 2.

AUTOTONE - attempts to automatically tone balance and image. This includes auto brightness/contrast adjustment, auto gray balance, auto white balance, auto gamma correction, auto noise removal and auto sharpening.

AUTOLEVEL - modifies an image to automatically stretch the dynamic range between full black and white and automatically apply a gamma correction. The minimum, maximum and gamma values may be computed from various gray level representations of the image or individually channel-by-channel. The script then passes these values to the IM function -level.

Fred's scripts are in one word, stupendous.

To use Negfix8 along with any of Fred's scripts, run negfix8 on your raw negative image with no extra switches and then run Fred's autowhite (for example) script on the previous result like this

$ ./negfix8 scan.tif result.jpg

$ ./autowhite result.jpg result_awb.jpg

Along with the previously mentioned methods of combining the negfix8 and one of Fred's scripts, it's also possible to use two extremely powerful scripts from Fred.

NEGATIVE2POSITIVE - converts from a (scanned film) negative image to a positive image. Options include: auto stretching the image channels with or without clipping, color filtering and to color balance via white balance, gray balance, both, automatic or via levels. 

Those are "monster" scripts and very complex. I'll not enter into details about how they work, but the key difference between both is that the NEG2POS can use a "blank" scan of the film base to find the proper calibration. 

NEGATIVE2POSITIVE does this automatically but they can deliver different results depending on the parameters set passed to the scripts.

Be aware that Fred's scripts are copyrighted and cannot be used in any "for profit" application without having his permission.

Final thoughts

So, do yourself a favor and try Vuescan or Sane and post-process the raw images with Negfix8 and the photo retouching/adjust software of your choice. My choices are DxO, followed by RawTherapee, but it’s up to you. RawPower also works great.

Comments are welcome!

P.S.

If you're not interested, don't know, use Windows or for any other reason are unable to try the way I told using ImageMagick, you still have some other decent ways to convert C41 images scanned as positives by using:

a) RawTherapee - If you scan or photograph in RAW, there is an option to convert to a positive image. Works well IF you know what you're doing. RawTherapee is freeware and there are binaries for Windows, Mac and Linux. Not recommended for beginners.

b) Silkypix Pro 10 / Silkypix JPEG Image 10: Version 10 can convert C41 negatives to positives but you may will have to compensate for color casts. Not exactly a cheap program and not the best results, but better than nothing.

c) Darktable has a very comprehensive negative to positive conversion tool, but you will need to tweak the result for color casts and tone curve, but the conversion can be very good. It's freeware.