Mathematically Balanced 16-Bit Luminosity Masks

• Basic luminosity masks
• Multiplying masks to target specific tonal regions
• Normalizing masks
• Creating range-balanced masks
• Tuning mask range with Levels gamma compression
• Free downloadable actions: Darks/Shadows/Midtones/Highlights/Lights

Pixel-based layer masks in Photoshop can control both where (spatially) and when (conditionally) an adjustment is applied.  The most common type of “when” mask is a so-called luminosity mask, using the luminosity information inherent in our image to target application of adjustment effects.  For example, if you want to apply a particular adjustment (saturation, brightness, color grade, etc.) to just the highlights of an image, you can add the adjustment layer, and embed a highlights luminosity mask into that adjustment’s layer mask. This is particularly powerful for targeted color grading. The other common use is for targeted dodging and burning.  The strong utility of these methods have led to the development of an active cottage industry of luminosity masking plugins (TK Panel, Lumenzia) and/or action sets available from various sources.

This plot below shows the most basic 3-set series of calculated luminosity masks (darks, midtones, and lights) overlayed on a grayscale gradient (a test image) of brightness from pure black – 0% luminosity – to pure white, 100% luminosity. The blue curves are their calculated opacities as a function of luminosity, and the three strips below show how they would appear as layer masks, used to control where an adjustment effect is observed on the image.

Note I’m showing normalized versions of these, also working with normalized luminosity (which Photoshop does implicitly), so that values on the x axis range from 0 to 1 instead of 0 to 255.

So for example if I wanted to add red to the midtones, I could add a red solid fill layer and add the M1 mask to that layer, forcing application of that red color to be focused in the midtones:

Mask Creation

Luminosity masks are often taught as a sequence of Photoshop operations. But at their core, they are simply mathematical functions of image brightness. Once we recognize that, we can build them more precisely, avoid common limitations, and extend them into more powerful workflows.

If you look for YouTube tutorials or online articles on creating luminosity masks, you’ll frequently encounter approaches that use “selection math”: making a first channel selection and using Photoshop keyboard gymnastics to intersect various selections, also creating new alpha channels (with save selection) in which to store them along the way.  This all works just fine, although it’s important to point out some key limitations. First, every intersection compresses dynamic range. That means each additional mask refinement reduces precision—something most tutorials gloss over.  The second, which greatly exacerbates the first issue, is that Photoshop selections, and any masks created from them, only work in 8 bit resolution. If you’re working in 16 bit resolution, and you should be, you could be throwing away important information with selections, which could potentially lead to artifacts accumulating in your image. To put things into perspective, in 8 bit there are only 256 possible grayscale values in a layer mask, but in 16 bit there are 2¹⁶ = 65,536 possible grayscale values.

As others have pointed out, there is a better way, where we can create 16 bit masks while also avoiding the maddening keyboard gymnastics in the process. Two similar methods Photoshop supplies for 16 bit operations are Image Calculations and Apply Image. This is an important improvement upon the 8 bit limitation of selection math, but also greatly mitigates the first problem of dynamic range loss.

Before we build these masks, it’s worth briefly defining what we mean by luminosity. In Photoshop, luminosity is not simply the average of the red, green, and blue channels. It is also not HSB brightness. Instead, it is a weighted combination that reflects human visual sensitivity, assigning higher luminosity to green and less to blue. Mathematically, this can be written as

Throughout this article, L will refer to this perceived brightness, as computed internally by Photoshop using the rec. 601 standard.

So let’s start by making the three foundational luminosity masks, called L1 (lights 1), D1 (darks 1), and M1 (midtones 1) using Image Calculations.

L1 is simply the luminosity of our image, and can be created with the following image calculations dialogue.

• Source 1:your document
• Layer:Merged
• Channel:Gray
• Source 2:your document
• Layer:Merged
• Channel Gray
• Blending:Normal
• Result:New Channel

_{Note I’ve used merged here for the Layer, as I’m typically creating masks after I’ve already made some other adjustments. Image Calculations won’t recognize Merged if there is only a single background layer, in which case replace Merged with Background. This unfortunate inflexibility of Photoshop will trip up any actions that recorded Merged at this step.}

D1 is the inverse of L1, which means its highest in the blacks and darkest in the whites.  That can easily be created using the same dialogue as L1, but checking the box for Invert.

• Source 1:your document
• Layer:Merged
• Channel:Gray ☑️invert
• Source 2:your document
• Layer:Merged
• Channel: Gray☑️invert
• Blending:Normal
• Result:New Channel

Finally, an M1 mask can be created as the intersection (using multiply) of lights and darks, and can be created with the following image dialogue

• Source 1:your document
• Layer:Merged
• Channel:Gray
• Source 2:your document
• Layer:Merged
• Channel: Gray☑️invert
• Blending:Multiply
• Result:New Channel

_{If we want to go further, we should rename each of these new alpha channels after they are created, as we’ll be using them to create additional masks, and ultimately deleting them (with an action) when we’ve finished processing.}

Mathematically these 3 masks as calculated are:

Let’s look at these initial masks, plotted in Desmos:

So at this point we have a D1 mask that peaks at luminosity of 0, the blacks of our image (R=G=B=0), and an L1 mask that peaks at luminosity of 1, the pure whites of our image (R=G=B=255). Our M1 mask will peak at 50% luminosity, at R=G=B=128 (note that because of the perceived luminosity weights there are actually a large number of RGB values that can give us 50% luminosity).

As you can see, both D1 and L1 give use full dynamic range in a mask from pure black (0% opacity) to pure white (100% opacity).  The peak of the midtones mask is at 50% luminosity (0.5 on our normalized plot), where both D1 and L1 have equal values of 50%.  But since we’re multiplying L1 by D1 that means the midtones mask has reduced dynamic range of opacity.  In fact at its peak it is only 25% (0.5*0.5=0.25=25%).  There is an easy fix for this in Photoshop, which is to apply auto Levels to our midtones M1 after we create it (a mathematically pure method is to move the white point slider to 64, which is 1/4th of 256).  When Levels Auto is set correctly (auto options: Find Dark & Colors), this will now give us a full opacity range from pure black to pure white (with white peaking at 50% luminosity).  By the way, this kind of dynamic range expansion via levels normalization is one reason why 16 bit operations are critical for this work. Mathematically, a correct midtones mask is actually defined as

These 3 masks, while somewhat useful as is, do not always target the ranges we want for color grading or dodging/burning. They are also rather broad in scope.  In fact, we often want shadows or highlights masks that peak somewhere between midtones and darks or between midtones and lights, respectively.

So the next set of masks can be calculated from our existing masks with additional multiplications, which represent intersections of our three basic masks.

Shadows = M1*D1 —> Levels auto —> M1D1 mask

Highlights = M1*L1 —> Levels auto —> M1L1 mask

Here are our current set of masks:

Looking through our 5 modeled masks, we have, from left to right, D1, M1D1, M1, M1L1, and L1. 

While these are good starting masks, spanning our complete tonal range with more than enough overlap, there are a few apparent issues.  First, our M1 mask is significantly broader in scope than our M1D1 and M1L1 masks.  The second is that our M1D1 mask peaks at 0.33, and our M1L1 mask peaks at 0.67.  If we want our 5 masks to be evenly distributed throughout our tonal range, then ideally our shadows mask would peak at 0.25, and our highlights mask would peak at 0.75.

Let’s solve the second problem first. As it turns out, the fix is very simple, we take M1D1 and multiply by D1 again.  That’s called an M1D2 mask (M1*D1*D1), and now we have a peak at precisely 0.25.  Likewise if we create an M1L2 mask (M1*L1*L1) we now get a peak at 0.75.

And now we’re also in a better position to tackle the range problem, which has actually gotten worse, with our further multiplied M1D2 and M1L2 masks being quite a bit narrower in scope than our M1 mask. We can quantify mask ‘scope’ using area under the curve (AUC). Lower AUC = narrower targeting.  Calculating AUC for our current mask set gives me:

AUC(M1D2)=0.47

AUC(M1)=0.67

AUC(M1L2)=0.47

That narrowing of scope, which is actually a favorable property for targeted adjustments, happens whenever we add another multiplication step.  So let’s keep the peak of M1 at the midpoint and try another multiplication, just M1 multiplied by itself, or

M2=M1*M1 

Now we get an area of 0.53, closer to our M1D1 and M1L2 masks.  Let’s try one more multiplication, M1*M1*M1, and we now get 0.46 for our AUC, as close as we can get to the 0.47 AUC of our M1D2 and M1L2 masks.  We can stop there – that’s all we need to make our final midtones mask.  Incidentally our D1 and L1 masks have areas of exactly 0.5, so now all of our masks are very similar in final coverage.

So our final 5 mask series is now:

• D1
• M1D2=M1*D1*D1
• M3=M1*M1*M1
• M1L2=M1*L1*L1
• L1

Here they are all plotted in Desmos:

We’ve created a balanced 5 mask set that are approximately equal in scope and evenly distributed across the tonal range. You can download them here:

All of these masks are pretty straightforward to create using Image Calculations using dialogues similar to the ones I posted above.

For a complete workflow, you’d also need a few actions for viewing each channel, and for either selecting that channel for a dodge/burn effort, or applying that channel to a layer mask (using apply image). 

These 5 masks may be most or all of what you need for general color grading or dodge/burn techniques.  They span the full tonal range, with significant overlap, and with similar scope. 

The range of these masks might be too broad for some precision work.  So let’s address that.  Recall that when we wanted to narrow the scope of our midtones mask (M1), we multiplied it by itself a few times.  With that sequential multiplication approach we can get incremental adjustments/narrowing of the tonal range, but there’s a way to get precision fine tuning of the scope of each mask with live feedback, and that is to use Levels on the mask itself.

If you’ve used Levels, the slider you’ve probably used the most is the gamma slider. The gamma slider in Levels follows the equation:

which is a gamma scaling of the input .  That may look complicated to some, but note for example if we use a gamma of 0.5, that reduces to

which is the same as multiplying a mask by itself.  But the nice thing about gamma is it’s a continuous slider, so we can narrow the scope of our mask in a continuous manner by using gamma < 1, and playing with that slider until we achieve the desired range of our mask.

Here’s a couple of different gamma < 1 compressions (Levels gamma < 1) applied to our original M3 mask:

We could actually go beyond the series of 5 masks we’ve just created, adding higher order multiplications to further push our peaks around (in a very mathematically predictable manner), but that quickly turns into cognitive and action overload (hence the wide availability of plugins that do all the hard work).  Nevertheless, for the ambitious types, here are the multiplications you’d need for a 7-series set of masks that cover the tonal range nicely, and with essentially equal coverage.

These all have AUC values ranging from 0.33 to 0.37, as close as I can get them (and we start to split hairs between the AUC of M1⁵and M1⁶ for instance). By the way, while the peaks of the D^x, M^x, and L^x always remain at 0, 128, and 255, the peaks of the shadows and midtones multiplicative masks can be cleanly predicted with the formulas:

I should note, there’s a useful shortcut that can spare you many of those multiplications in image calculations. And that is to use Levels gamma (g) during mask creation. Recall, Levels gamma applies the formula:

And g is just the inverse of the multiplication power. So for example:

In other words, instead of multiplying D1*D1*D1*D1, which is D⁴, I can just take a copy of D1 and apply Levels g=0.25 to get the same result. Let’s see how we can efficiently get to these 7 masks with a combination of Image Calculations and Levels gamma applications. I’ll assume all of the starting masks (in particular M1) are already Levels-normalized to 0 to 100% opacity.

• D1²: copy D1–> Levels g=1/2=0.5
• M1*D1⁴: copy D1–>Levels g=1/4=0.25–>Multiple by M1–>Levels normalize
• M1³*D1³: Multiply M1*D1–>Levels 1/3=0.33–>Levels normalize
• M1⁶: copy M1–>Levels g=1/6=0.17–>Levels normalize
• M1*L1⁴: copy L1–>Levels g=1/4=0.25–>Multiply by M1–>Levels normalize
• M1³*L1³:Multiple M1*L1–>Levels g=1/3=0.33–>Levels normalize
• L1²:copy L1–> Levels g=1/2=0.5

Even with this shortcut, things are getting quickly out of hand, and obviously more so if we wanted a 9 or 11 mask series. If you need that kind of precision, preferably with live tuning of peaks, compression, and live visual feedback, there is another way that completely avoids channel math with a carefully constructed layer stack, and that will be the subject of a future article.

A few final thoughts. My recommendation when working with these channel masks is to create them all upfront, as the calculations can be a little slow to run.  I personally don’t mind a little wait time for setup before I tackle an image, but having to wait for calculations to run while I’m already knee deep in a processing session is highly detrimental to the creative process (akin to walking into another room and forgetting why you went there).  Also, this particular channel math approach to luminosity masks, in creating new pixel-based alpha channels, can make your saved file grow dramatically in size.  So I’ll always create a cleanup action that deletes all these extra channels (which we should name carefully for this very reason, among others) before finalizing and saving a processed image.

What about application? The most common use for these masks is for dodging and burning techniques. A typical approach is to load a channel as a selection and use it to constrain painting onto a (rasterized) 50% gray layer set to Soft Light or Linear Light (my preferred blend mode, for reasons we’ll explore in a later article). These masks are generally sufficient for that purpose, but there is an important nuance: our shadows and highlights masks are inherently skewed. That skew is a natural consequence of how they’re constructed, and while it provides smooth tonal targeting, it can also introduce subtle contrast compression when used for dodging and burning. In my experience, more symmetric masks tend to produce better results for that workflow.

That same skew, however, is actually a real feature for color grading. Try, for example, adding a red solid fill layer (HSB 0/20/58 or RGB 148/118/118) in Linear Light mode at around 20% fill opacity, and apply the M1L2 highlights mask from our 5 mask series. The result is a smooth, natural color grade that can be further refined by adjusting hue, saturation, or brightness of the solid fill layer. 

Circling back to our luminosity discussion, note that I chose those RGB values (148/118/118) carefully to make them approximately luminosity neutral (L=127), an important consideration when color grading with linear light or soft light blend modes:

Download 5 Mask Actions (Darks, Shadows, Midtones, Highlights, Lights)

• Includes The Following Actions:
  • Create Dodge/Burn Layer in Linear Light Blend Mode
  • Create the 5 Masks
  • Actions to view each of the masks
  • Apply Mask to current adjustment layer
  • Select and Dodge, Select and Burn
    • sets paint brush to 60 or 40% gray, brush must be active – press “B”
  • Delete the 5 Masks (key for preventing file size bloat before saving)