One of the most powerful features of Adobe Photoshop is the ability to restrict adjustments to specific regions of an image using masks. The simplest masks are based on gradients—linear, radial, or reflected—which allow broad spatial control over where an adjustment is applied.
More advanced masks, however, can be defined directly from image data. These include luminosity masks, as well as color range masks, which are the focus of this article.
Photoshop does provide a Color Range selection tool, but it has several limitations: it can be slow to use, has some interesting quirks in terms of defining range, and most importantly, it produces masks with only 8-bit precision (selections, and masks made from them, only have 8 bit precision). By contrast, operations such as Image Calculations and Apply Image work in 16-bit space. For the purposes of this article, we will assume a 16-bit workflow and explore how to construct high-quality color range masks directly from RGB channel math.
Image Calculations and Apply Image can be used to add, subtract, invert, multiply, and divide color channel information contained in the R, G, and B channels. While at first these methods appear daunting, once learned and practiced a bit they can become second nature. As this series of articles is aimed at advanced Photoshop users, a tutorial on these tools is, respectively, out of scope (there are many YouTube videos available on this topic).
Subractive Color Selections
So using our Desmos hue strip, let’s explore a few simple color operations.
First, suppose I want to select warm colors. Such a selection should be centered somewhere around orange, a hue of 30 degrees. And of course, orange is half way between red and yellow. Since I only have R, G, and B channels to play with, let’s look at our Desmos hue strip and figure out how to define red and yellow. Well, red I get for free, it’s simply the R curve/channel. Yellow (Hue = 60), on the other hand, is most clearly defined as where blue reaches its minimum value. So what happens if I subtract the blue channel from the red channel (R-B) In other words, I’m saying, find pixels where there’s a high red value and a low blue value. High red means close to a red hue, whereas low blue value typical means close to a yellow hue. More simply put, red means red, and LOW-blue means yellow.
Trying that simple R-B calculation in Desmos does indeed select warms, and it does it very nicely and fairly smoothly. In fact you can see in the resulting curve that it peaks exactly at 30 degrees, and crosses zero, into negative values, at precisely 300 degrees and 120 degrees. This simple calculation is actually quite useful as is, especially since “warms” is such an important color range in photography. Importantly, the negative values of that calculated R-B curve become complete black in our Photoshop mask, as Photoshop clips those values to 0 luminosity.
And of course, cools are equally important and they can readily be calculated/masked using the opposite approach: B-R.
So Let’s see what these look like in a photoshop hue wheel, which I’ve created using a constant chroma base, with a radially fading brightness level. I do this because one can be easily fooled when doing analysis at a single brightness level (so many effects can depend on the brightness!). In this particular case, however, we can see that our R-B and B-R selectors are in fact brightness independent.
For any of these masks, their strength (extent of opacity, or how much white there is in the mask) will depend on the underlying image. So the best practice is to normalize the mask to encompass the full dynamic range from black to white. This is done by applying Levels—>auto (auto settings:Find Dark & Light Colors) to the layer mask or channel itself.
I was pretty excited when I figured this out, as I previously had no mechanism for 16 bit masking based on hue ranges. Applying these warms/cools selections does work well on actual images. But I was bothered by something I was seeing when applying these selections to real images. When I looked at real image versions of these masks, they seemed just a little too abrupt at their transitions from warms (300 to 120) to not-warms. I’m a bit obsessive about smoothness in PS adjustments (and masks), and despite no glaring issues in applications of these masks, I wanted a better solution with perfectly smooth transitions.
But this is a familiar problem in Photoshop, and the solution is simple. Recall, in our creation of luminosity masks, we could compress their ranges by applying a gamma correction, which represents either a full (eg g=0.5) or partial multiplication (eg 0.75) of our starting mask depending on the gamma value (<1).
or in this case
Here you can see how a g=0.5 compression slight narrows the range of the mask, while also smoothing the “feet” of the mask toward zero opacity, a natural consequence of a gamma compression.
This is of course very straightforward in Photoshop, using the Levels adjustment applied directly to the layer mask. The beauty of this is one can get live feedback on what’s actually being selected in the image, and there’s two ways to do this. You can either view the monochrome mask directly and decide what your selection will be, or, if you’ve already got an adjustment applied with that layer, you can view how the Levels gamma adjustment affects the appearance of your adjustment in full color.
To create this warms mask, your image calculations dialogue should be set up like this:
- Source 1: your document
- Layer: Merged (or background)
- Channel: Blue
- Source 2: your document
- Layer: Merged (or background)
- Channel: Red
- Blending: Subtract
- Offset: 128
- Result: New Channel
Note: order is important in the Image Calculations Dialogue: in this case we put the Blue Channel on the top to make sure it is Subtracted from the Red Channel. Also, image Calculations won’t recognize Merged if there is only a single background layer
We’ve talked so far about creating “warms” (R-B —> normalize—> gamma) and “cools”(B-R—>normalize—> gamma) masks. There’s one more frequently useful “foliage” mask, which is G-B. Most of what we see as green leaves in landscapes are actually closer to yellow/green (hue of 90 degrees) than to pure green at 120 degrees, so G-B—>normalize—>gamma actually works quite well for selecting foliage.
In my own processing, I’ve rarely needed more than these 3 landscape-friendly masks, but then again why stop there in our analysis. What if I wanted to dial in a hue value directly? What if I could go beyond simple subtraction and actually get a weighted combination of each R, G, and B channel? This may be possible with image calculations (I haven’t tried but suspect it would be very cumbersome). But there’s a much simpler approach, which brings us to the Channel Mixer adjustment layer. Specifically, channel mixer in monochrome mode, where I can assign a monochrome brightness value to each R, G, and B channel.
As usual, the 0-360 range of all possible hues points me to a cosines approach as a first attempt. So what if I set my relative R, G, and B CM/monochrome values by the following?
As you can see from Desmos (with v=0 and including an offset), that gives me a nice smooth curve with a peak at hue = 0 (pure red).
Let’s plug in v = 0. That gives me values of 1 for the R weight, -0.5 for the G weight, and -0.5 for the B weight. Since Photoshop uses percentage ranges of 0 to 100 in channel mixer that means I can use monochrome Channel Mixer to select reds by using values of 100R, -50G, -50B.
What if I want to select pure cyan, a hue of 180 degrees? Plugging that into my cosine weighted formula, I get values of -1, 0.5, 0.5 (or -100, 50, 50 in Photoshop CM terms). And that gives us a beautiful selection/mask that peaks at cyan. We can now select any hue range precisely using this formula!
For reference, and action building, here’s a full table of cosine-derived R, G, B channel mixer monotone values defined at 30 degree hue increments.
| Hue Angle | Red (%) | Green (%) | Blue (%) |
|---|---|---|---|
| 0° | 100 | -50 | -50 |
| 30° | 87 | 0 | -87 |
| 60° | 50 | 50 | -100 |
| 90° | 0 | 87 | -87 |
| 120° | -50 | 100 | -50 |
| 150° | -87 | 87 | 0 |
| 180° | -100 | 50 | 50 |
| 210° | -87 | 0 | 87 |
| 240° | -50 | -50 | 100 |
| 270° | 0 | -87 | 87 |
| 300° | 50 | -100 | 50 |
| 330° | 87 | -87 | 0 |
What about our warms and cools masks we made earlier? Recall that the R-B warms mask peaked at 30 degrees. Plugging that value in gives us R = 0.87, G=0, B = -0.87. Aside from a difference in magnitude (which will disappear upon normalization with Levels/auto), that is identical to our original R-B warms mask. In other words, R-B for warms (hue peak = 30), B-R for cools (hue peak = 210), and G-B for foliage (hue peak = 90), are just special cases of this cosine R/G/B weighting system where one of the channels just happens to get assigned a weight of zero.
So to make a mask with this approach, we set our Channel Mixer monochrome slider values (using our cosine weights), and while that layer is active (you should see a monochrome image where your target color is the brightest) click on the layer mask of your target adjustment layer, then run apply image (RGB, blending mode normal). Done! Then, I’ll typically also apply the levels normalize and gamma compression with levels applied to the layer mask itself. And then finally you can either delete or hide the CM monotone layer. As usual, my practice is to build all these steps into a set of actions and fill my panel (see last paragraph) with a set of actions for selections centered on various hues (30 degree increments seems to give me fine enough sampling for most images). This is a nice improvement for example, over the use of the Selective Color adjustment, whose ranges are restricted to primary and secondary colors (RYGCBM).
I should emphasize that while these masks are very useful, their biggest strength, being brightness independent, and (mostly) saturation independent, can also be somewhat of a liability: it can often look unnatural to make an adjustment masked to a hue range without any tonal dependence. So my advice is to use these masks as a base for color range selection, then modify them using other types of masks such as luminosity masks (one can intersect a color range mask with a separate luminosity mask (eg midtones) by using Apply Image/multiply, for example).
These subtractive color masks are excellent for targeted work, but in many cases too narrow for the kind of expressive color adjustment I’m looking for.
Multiplicative selections
So far we’ve only explored subtractive color selections, which while very powerful, have no built in tonal dependence. A multiplicative variant of R-B is R*(1-B). R is simply the red channel. 1-B is the inverse of the blue channel. And as discussed above, the inverse of blue is yellow. And now, if we multiply the inverse of one channel by another channel, we have a tonal dependence built in. The red channel gets stronger at higher R values, which means it’s correlated with overall brightness. Same for the blue channel, so that means its inverse (1-B) is actually darks biased. As discussed in a prior article, the classic midtones M1 mask is defined as L*(1-L), where L is luminosity. So, extending this logic, multiplying the red channel by the inverse of the blue channel R*(1-B) should be a midtone-biased warms selection. Let’s compare this multiplicative selection to our original R-B selection. You’ll see that, at least for the midtones (where the hue strip is currently set) these selections have some similarities and differences. Both peak at 30 degrees, and both reach their minimum at 210 degrees. However, the R*(1-B) selection is significantly broader and doesn’t completely select against the blues.
But there’s another twist. What if I shift my modeled hue strip toward highlights? The peak of the R*(1-B) selection shifts rightward toward yellows. On the other hand, if I shift my hue strip to darker values, now my peak shifts toward the reds. Also note that the amplitude of the selection gets smaller in both the highlights and the shadows, consistent with the midtone bias we expect from multiplying one channel by the inverse of another.
Let’s take at what this selection looks like in a photoshop hue wheel.
As you can see the new R*(1-B) mask is quite bit broader in scope than the original R-B subtractive selection above. I also applied a bit of gamma compression to the mask using Levels to narrow its scope somewhat.
At first I was put off by these peak shifts in shadows and highlights, but I’ve come to embrace them: in a typical landscape image, because colors shift cooler in shadows, the red shadow bias (red is cooler – closer to blue – than orange) is actually a good thing. In many natural scenes, warm-colored surfaces tend to shift from red in the shadows to yellow/orange in highlights. This is not an intrinsic property of the colors themselves, but rather a consequence of illumination and reflectance. In other words, this new R*(1-B) warms selection seems to nicely follow a typical warms color trajectory from reddish shadows to yellowish highlights, with the strong bonus that it is midtone biased. For these reasons, it has now become my go-to warms selection method (notably, this exact selection can also be done with Blendif, with a small caveat, but we’ll address that in a future article). And of course there’s also a cools multiplicative mask (B*(1-R)) and a yellow-greens (G*(1-B) multiplicative mask.
Unlike with the subtractive (R-B, cosine weighted, etc.) masks, which we can control very precisely with monotone channel mixer and a cosine-weighted mix of values, with multiplicative masks we’re restricted to 6 combinations , at least for midtones, with the R*(1-B), B*(1-R), and G*(1-B) being the most generally useful for warms, cools, and foliage, respectively.
| Peak Hue | Multiplicative Selection |
|---|---|
| 30° | R × (1 − B) |
| 90° | G × (1 − B) |
| 150° | G × (1 − R) |
| 210° | B × (1 − R) |
| 270° | B × (1 − G) |
| 330° | R × (1 − G) |
If we’re being extra ambitious, we can also multiply to generate 3 highlights and 3 shadows masks by adding another multiplier. For example, a yellows highlight masks is R*G*(1-B). Because we’re multiplying two lights (R and G) by one dark (1-B), we essentially get a color biased highlights mask similar to a classic M1L2 mask.
As I’ve said a few times, all of these masks are best defined by a series of actions. As I like the midtones bias of the multiplicative masks, I keep a series of actions handy for warms (R*(1-B)), cools (B*(1-R)), and foliage (G*(1-B)).
Final thought: A Plug for a Plug-in
I build tons of actions for various purposes: luminosity masks, color range masks, color grading, and so forth. The actions panel in Photoshop can become bloated pretty quickly, although I gather there are ways to organize them, including the useful “button mode”.
But I solved this problem a different way a few years ago by purchasing Configurator Reloaded. This remarkable plugin enables drag and drop panel building, multiple workspaces, and button based access of tools, actions, adjustment layers, and so forth. That single plugin purchase has changed my life, and is without doubt the most useful Photoshop Plugin I’ve purchased. If you build many actions, or think you might start, I highly recommend making the investment. It is truly superb. Please note I have no relationship with Configurator Reloaded and derive no financial benefit from recommending it.
Leave a Reply