Erasing background from an image

I have two opaque images –  one with an object and a background, and another with just the background.  Like:

I want to subtract the background from the image so that the alpha blended result is visually identical, but the foreground is as transparent as possible.

E.g:

Desired output (All images under Reuse With Modification license)

I’m sure that this must have been, but I couldn’t find a single correct way of doing this!

I asked a developer from the image editor gimp team, and they replied that the standard way is to create an alpha mask on the front image from the difference between the two images.  i.e. for each pixel in both layers, subtract the rgb values, average that difference between the three channels, and then use that as an alpha.

But this is clearly not correct.  Imagine the foreground has a green piece of semi-transparent glass against a red background.  Just using an alpha mask is clearly not going to subtract the background because you need to actually modify the rgb values in the top layer image to remove all the red.

So what is the correct solution?  Let’s do the calculations.

If we have a solution, the for a solid background with a semi-transparent foreground layer that is alpha blended on top, the final visual color is:

$out_{rgb} = src_{rgb} * src_{alpha} + dst_{rgb} \cdot (1-src_{alpha})$

We want the visual result to be the same, so we know the value of $out_{rgb}$ – that’s our original foreground+background image.  And we know $dst_{rgb}$ – that’s our background image.  We want to now create a new foreground image, $src_{rgb}$, with the maximum value of $src_{alpha}$.

So to restate this again – I want to know how to change the top layer $src$ so that I can have the maximum possible alpha without changing the final visual image at all.  I.e. remove as much of the background as possible from our foreground+background image.

Note that we also have the constraint that for each color channel, that $src_{rgb} \le 1$ since each rgb pixel value is between 0 and 1.  So:

$src_{alpha} \le (out_{rgb} - dst_{rgb})/(1-dst_{rgb})$

So:

$src_{alpha} = Min((out_r - dst_r)/(1-dst_r), out_g - dst_g)/(1-dst_g), out_b - dst_b)/(1-dst_b))\\ src_{rgb} = (dst_{rgb} \cdot (1-src_{alpha}) - out_{rgb})/src_{alpha}$

Proposal

Add an option for the gimp eraser tool to ‘remove layers underneath’, which grabs the rgb value of the layer underneath and applies the formula using the alpha in the brush as a normal erasure would, but bounding the alpha to be no more than the equation above, and modifying the rgb values accordingly.

Result

I showed this to the Gimp team, and they found a way to do this with the latest version in git.  Open the two images as layers.  For the top layer do: Layer->Transparency->Add Alpha Channel.  Select the Clone tool.  On the background layer, ctrl click anywhere to set the Clone source.  In the Clone tool options, choose Default and Color erase, and set alignment to Registered.  Make the size large, select the top layer again, and click on it to erase everything.

Result is:

When the background is a very different color, it works great – the sky was very nicely erased.  But when the colors are too similar, it goes completely wrong.

Overall..  a failure.  But interesting.

One thought on “Erasing background from an image”

1. When you put your high-alpha lion in its place against the background picture, the result is visually identical to the second picture, as demanded.

Comparing the nonalpha lion with the highalpha lion, we see that the highalpha lion has the higher alpha values.

So even though you don’t seem to want a semitransparent lion, it is because you requested production of a foreground image with as much alpha as possible.

Perhaps you should have produced an image with 0 alpha where the difference between the background and second photo exceed some small level of contrast, and 100 percent alpha where the two images are less than or equal to that chosen parameter.

Like