How do you key something semi-transparent without leaving the green stuck inside it?
A semi-transparent pixel carries two numbers - how much of the backdrop reaches the camera through it, and how much light the subject sends of its own - and a chroma key produces only one, so it cannot tell the two apart. Measured on costume wings whose transmittance runs from 0.40 at the veins to 0.98 at the tips, the default key deletes 46.95 per cent of the wing outright and leaves the surviving 23 per cent stored as (13.5, 91.0, 44.8) where the wing's own light is (13.5, 12.2, 10.2), which is 78 code levels of borrowed green; over a white background that composites 65.7 code levels RMS away from correct. Shoot the same frame twice against two different backdrops and the two unknowns separate, because two frames give six equations for four unknowns: the same wings then come out at 0.84.
A semi-transparent pixel carries two numbers, and a key gives you one
Put anything in front of a uniform backdrop S and the camera reads C = k x S + R. The first term is k, how much of the backdrop reaches the lens: zero for a solid object, one for bare backdrop, and somewhere in between for anything you can see through. The second is R, the light the subject sends of its own. Every correct composite depends on both, and they are independent - a dim thing that blocks a lot looks nothing like a bright thing that blocks a little.
The wings rendered for this piece transmit 0.40 at the veins and 0.98 at the tips, averaging 0.742, and reflect a further 0.0285 of the screen off their surface, so k averages 0.771 across 60,945 pixels of wing. Three colour channels give you three equations. The unknowns are k plus the three components of R, four in total. One frame is one equation short, and no amount of slider work supplies the missing one.
The keyer reads one magnitude, and the magnitude mixes the two
What a chroma key actually measures is the distance from the pixel to the key colour. Here that is the length of C minus S, which works out to the length of R minus (1 - k) times S. It is a single number that depends on both unknowns at once, so a pixel passing nine tenths of the screen but glowing faintly can land in exactly the same place as a pixel passing nothing at all.
Run the default 42 similarity and 14 smoothness over the wings and the average alpha comes out at 0.2019 against a true occlusion of 0.2312. Nearly right on average, hopelessly wrong per pixel: the mean absolute error is 0.1104. The thin end collapses first. Just under half the wing, 46.95 per cent, is deleted outright, 76.87 per cent falls below half alpha, and only 23.13 per cent survives at all.
The part that survives is opaque, and it is green
Nothing in the frame tells you it happened. The bare screen leaves 0.00 per cent residue and the opaque body loses not one pixel, so the picture looks like a clean key with a slightly ragged wing. The pixels that did survive carry the screen inside their colour: averaged over them, the file stores (13.5, 91.0, 44.8) where the wing's own light is (13.5, 12.2, 10.2).
Notice the red channel is identical on both sides. The preset green #00B140 is zero in red, so none of the contamination can land there; all of it went into green, with a smaller share into blue. Measured as green above whichever of red and blue is higher, the surviving wing sits at 46.2 code levels with the default spill suppression and 79.0 with it off, against minus 1.3 for the wing by itself. Those pixels are fully opaque as well, so the new background never gets a look in.
Source-over is the wrong blend, even with a perfect alpha
This tool composites by drawing the keyed layer over the new background, which is source-over: out equals alpha times C plus one minus alpha times B. The physically correct result for a transmitting subject is out equals k times B plus R. Put the two side by side and the gap between them is k times ((1 - k) times S minus R). That is not a rounding error, it is a different model of what an edge is.
Measured over the wing against a near-black board, the tool's composite is off by 42.1 code levels RMS, against mid grey 29.2, against white 65.7. Hand the key a mathematically perfect alpha - every pixel exactly one minus k, which no keyer manages - and it barely helps: 35.8, 19.3 and 36.8. A flawless matte still leaves the green inside, because the blend never lets the new background through the wing.
Spill suppression takes the green out and leaves the problem
Spill suppression here clamps the green channel down towards whichever of red and blue is higher. Pushed to 100 it does exactly what it promises: green excess falls from 79.0 with it off, to 46.2 at the default 35, to 0.0 with both sliders at 100. The deletion does not move at any of those settings - 46.95 per cent of the wing is gone in all three.
The composite barely improves, and in one direction it gets worse: 49.8, 33.7 and 63.6 code levels with suppression off, 42.1, 29.2 and 65.7 at the default, and 35.5, 30.7 and 72.1 at maximum. Better on black, worse on white. The wing is still opaque and now too dark, because making it grey is not the same as making it transparent. Tolerance is no escape either: at similarity 20 nothing is deleted and the white-board error is 248.0, at 60 the whole wing is gone, and smoothness at 40 stops the deletion entirely while still landing at 64.6.
Two backdrops turn one equation into two
Shoot the same frame twice, changing only what sits behind the subject, and every pixel hands over six equations for four unknowns. Subtract one frame from the other and the R cancels: k is the dot product of (C1 - C2) with (S1 - S2) over the squared length of (S1 - S2), and then R is just C1 minus k times S1. Nothing is guessed, and nothing depends on how green the screen was.
With a second take over blue #0047BB the recovered k is within 0.0021 of truth and the composite errors fall to 1.44, 0.74 and 0.84 code levels, against 65.7 for the single-frame route. Switching the screen lights off for the second take works just as well, since that take is R on its own: 0.91, 0.71 and 0.86. What matters is that the two backdrops really differ. A second green half a stop down separates them by only 0.109 and leaves 5.31 code levels, five times worse than blue.
The sheen is the small term, and the one you have to decide about
Screen light reaches a wing three ways: straight through it, off its surface, and - in a wraparound set - as diffuse bounce. The solve folds all three into the single k, because each one is a scalar multiple of the same S. Measured here the specular part averages 0.0285 and peaks at 0.0601, and just over half the wing, 51.83 per cent, has its mirror direction pointing past the screen at the room instead. Against a mean transmittance of 0.742, the sheen carries 3.8 per cent as much of the screen as the light going straight through.
That is why finish changes so little at the dull end and so much at the shiny end: with a Fresnel floor of 0.02 the sheen adds 0.9 code levels of green, at 0.06 it adds 2.7, at 0.15 it adds 6.5 and at 0.40 it adds 17.6. It also decides what k means. Treat all of k as transmittance and you let that much extra new background through the sheen - 6.7 code levels on white for a glossy film, 45.0 for a near-mirror one - which reads as a bright streak where a highlight should be.
Related questions
Can I get there with the tolerance slider instead of a second take?
No. Across the whole range the two failures bracket each other: at similarity 20 nothing is deleted but the wing keeps every scrap of the screen and the white-board error is 248.0 code levels, and by 60 the entire wing is deleted. Nothing in between recovers both the shape and the transmission, because a slider changes one threshold and the problem is a missing equation.
Does the second take have to be a different colour?
It has to be a different radiance, not necessarily a different hue. Blue #0047BB separates the two backdrops by 0.637 and works well; switching the screen lights off for the second take separates them by 0.738 and works even better. The same green dimmed by half a stop separates them by only 0.109, and the error stays five times higher. The camera has to be locked off too: a one-pixel shift between takes costs 7.44 code levels against 0.86 for a perfect match.
Does this tool do the two-pass composite for me?
No. Its background switch draws the keyed layer with a plain canvas draw, which is source-over, and source-over cannot express transmission. Set Background to Transparent, export the keyed PNG or MOV with its alpha, then build out = k times B plus R in the editor: the new background multiplied by the recovered k, with the recovered R added on top.
Is the green inside a semi-transparent object reflection or transmission?
On this wing, overwhelmingly transmission - the reflected part carries 3.8 per cent as much of the screen as the part going straight through. It rarely matters for the removal, because both land in the same unknown and the two-backdrop solve strips them together. It matters afterwards, when you decide how much of the new background to let back through: that slice is transmission if the wing is see-through there, and reflection if it is a highlight.