Why does keying only fix the part of your subject that's actually on the screen?
Because a key can only hand back transparency where the key colour was what stood behind the edge, and it has one colour to work with. Shot of a figure standing with its legs on a mat: the wall keys perfectly, 0.00 per cent of it surviving, and still 76.6 per cent of everything left standing is floor — 74,589 pixels of mat against 22,387 of subject. A grey mat sits 0.503 from #00B140 and would need similarity 110 on a control that ends at 100. And the part of the outline crossing the mat gets no transparency at all: alpha 1.000 where the true coverage is 0.500, so the mat's colour is welded into the subject's edge at full strength.
The key is not the part that failed
Take the usual studio arrangement — a green wall that stops at some height, a floor that is not green, and someone standing with their legs on it — and the key itself comes out clean. Not a trace of the wall survives: 0.00 per cent of it is left behind at the default settings, with the usual gentle falloff across it and a contact shadow at the floor line.
What comes out the other end is the surprise. The subject occupies 22,387 pixels and the floor 74,589, and every one of those floor pixels is still fully opaque. Add it up and 76.6 per cent of what is standing after a perfect key is floor. The tool did exactly what it promises; the shot simply contained two backgrounds and only one of them was the key colour.
Why the slider cannot reach it
Distance from the key decides everything, and a floor is nowhere near it. A grey exercise mat measures 0.503 from #00B140, a blue one 0.535, a tan doormat 0.564 and black foam 0.625. To have the mat even begin to move, similarity would have to reach 110, 117, 123 and 135 respectively. The control ends at 100.
Push it there anyway and you can see the shape of the trade. Nothing happens through 42, 63, 80 or 90 — the mat sits at full opacity. At 95 it finally drops to 0.884 and the subject drops with it, to 0.904. At 100 the mat is down to 0.555, still more than half there, and the subject has fallen to 0.701. You are buying floor with body.
The edge is where it actually hurts
The body of the subject is fine wherever it stands, because those pixels show the subject and not what is behind it. The damage is on the outline, where anti-aliased pixels are a real blend of subject and whatever was behind. Over the wall, those pixels blend subject with green and the keyer reads them correctly enough: mean coverage 0.604, alpha 0.716 against the 0.396 it should be — wrong by 0.320, which is the ordinary cost of a soft edge.
Over the mat the same pixels blend subject with floor, and neither half is anywhere near the key, so nothing moves them. Mean coverage 0.500, alpha 1.000 against the 0.500 it should be: not a soft edge, no edge at all, and the mat's colour carried into the subject's outline at full strength. Composited over a dark background that costs 17.4 levels of error, over mid grey 73.5, and over a light one 167.7 — against 28.4, 32.1 and 64.2 for the part of the same outline standing in front of the wall.
Keying the mat instead trades the problem
The obvious move is to key the floor's colour rather than the wall's. It works, in the sense that the floor goes: with the grey mat as the key, 0.00 per cent of it survives. It also takes the subject with it. Dark trousers and shoes sit 0.104 from that grey, comfortably inside the window, and mean alpha over the subject collapses to 0.280. And the entire wall comes back — 100.00 per cent of it — because there is only one key colour at a time.
On the other mats it does not even do that. Keyed on blue, tan or black foam, the floor is left at 0.997, 1.000 and 0.993, because a real floor has a gradient and a shadow under the subject and only a narrow band of it is ever exactly the value you picked — while the subject still pays, at 0.506, 0.688 and 0.656. Two passes would be needed and this tool does one colour per run.
Neither of the other controls touches it
Spill suppression is a green-channel limiter. It acts only where green is already the strongest channel, and on a grey or blue floor green is not, so it has nothing to bite on. It is the right tool for a green fringe on the wall and the wrong one for a floor.
The AI overlay cannot help either, and for a structural reason rather than a tuning one. It combines as a union, taking whichever of the two alphas is greater, so it can only make pixels more opaque. The floor is not part of the subject, so its alpha there is zero and the union adds nothing: measured with an ideal subject mask, the floor is still 100.00 per cent present and the subject unchanged at 1.000. Nothing in this tool subtracts.
What does work
Run the green onto the floor. The moment the floor is the key colour, the whole problem disappears and nothing else has to give: 0.02 per cent of it survives, it accounts for 0.1 per cent of what is left standing, and the subject is untouched at alpha 1.000. A sweep of green material from the wall across the floor, or a floor painted for the job, is worth more than any amount of slider work afterwards.
Failing that, get the subject off the floor — stand them on a low green-faced rostrum, or shoot them clear of the ground entirely — or accept that the bottom of the frame is a hand-mask. The contact shadow makes no difference either way: with it removed the floor is still 100 per cent present, so there is no version of this where shading saves you.
How to catch it before you ship
Look at the alpha, not at the composite. Three quarters of the opaque area being floor is obvious the moment you view the channel, and almost invisible once it is sitting over a background that happens to be dark, where the edge error is only 17.4 levels. The same cut over a light background is 167.7.
So check the cut against mid grey rather than against the background you intend to use, and walk the outline — top, sides, and specifically whatever crosses the floor line. Any stretch of edge that is hard where the rest is soft is a stretch where the key colour was not behind it, and no setting will convert it into transparency afterwards.
Related questions
Can I remove a non-green floor with the similarity slider?
No. A grey mat sits 0.503 from #00B140 and would need similarity 110 on a control that ends at 100; blue needs 117, tan 123, black foam 135. At similarity 100 the mat is still 0.555 opaque and the subject has dropped to 0.701 — you are buying floor with body.
Can I key the floor's colour instead of the screen?
Only if nothing in the subject shares it. Keyed on a grey mat the floor goes completely, but dark trousers and shoes are 0.104 from that grey and mean alpha over the subject falls to 0.280, while 100 per cent of the wall comes back. On blue, tan or black mats the floor does not go at all, because its own gradient and shadow keep most pixels off the sampled value.
Does AI subject mode get rid of the floor?
No. It combines as a union, taking the greater of the two alphas, so it can only add opacity. The floor is outside the subject, its alpha there is zero, and measured with an ideal subject mask the floor is still 100.00 per cent present. Nothing in this tool subtracts opacity.
What is the actual fix?
Make the floor the key colour. With green run onto the floor, 0.02 per cent of it survives, it is 0.1 per cent of what stays, and the subject is untouched at 1.000. Otherwise lift the subject clear of the ground or hand-mask the bottom — the contact shadow makes no difference, so there is no version where shading rescues it.