New to forum, question about virtual backgrounds w/green screen

[sethym]

Hardware: iMac 27" running Sierra. External camera: Logitech Brio 1080p.
I make in-studio videos to post on YouTube, using QuickTime with a high quality green screen, then uploading to iMovie and adding a virtual background.
Finished videos (6-20 minutes in length,) look good, many already posted on YouTube.

Overall sharpness is great, both of the subject (me) and the backgrounds. However, there is some 'feathering' of the edges, when my head or fingers move against certain BG colors, If I'm perfectly still, even the edges of my short grey-white hair meeting the virtual background remain crisp. But when I rotate my head, just a bit of stray pixels at the edges. I don't know if this is camera-limitation-based, lighting-based (my lighting is near-perfect and shadowless against the green screen,) or simply can't be helped and to be expected wherever certain foreground/BG colors meet.

If this Newbie/General is not the best forum category for this type of question, please advise, thanks.

[sum_guy]

Motion is tough with chromakey, high contrast colors work best but as soon as a color is close to bg and then add motion, you will get some pixelation. It sounds like lighting isn't the issue, as far as camera limitation goes, it would be interesting to try another camera, maybe not a webcam, something more dedicated to video. Most webcams are great for close up static video, at least that's been my experience.

Maybe try a GoPro cam?

Welcome to the forums.

[aedipuss]

yeah. just don't move if you want greenscreen to work. even network broadcasts can't handle it with million dollar equipment.

[Cornucopia]

As mentioned, there are some inherent issues.

However, much is due to the compositing app. Something like mocha or after effects will do a much better job than imovie. They have spill supression tools, feathering clamp tools...

Then there is the professional practice of using rim lighting to isolate those edges so they do not have(as much) green color bleed and make it easier to see the difference in the 2 sides of the edge.

Then there is the IMPORTANT part of hardware, where you do NOT want a cam that is outputting a compressed stream, nor one that is using 4:2:0 color subsampling. Even 4:2:2 is not preferable, if you can get a cam that outputs 4:4:4, that is what you want. And that is how pros do it better. Not nearly as blocky or smeared.


Scott

[poisondeathray]

Also reduce motion blur by shooting at a higher shutter speed

This makes edges more distinct during motion , easier to key and roto ( ie. use a different camera if you can, because you probably can't with that one) .

[sum_guy]

if you can get a cam that outputs 4:4:4

If I'm not mistaken, and I could be, I think the GoPro cineform codec is 4:4:4

[Cornucopia]

Good points, both

[jagabo]

Also reduce motion blur by shooting at a higher shutter speed

This makes edges more distinct during motion , easier to key and roto ( ie. use a different camera if you can, because you probably can't with that one) .

I agree. Higher shutter speed, lots of light, a real camera, not a webcam, all to reduce motion blur.

[sethym]

Thanks, guys, these are great answers for a 'newbie.'
Sounds like upgrading to a better camera might be needed (as per Cornucopia and sum_guy.)

I can't find color-compression-stream specs for my Logitech Brio Ultra 4K 1080p webcam. I bought it because it was reasonably-priced ($175) and easy mount/plug & play on top of my iMac 27, and because it was 'claimed' to be the top-performing webcam out there. And, together with Logi's Game Center app (not the useless Camera Settings app that comes with the Webcam) I can get a very sharp-focus, color-balanced image of me sitting/streaming at the computer.

Under specs, this is what they list:
Multiple resolutions
4K/30fps (up to 4096 x 2160 pixels)
1080p/30 or 60 fps (up to 1920 x 1080 pixels)
720p/30, 60, or 90 fps (up to 1280 x 720 pixels)
Camera mega pixel: 13

I'm happy to upgrade to a GoPro model made for Mac plug & play, and easily mountable or 'stand-able' atop my iMac, if the 4:4:4 output (along with 'rim lighting, etc.) will truly eliminate those pixy edges. Model recommendations? (Looks like the Hero10 might be best.)

[sum_guy]

GoPro Hero 10 is the latest greatest. 5.3k / 60 fps, 4K / 120 fps, 2.7K / 240 fps, 1080p / 240 fps, 23-megapixel photos. These will capture to V30 SD Cards, it also has a front and rear display. And if you ever decide to go outside the studio or do any action video it's great for that too. This bundle comes with 2 batteries and every mount you could ever need.

https://www.amazon.com/GoPro-HERO10-Hero-Black-Waterproof/dp/B09G9KHMCT/ref=sr_1_2_ssp...dDbGljaz10cnVl

[Cornucopia]

I use the BRIO a lot in classrooms & conf. rooms. Pretty sure it is either uncompressed YUV (guessing 4:2:0) or lightly compressed MJPEG (then very likely 4:2:0). It is fine for basic stuff, including webcam use. I would doubt it being optimal for GreenScreen work, though.

GoPros are great (I have 2), but I usually wouldn't use them for indoor, permanently-mounted work. You want something that doesn't require batteries, for one thing.

Scott

[sethym]

Thanks Scott. Since all my YouTubes are made indoors, sitting in front of my desktop, with virtual backgrounds added later, do you have a recommendation for a better camera? One that hits the 4:4:4 specs you mentioned, still mountable on or near computer (and not a small fortune?)

I can't use After Effects or Mocha yet, because I'm on an older Mac that cannot run any OS later than Sierra (10.12.) I'll be upgrading once the new iMacs come out in June. Till then, kinda stuck with iMovie...which works pretty good for my simple YouTubes, without the finesse of removing edge-pixelation.

[sum_guy]

Well, all the better video cameras are pretty expensive, I don't know how serious you are about this, if it's going to be a serious hobby, expect to spend a minimum of $1300 USD to the sky's the limit. If it's just a fun thing to do and you don't want to break the bank, do a lot of research before you plunk down hard earned money.

[Cornucopia]

One way to go less than than cost is to get a budget DSLR or mirrorless (one that is known to have clean HDMI out), and run live HDMI out to an HDMI cap card that supports 4:4:4 like some (most?) of the Magewells.

The Nikon D3500 goes for ~$600-650.
The Magewell USB Capture HDMI 4K Plus (which can do 4:4:4 at 4k) goes for $400-450.

So that would be $1000-1100.

There may be ways to get it less than that (e.g. going down from 4k to 1080p60 HD, or 4k30, or even 1080p30). But sum_guy is correct, if you are serious about needing clean greenscreen, you won't be able to do it with a budget only limited to the $200 alloted to something like a BRIO.

MacOS Sierra should be able to support AE. But a new computer will always work better in that respect.


Scott

[poisondeathray]

Under specs, this is what they list:
Multiple resolutions
4K/30fps (up to 4096 x 2160 pixels)
1080p/30 or 60 fps (up to 1920 x 1080 pixels)
720p/30, 60, or 90 fps (up to 1280 x 720 pixels)
Camera mega pixel: 13

No doubt spending on better equipment will help, but did you try the "4K" mode ?

This review mentioned "soft edges" at 1080p, maybe it's similar to the issue you are having
https://www.streamtechreviews.com/blog/brio

At 1080p, using a green screen with the Brio is pretty good, but the soft edges make removing the green from stills a bit inaccurate. Using a green screen at 4K is excellent.

(4K 4:2:0 has essentially the same color resolution as 2K 4:4:4)

There are free versions of better software that can run on a mac - like davinci resolve, hitfilm, blender, natron that should be able do a better job than imovie . If some of them are limited to newer versions of Mac OS, you can download older releases . If you've got the lighting and setup optimized, there are techniques to improve if you 're stuck with what you have in software - multiple keys and compositing is the big one

[sum_guy]

I don't know how ethical this is but Amazon has a pretty liberal return policy, for $550 and a GoPro Hero 10, you have a couple weeks to try it out and if it sucks, return it. I'm not this kind of guy, I like to purchase what I need the first time. If the budget is $600 and not $1300 it's worth a go. I've also tried some of the cheapo Chinese 4k cams with outstanding results, I'm not going to promote one but I have a Zohulu 4k that produces fantastic video and I spent around $250 for it thinking if it falls out of the boat and drowns who cares.

[sethym]

I must say, I'm very happy with this forum...you guys are great!
Within less than 48 hours of registering as a forum member, all of you have provided clear intel into this problem, with much more depth and options (and learning, for me) than I could imagine. Like poisondeathray's comment "...there are techniques to improve what you have in software - 'multiple keys' and 'compositing' is the big one." I'm only a beginner at understanding what 'multiple keys' means and just beginning to grok 'compositing.'

Right now I'm just using a greenscreen with one central talking-head (me) in front of it, and some free downloaded virtual motion-background clips, which aren't perfect but look better than my office walls, for simple educational YouTubes. I currently have about 30 videos up, and still less than 1000 subscribers, so perfection is not essential. Plus, I've only been making these videos for a year or so.

So sticking with the limited hardware (an under-$200 webcam) and software (iMovie & QuickTime only) was a function of simplicity and 'beginners's needs,' not because I can't afford the expensive stuff or dive into the sometimes-steep learning curves of more versatile software. Interest my YouTube channel is growing, and I will be upgrading to a very powerful new Mac this year, so perhaps I'll upgrade my camera and software just for more fun, options and 'perfect' results. My 'educational' channel is not, and will never be, monetized, so diving deeper into video perfection would only be for personal satisfaction with the final look.

Thanks again!

[sethym]

If anyone here is willing to check out a few seconds of two of my YouTube videos, you'll see what I'm confused about, and trying to stabilize:

On this first video I am in front my green screen with good lighting, and everything came out fine. Very sharp facial features, and even the edges of my head where it touches the virtual background look crisp...even when rotating my head quite a bit. You can just watch the first minute:
https://youtu.be/pwxiI3xVN4o

Second video, everything - green screen, distance, lighting, Logitech Brio webcam, software (recorded in QuickTime, imported into iMovie.) - exactly the same. Yet, as I move my head, all kinds of pixelation and 'feathering' is sparking off the edges of my head. Even though there is actually more contrast between my white hair and the darker background in the 2nd than the first. Just watch a few seconds starting at the 30-second mark, and you'll clearly see this.
https://youtu.be/0V8WbSL1OeE

Any thoughts? Any reasons or fixes? Thanks to those of you who checked these vids out.��

[poisondeathray]

Noise is making a significant contributing factor - look at the the foreground difference between the two (right in the center). The 1st is slightly noisy too, but the 2nd is definitely worse. Noise contributes to indistinct edges, low quality macroblocks - that often gives you that flickering edge look when you key . When the erosion of edges is inward, a common cause is reflected green spill. Your keying program is making the inner edges slightly transparent. You can see some evidence of this on the shirt as well

You need to examine the pre-key footage characteristics and alpha channel for the keyed intermediate. If you upload a few seconds of the original recording, that would reveal more information . Denosing prior to keying might be beneficial for you; also recording less compressed or the highest bitrate possible

[sum_guy]

In your capture software, if this is an option, turn off any/all automatic lighting correction. If you can denoise first do it, and definitely avoid compressing too much.

If QT isn't giving you any chromakey options, try capturing with OBS Studio for MAC, then try these settings:

Similarity: 300-400
Smoothness: 20-80
Key Color Spill Reduction: 20-100

[jagabo]

That's some really noisy video. And webcams usually send HD video as an MJPEG stream which would exacerbate the noise from the CMOS sensor. Try shooting some video outside in direct sunlight. Does it have much less noise? If so, invest in brighter lights for indoor shooting. If your webcam has a sharpness setting try turning it down -- that also makes the noise worse.

[sethym]

Thanks Jagabo, I think you are correct that the webcam is really exacerbating it and I need a better camera. The only thing that confuses me is that the edges and noise are fairly clean in the first video, then get really dirty in video #2, even though they were both shot with same webcam, same lighting, etc. just a few days apart, so... I have lots of strong lighting sources hitting both me in the foreground and the green screen, so I don't think it's a need for more light or sunlight (can't take the green screen outside in the direct sunlight anyway.)

Poison's info is very helpful. The pre-key footage for the 2nd video has a lot of noise throughout. I'm new to 'de-noising' options, and QT doesn't offer much, so sum_guy's recommendation of other capture software, plus (eventually) a camera much better than a webcam, might be my best/only options for real improvement here. With all your great suggestions, I'll work on it and update, thank you.

[jagabo]

I don't think it's a need for more light or sunlight (can't take the green screen outside in the direct sunlight anyway.)

That was just to see if the webcam does better (less noise) with very strong lighting. You don't need the green screen to see that. Just compare your indoor video to the outdoor video.

The pre-key footage for the 2nd video has a lot of noise throughout. I'm new to 'de-noising' options

Noise reduction filters may also have the side effect of blurring the video, making the chromakey worse, not better. It's best to avoid the noise from the start -- by shooting properly and/or getting a better camera.