Realtime Editing for Psychology Experiment

[lholmeb]

Hi guys,

I'm currently helping to run a psychology experiment in which participants watch a video on a screen whilst having their eye movements tracked.

On the experimenter desk, we have one monitor which displays a duplicate of what the participants can see (i.e. the video), and another, where their eye movements are displayed in real time. This is shown as a blank box in the corner of the screen, with a dot which moves around showing their fixation. Both of these are connected to their respective PCs using a VGA cable, as is the larger screen which participants watch.

Basically, we would like to add a third screen which combines these two views. In other words, I would like to - in real time - overlay the box from the eye tracking screen onto the video feed. I imagine it would be necessary to select the background colour of the box to be filtered out, select the box as the area of the screen I would like overlaid, and then stretch it so that it covers the entire area of the video.

Does anyone have any idea how this might be achieved? In a previous but similar setup this was done using a BlackMagic Media Express, as well as two "scan converters", a TVone Cs-320 Connect, by Corio; and a Scan Converter AVT-3170, by AVTool. However, I am unsure if these are still needed, since monitor resolutions have advanced so much in the meantime. I contacted blackmagic support, who pointed us towards an ATEM switcher (https://www.blackmagicdesign.com/uk/products/atem), which they said could "overlay any input over another with the ability to move it, resize it, key it and put borders on it". This sounds promising, but will we need any of the "scan converter" things as well? What do they even do? We are wary of spending so much money for something which might not work

I'm sorry for such basic questions, but media is really not my strong suit. Any information you can offer which might help us would be more than welcome. We're not committed to BlackMagic hardware either - any hardware which can achieve this would be a potential alternative.

Thanks very much for your time!

[Cornucopia]

A "Scan-converter" is basically a hardware device that takes a signal meant for one kind of system and converts it to be compatible with another kind of system, in real time. There are a variety of scan converters out there, mostly PC->NTSC/PAL TV or similar, but there are also those that convert NTSC<->PAL: those are usually referred to as "Standards converters". The rise in prevalence of outputting directly to HDTV-compatible signals from a PC has lessened the need for scan converters overall, but there are still quite a few areas where it is necessary (Medical imaging is a big one, military is another). Scan converters usually include a frame store/buffer and circuitry to adjust framerate, resolution, colorspace and encoding. Some are analog, some digital, some both (thus including A<->D converters as well).

To bring multiple images together and overlay/composite them in real time (including the aforementioned Picture-in-Picture), you need a switcher. There are hardware & software switchers, even ones that do it in real time, but most software switchers have some major lag time (latency), so for Human Feedback/Interaction purposes, it might not be good enough (plus for software, to do this right you need a very beefy computer). Even hardware ones have SOME latency, digital being more than analog, but most hardware switchers are fast enough to appear instantaneous, without any noticeable lag.

Your PC's VGA cable is outputting analog video in RGB colorspace, probably set to show 1024x768 @60FPS/Hz or similar. If all your input & output systems are the same, you don't need SCAN converters, just the frame buffer to help put them into sync with each other to enable compositing (though professionals use a common genlock word clock source to force all of them into time step with each other, negating the need for frame buffers). Again, frame buffers still have some latency (they have to wait to capture a whole frame and be able to hold it and it's neighbors to be able to do any kind of combining). But my guess is that you will need at least one or 2 scan converters.

The ATEM is a switcher with frame buffers (and I assume the remainder of the scan converter circuitry/logic, though from your post it might not be sufficient for your needs). You may not need exactly that (including # of inputs, etc.), but you will need many of those features, and it isn't cheap once you get to that level. In addition to Blackmagic, there is the NewTek TriCaster, and probably something from Grass Valley, too.

The kind of thing that you're talking about with changing background color & resizing is usually known as a DVE (digital video effects) unit, which is part of nearly every decent/large switcher (though not on the smallest ones).

I suggest you check with a "systems integrator" at most A/V resellers as they not only sell the equipment that you're looking for, but make it part of their job to piece together the right combination of equipment to meet your special needs (for an additional fee, of course).
I've done this sort of thing also in the past. They'd need to have a few discussions with your regarding your current equipment (with exact details on models #s, etc.) and your budget and your ultimate intentions & constraints, so they can fine tune the choices of suitable equipment.

Good luck,

Scott

BTW, none of what you've discusses so far falls strictly under what most would call "EDITING". More "compositing". But I don't think it matters as far as the forum topic is concerned.