German university makes objects disappear in live video

[roma_turok]

Link

[SLK001]

Interesting video... I would like to have a AVISynth filter to play with. I am kinda leery of their stated proceedure - reduce the quality, remove the object, then recover the quality. How do you recover quality once you have discarded it?

[guns1inger]

I have seen and used software that does this as a post effect - it has been around for years, but I am also pretty wary of the method that they state they are using.

[poisondeathray]

What's amazing is the realtime nature.

What's not-so-amazing is there weren't any motion examples (the camera moves, but the "objects" were always on static backgrounds).

The typical software that does this sort of thing in post (object and rig removal for special effects, e.g. mokey, nuke) usually use motion tracking and cloning from adjacent frames, but it takes a long time to process, and hardly done in "realtime"

[PuzZLeR]

This is not new. I've seen an Adobe Webinar almost 4 years ago where they demonstrate how their latest professional-ware "redrew" the Golden Gate Bridge without the vehicles (from footage captured during high traffic).

I've seen lots of this stuff since too.

How do you recover quality once you have discarded it?

He he, that's when the bull detector radar in my head started to light up.

[El Heggunte]

How do you recover quality once you have discarded it?

Goa'uld technology.

[PuzZLeR]

With all due respect, there is emerging technology ("increasing quality") with mathematics using Fractals. But this is incredibly CPU intensive for today's normal PCs, which can take weeks/months of processing time for one movie.

I highly doubt this application is employing such a scheme hence my radar goes off, especially since they don't mention how they "increase the quality..." with "...and improve the result".

BTW - Speaking of AviSynth (or even VirtualDub), isn't this nothing more than some higher-end logo/watermark remover?

[jagabo]

It's like Photoshop's latest content aware fill. But realtime.

http://www.youtube.com/watch?v=9L8XogFJbAc

Notice most of the realtime samples used boring flat backgrounds. And even then it didn't work really well (lots of artifacts). Let's see them remove a person walking down the sidewalk in a panning, busy, downtown traffic scene.

[PuzZLeR]

Let's see them remove a person walking down the sidewalk in a panning, busy, downtown traffic scene.

In that case we'd see ghosts with this method. Now that's "changing your reality."

[oldfart13]

I can still see where the original object is/was. Needs work methinks....

[lordsmurf]

"increase the quality" = bullshit. I keep rather in touch with forensic-level restoration methods, and that crowd would have a collective orgasm over technology of that nature. It would revolutionize the field. Yet I've heard nothing.

There's some level of slight-of-hand going on here. The blipped sink drain is a cause of concern -- likely bad editing, meaning the whole thing is faked in some way. At best, they've chosen examples that are hard to see flaws with, due to the crap quality of Youtube, mixed with relatively bland background that can easily be blurred and go unnoticed.

Content-aware filters in Photoshop are based off cloning. This doesn't make a claim to cloning, but rather only magical methods that don't exist ("increase the quality").

I'm suddenly reminded of CherryOS.

[edDV]

Like any claims, this needs independent confirmation. Scientific Method rules.

[Noahtuck]

I can still see where the original object is/was. Needs work methinks....

Yep!!!!

I saw it the first time i watched the video and it was very obvious!!!!!!

It was extremely obvious when they tried to hide the drain in the sink, it was not such a stagnant/static background as the other examples, there are reflections and they even edited it to late and you can actually see a part where the drain is visible for a second or two, the other examples are obvious if you actually look, but the drain is the most obvious.

I am thinking some german's are smoking the glass pipe!!!
And i am allowed seeing as i am 3/4ths german!!!
LOL!!

"increase the quality" = bullshit.

LOL!!!
Yep, let's take a Bluray video and reduce it to VCD quality, blur out something, edit it, then bring it back up to Bluray quality!!!
No one will ever know the difference!!!
(unless we live in India)

And i can take an original uncompressed wav file, convert to a 128kbps MP3, edit it, then convert it back to a WAV file and get back the full quality!!!

LOL!!!

[edDV]

It all comes down to doing it realtime even if artifacted. That takes CPU+GPU grunt.

There is demand for realtime effects. Current state of art is the virtual studio.
http://www.google.com/search?client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&chann...=Google+Search



See this for non-realtime fun... Go to "Behind The Scenes / Tech Demos" "Knight Rider" first.
Then watch the rest...
http://www.stargatestudios.net/page.php?section=4

[guns1inger]

These guys have done a lot in the augmented reality research space, which by necessity includes a lot of real-time tracking in 2 and 3d space, so I don't doubt all of it. However the process as described still seems, well, odd.

[jagabo]

I don't think anything was faked. The parts where the objects poke through are just where their realtime algorithms break. Where the background is abnormal are errors in their content aware fill. By "increase the quality" they're talking about the intermediate blurred image with the object removed -- they still have the original image around to restore details, their doing some kind of alpha blend. It's just a video for the lay-person, not intended to give all the exact details of the process.

[PuzZLeR]

...not professional-ware IMO at any rate.

[jagabo]

It's just a proof of concept demonstration. College students playing with their new slate computer.

[lordsmurf]

It's just a video for the lay-person.

If true, then this was stupid to do.
Laymen should not be the audience of researchers at the R&D level.

Those aren't the people who will ultimately fund or buy what they have made.

[mbur]

Laymen should not be the audience of researchers at the R&D level.

Those aren't the people who will ultimately fund or buy what they have made.

I would love to agree with you but there are plenty in high places who will be influenced by the groovy soundtrack and the sensational way this proported technology has been presented.

The glitches at 0:34 etc would tend to indicate that if an algorythmic method is indeed being used it only works where there is a consistent unform background. Where there is a change (ie when the cameramans shadow hits the touched up area) it all goes a bit wrong.

[PuzZLeR]

Depends which stage this thing is at: proposal or product.

As "product", yes this video is indeed geared towards laymen, or consumers, and could work. If it needs funding, and is still in "proposal", it's a bad move to pitch investors - you'd need something different.

Sensational maybe, but does not have a business angle to impress the deep pockets.