Seems like I have to settle for either jaggies OR interlacing combing?!

[jonfopp]

I hope you can see the image at large size to understand my issue. Basically, I have a large collection of *.MTS footage from a Canon HG10 that I want to use for a high quality edit in Adobe After Effects CS5.5. I have been aware of the difficulties with AVCHD footage, and have never been able to find a perfect way to play them back. VLC gives me some combing, but AE seemed to read the files fine, and I can render reasonably quickly with my machine.

However, on closer inspection I noticed that line detail seemed to gain an aliasing effect on import (see jaggies on branch in the top image). This is not present on the original MTS clip played with the camera or through VLC media player. It is present within After Effect's preview window, a lossless AVI render from AE, and through Windows Media Player.


I noticed that by default AE interprets the footage as Interlaced, upper field first. I thought this was the key to my issue, as it might be that some software (AE and WMP) misinterprets how the clip is wrapped (..I don't have much technical knowledge, I've just read around a bit). Anyway, I change the clip settings in AE to force progressive (by turning off separate fields). The jaggies vanish and clean up my image quality.. BUT, I am now faced with interlacing flicker and combing on any moving parts (see bottom image).






I have tried every combination on those dropdown menus / check boxes, but I either get jaggies or combing. I also tried a tool that rewraps the file as a MOV and an MP4, but both have the same problem. I also tried a tool that transcodes the file to AVI. This also resulted in the same problem and an unmanageble file size for the whole project. It seems like the only way I can see the clips without any decoding errors is through the camera. I have searched the net endlessly and know there is a lot of stuff on this..but I'm really stuck here! Please tell me there is a way to fix this!


PS the project will involve 250 clips at around 50GB. Quality is important and eventual use will be an HD upload to Vimeo.


Thanks

[poisondeathray]

What mode did you shoot at ? 25p ? "50i" ? (It's also called 25i, same thing, different naming conventions)

However, on closer inspection I noticed that line detail seemed to gain an aliasing effect on import (see jaggies on branch in the top image). This is not present on the original MTS clip played with the camera or through VLC media player. It is present within After Effect's preview window, a lossless AVI render from AE, and through Windows Media Player.

Note - Your screenshot is at 200%, so some aliasing is "normal" in the viewer

If your footage is truly interlaced (50 fields per second), the only good way to get rid of it is to use a higher quality deinterlacer (e.g. QTGMC in avisynth) . Adobe is terrible for deinterlacing. Since your final product is for web, progressive should be the goal . To keep high quality standards throughout, you should use lossless or near lossless intermediates (so yes, this means large filesizes)

[jagabo]

Your deinterlaced shots show an obvious "discard field" or "bob" deinterlace. You'll need a better deinterlacer to get rid of those artifacts. You may find Yadif adequate. QTGMC is usually the best.

[jonfopp]

Thanks for the quick responses.

What mode did you shoot at ? 25p ? "50i" ?

As far as I remember the camera was not in 'Movie Mode', so I would guess it is all 50i. Is there a way I can find out? This wasn't something I considered at the time! The upper screengrab shows how the footage is interpreted (25FPS, separate upper field first). I have to admit I don't fully understand what all these different modes mean.

Note - Your screenshot is at 200%, so some aliasing is "normal" in the viewer

The aliasing I'm referring to is much chunkier than individual pixels, and appears on lossless export as well as in the viewer. (See the tree branch outlined and compare both images). When I make the footage progressive, I actually gain significant detail in these contours, and the image quality is improved (besides the combing).

Your deinterlaced shots show an obvious "discard field" or "bob" deinterlace. You'll need a better deinterlacer to get rid of those artifacts. You may find Yadif adequate. QTGMC is usually the best.

Hi Jagabo, do you mean the branch with jaggies (upper field first) or the combing (forced progressive)? I'm not sure which artefact shows a bob/discard field?


So, if I've got this straight - the footage was probably shot as 50i, and most software struggles to deinterlace it accurately. I need to use a tool such as 'QTGMC' to transcode to an intermediate lossless format before editing. ?


Are there any recommendations for my method?


Another question would be - for shots where there is little movement, can I continue using the MTS footage in forced progressive alongside my converted files in 25p?

[jagabo]

Your deinterlaced shots show an obvious "discard field" or "bob" deinterlace. You'll need a better deinterlacer to get rid of those artifacts. You may find Yadif adequate. QTGMC is usually the best.

Hi Jagabo, do you mean the branch with jaggies (upper field first) or the combing (forced progressive)? I'm not sure which artefact shows a bob/discard field?

I meant the jagged branch in the top image. It's jagged because every other scan line was discarded.

Your lower image image is showing both fields so you see comb artifacts where things are moving. Interlaced video is intended to be viewed one field at a time. So 25 interlaced frames per second should be viewed as 50 fields per second.

Your lower image after a simple bob deinterlace, slowed to 2 fps:


after QTGMC:

So, if I've got this straight - the footage was probably shot as 50i, and most software struggles to deinterlace it accurately. I need to use a tool such as 'QTGMC' to transcode to an intermediate lossless format before editing. ?

Are there any recommendations for my method?

QTGMC is only available in AviSynth -- a script driven video filtering system. Yadif has been implemented in a few GUI based tools. Like VirtualDub.

[poisondeathray]

Depending on what you plan on doing in AE (It's not really the best "editing" tool, I would use an NLE for that) , it might be ok to edit as interlaced, export interlaced, then use QTGMC at the end of the workflow. The advantage of that is less HDD requirements, faster workflow (presumably you're cutting out segments, and applying QTGMC and re-encoding all the clips beforehand - even to segments that you edit out - will add up to lot of wasted extra time and storage). The disadvantage is depending on what effects you plan on doing in AE, it's sometimes better to work progressively . Typically people would do the cuts and editing in premiere (it's alot faster for editing), then the effects in AE . QTGMC is pretty slow to process for HD footage, even on one of the faster presets

[jonfopp]

Jagabo, thanks a lot for demonstrating.

So the simple bob deinterlace is similar to what After Effects is doing to my 50i footage. I guess it must be inventing the missing scan lines on one field, creating jaggies on near horizontal shapes (visible in your first image). Am I correct in thinking that at 25FPS I must therefore be losing 50% of the information within my footage and gaining 50% worth of interpolated info when set as upper field first? Then again, 'upper field first' would imply that the lower field (the other 50% of my data) would be done next. But at 25FPS, I don't see how that's possible?


Alternatively, taking the footage from 50i to 25p integrates the two fields 1/50th of a second apart, creating combing on fast movement. QTGMC must have some kind of 'smart filter' where it recognises harsh contrast between lines and only interpolates those sections..or something? That's exactly the effect I want (your second image).

Depending on what you plan on doing in AE (It's not really the best "editing" tool, I would use an NLE for that) , it might be ok to edit as interlaced, export interlaced, then use QTGMC at the end of the workflow.

Sounds like a great idea in principle, because I don't think I could go through the pain of converting all of these clips and dealing with the massive amount of data. My only question would be, can I do this in reverse? (For the reasons above). If I render out as 25P and then process with QTGMC, I guess I would end up with Jagabo's second image - with nice detail and correctly blurred movement. I have the advantage that the majority of my footage does not have fast motion. (I guess I can stick to interlaced when I have a problem clip).


I use After Effects just because I like to do lots of different processes on each clip (complex image adjustments, masking, vignettes, magic bullet adjustment layers that change for each clip etc). For some reason doing post/grading after an edit is more complicated to me because of altering the length of all the adjustment layers rather than working as I go.

[poisondeathray]

So the simple bob deinterlace is similar to what After Effects is doing to my 50i footage. I guess it must be inventing the missing scan lines on one field, creating jaggies on near horizontal shapes (visible in your first image). Am I correct in thinking that at 25FPS I must therefore be losing 50% of the information within my footage and gaining 50% worth of interpolated info when set as upper field first? Then again, 'upper field first' would imply that the lower field (the other 50% of my data) would be done next. But at 25FPS, I don't see how that's possible?

It's not inventing anything. AE is doing a drop field deinterlace when you export 25p . 1/2 the fields are missing. When you export 50p, all the fields are included, just resized as frames. Each field is like 1/2 a spatial frame. Adobe doesn't do any interpolation, it just resizes the field as a frame like a simple bob. Hence the "jaggies"

Alternatively, taking the footage from 50i to 25p integrates the two fields 1/50th of a second apart, creating combing on fast movement. QTGMC must have some kind of 'smart filter' where it recognises harsh contrast between lines and only interpolates those sections..or something? That's exactly the effect I want (your second image).

Yes, QTGMC attempt to fill in that missing information by looking at fields before and after. It does extensive temporal processing, antialiasing and smoothing . In static scenes , it attempts to weave the fields to prevent unnecessary damage

Depending on what you plan on doing in AE (It's not really the best "editing" tool, I would use an NLE for that) , it might be ok to edit as interlaced, export interlaced, then use QTGMC at the end of the workflow.

Sounds like a great idea in principle, because I don't think I could go through the pain of converting all of these clips and dealing with the massive amount of data. My only question would be, can I do this in reverse? (For the reasons above). If I render out as 25P and then process with QTGMC, I guess I would end up with Jagabo's second image - with nice detail and correctly blurred movement. I have the advantage that the majority of my footage does not have fast motion. (I guess I can stick to interlaced when I have a problem clip).

No , if you render out as 25p, the damage will already be done by Adobe. You can't "fix" it with QTGMC afterwards - you need everything intact for QTGMC to work it's magic

That's the reason for my suggestion .

You either pre process with QTGMC giving you clean progressive footage - but that will lead to lots of "wasted" footage that you're going to cut out anyway . It's a waste of HDD space and processing time

Or you at least do rough edits and keep interlaced until the end ,where you apply QTGMC (the disadvantage is some effects and graphics will suffer) , or some combination of the two methods

[jagabo]

Am I correct in thinking that at 25FPS I must therefore be losing 50% of the information within my footage and gaining 50% worth of interpolated info when set as upper field first?

Yes. That's what it looks like the program is doing.

Then again, 'upper field first' would imply that the lower field (the other 50% of my data) would be done next. But at 25FPS, I don't see how that's possible?

Yes, it should say "upper field only" or "lower field only" since only one of the fields is used.

Alternatively, taking the footage from 50i to 25p integrates the two fields 1/50th of a second apart, creating combing on fast movement.

Yes. Digital video is almost always stored as frames. If those frames are interlaced it becomes the job of the player or TV to split the frames into fields for display. TVs are designed to handle interlaced video. Computer monitors are not.

QTGMC must have some kind of 'smart filter' where it recognises harsh contrast between lines and only interpolates those sections..or something?

Yes. It also uses motion interpolation to try fill the missing scan lines with data from other fields.

[jonfopp]

1/2 the fields are missing. When you export 50p, all the fields are included, just resized as frames. Each field is like 1/2 a spatial frame. Adobe doesn't do any interpolation, it just resizes the field as a frame like a simple bob. Hence the "jaggies"

Ah right..so it's just making every scan line 2 x height?

if you render out as 25p, the damage will already be done by Adobe. You can't "fix" it with QTGMC afterwards - you need everything intact for QTGMC to work it's magic

Are you saying AE should interpret my footage as "50i", to include all the fields? And I needn't worry about the aliasing, because a 50i export would contain all of the information to deinterlace to 25P with QTGMC, and remove aliasing?


I guess I'm confused because the QTGMC example posted by Jagabo looks like I want it to. That's combined fields with the combing interpreted and removed by QTGMC. I guess it will depend on each and every piece of footage.


Also, I'm not sure how to force AE to interpret it as anything other than 25FPS with either upper or lower fields exclusively, or combined together into 25P...


Sorry, although I'm not new to video editing, this seems like some kind of advanced science!

[poisondeathray]

1/2 the fields are missing. When you export 50p, all the fields are included, just resized as frames. Each field is like 1/2 a spatial frame. Adobe doesn't do any interpolation, it just resizes the field as a frame like a simple bob. Hence the "jaggies"

Ah right..so it's just making every scan line 2 x height?

Yes, it's just using a cubic resize for each field (each individual field is 1920x540 when the scanlines are stacked together, and each field is resized to 1920x1080 full height)

if you render out as 25p, the damage will already be done by Adobe. You can't "fix" it with QTGMC afterwards - you need everything intact for QTGMC to work it's magic

Are you saying AE should interpret my footage as "50i", to include all the fields? And I needn't worry about the aliasing, because a 50i export would contain all of the information to deinterlace to 25P with QTGMC, and remove aliasing?

Also, I'm not sure how to force AE to interpret it as anything other than 25FPS with either upper or lower fields exclusively, or combined together into 25P...

Yes, that's option B. Certainly an option, but suboptimal IMO:

In order to handle interlaced footage in AE, you leave the comp settings as 1920x1080 , 25 frames per second . But when rendering out you specify "field render: upper field first" in the render settings . ie. you export 1920x1080i50 (same as source) . AE knows the assets are interlaced and the fields will be accessible . With intact fields in the export, QTGMC can do it's magic

Within AE, the preview will look like crap (because it's being single rate deinterlaced using the poor algorithm - BUT only for preview purposes), but the fields will be intact when you export out an interlaced format. That's how broadcasters do it for 50Hz spec broadcasts

The potential problems doing it that way will be 1) suboptimal interlaced chroma upsampling (when it converts to RGB, you will get chroma "notching") , 2) some interlaced graphics /overlays and some effects (especially anything with thin lines) won't look as good than if you had applied them on progressive footage

Arguably, the "best" workflow - at least quality wise - IMO would be to use QTGMC to create 1080p50 footage and use that for everything (I would still do rough edits before, cutting out footage that you're sure you're not going to use). That gives you the most options later. If you drop 1/2 the frames, to 1080p25 you cannot make a proper 1080i50 blu-ray or interlaced PAL DVD later. In the not too distant future, I believe 1080p50 will be accessible for streaming, vimeo, youtube etc.... In the future, I believe 1080p50 will be accessible for streaming, vimeo, youtube etc... You don't want to limit your options so early .

It should go without saying, but you should always keep the original footage archived (there might be improvements or better deinterlacers developed in the future) . But at this point in time (and for the last few years, the precursor TGMC and QTGMC are hands down the best overall deinterlacers)

Do some small tests and see which workflow meets your needs

I guess I'm confused because the QTGMC example posted by Jagabo looks like I want it to. That's combined fields with the combing interpreted and removed by QTGMC. I guess it will depend on each and every piece of footage.

It's clearly a better deinterlacer for most situations overall (avisynth has wide variety of options and different choices for deinterlacers, and QTGMC itself has dozens of different settings and options)

[2Bdecided]

Quality is important and eventual use will be an HD upload to Vimeo.

As others have said, edit the whole lot as interlaced, and then deinterlace the final render using something half-decent.

Vimeo isn't that good quality, and it only supports 25p rather than 50p. Unless it's mostly still shots that Vimeo can compress cleanly, top-quality deinterlacing isn't that important. The low bitrate that Vimeo uses will hide most of the problems when anything moves. I'd still choose to do the job properly, but it's not 100% essential.


btw, though I've recommended VLC for many years, I've finally moved with the times and got Windows 7 (OK, nearly moved with the times! ) and the version of Windows Media Player it ships with is more than good enough for playing back interlaced video. I reckon it's better than VLC.


Cheers,
David.

[jonfopp]

In order to handle interlaced footage in AE, you leave the comp settings as 1920x1080 , 25 frames per second . But when rendering out you specify "field render: upper field first" in the render settings . ie. you export 1920x1080i50 (same as source) . AE knows the assets are interlaced and the fields will be accessible . With intact fields in the export, QTGMC can do it's magic

Cool. I tried this on a 2 second clip and exported as a lossless AVI. I can tell it's '1920x1080i50' because VLC shows combing artifacts which must mean both fields are present. Also the clip is somehow 14GB. It might be besides the point, but how can it be so big?? I mean, exporting 100 lossless PNGs of each field would be nowhere near that size..
On that point, what would be a good intermediate format post edit, pre deinterlace?

The potential problems doing it that way will be 1) suboptimal interlaced chroma upsampling (when it converts to RGB, you will get chroma "notching") , 2) some interlaced graphics /overlays and some effects (especially anything with thin lines) won't look as good than if you had applied them on progressive footage

1) sounds worrying. I didn't realise about colour conversion, but can always keep my intermediate 50i incase I can ever improve my conversion.
2) There won't be much in the way of graphics, but I can see the potential for interpolation errors. I guess I could do that on my 25P footage afterwards if I want.

If you drop 1/2 the frames, to 1080p25 you cannot make a proper 1080i50 blu-ray or interlaced PAL DVD later.

I assume this is a good reason to keep hold of my 50i intermediate file?

avisynth has wide variety of options and different choices for deinterlacers, and QTGMC itself has dozens of different settings and options)

I'm sure this stage will kick up a whole load more questions. Are there any general pointers or shall I cross this bridge when I come to it? Eg. Should I be aiming for 1080P 25 near lossless AVI? Then go on to use a Vimeo friendly preset..

btw, though I've recommended VLC for many years, I've finally moved with the times and got Windows 7 (OK, nearly moved with the times! ) and the version of Windows Media Player it ships with is more than good enough for playing back interlaced video. I reckon it's better than VLC.

Hi David. I noticed that too with Windows 7. Also, I just downloaded the latest version of VLC and it no longer plays *.MTS or *.M2TS files - only the sound. On their forums they say they are looking into it. Maybe time to move on.

[poisondeathray]

In order to handle interlaced footage in AE, you leave the comp settings as 1920x1080 , 25 frames per second . But when rendering out you specify "field render: upper field first" in the render settings . ie. you export 1920x1080i50 (same as source) . AE knows the assets are interlaced and the fields will be accessible . With intact fields in the export, QTGMC can do it's magic

Cool. I tried this on a 2 second clip and exported as a lossless AVI. I can tell it's '1920x1080i50' because VLC shows combing artifacts which must mean both fields are present. Also the clip is somehow 14GB. It might be besides the point, but how can it be so big?? I mean, exporting 100 lossless PNGs of each field would be nowhere near that size..
On that point, what would be a good intermediate format post edit, pre deinterlace?

You probably exported an RGB uncompressed intermediate.

PNG is lossless, but losslessly compressed. Exporting uncompressed RGB would be the equivalent of exporting BMP's or TIFF sequences in still images

There are a number of lossless video codecs, and there are trade offs for various lossless compression schemes: speed of compression , speed of decompression (latency and editing speed), compression ratio, compatibility with software and container formats... many factors to look at

Popular lossless windows RGB compression include lagarith, ut video codec, ffv1, or even image sequences like PNG . Popular near lossless ones would be cineform (the free one is YUV 4:2:2, not YUV 4:4:4 , not RGB), DNxHD. On a Mac, Prores is used very frequently

The potential problems doing it that way will be 1) suboptimal interlaced chroma upsampling (when it converts to RGB, you will get chroma "notching") , 2) some interlaced graphics /overlays and some effects (especially anything with thin lines) won't look as good than if you had applied them on progressive footage

1) sounds worrying. I didn't realise about colour conversion, but can always keep my intermediate 50i incase I can ever improve my conversion.

AE works in RGB . But your original videos are Y'CbCr (YUV) 4:2:0 . That can pose big problems converting back & forth if not done properly

I can't recall if it was dramatically improved in CS5.5 or CS6. It definitely affected CS4, CS5 . There was a severe bug that was fixed, especially on progressive footage that was upsampled as interlaced. Interlaced footage wasn't upsampled that nicely either. Eitherway, using another format other than AVCHD is immune to those errors . (And even though it's dramatically improved, many still chose to use an intermediate instead of native AVCHD import for that and various other reasons)

2) There won't be much in the way of graphics, but I can see the potential for interpolation errors. I guess I could do that on my 25P footage afterwards if I want.

I'm referring to chroma upsampling errors (CUE). It was worse in the progressive AVCHD case. But the interlaced AVCHD still had problems. Anything with strong visible color borders gets these notching artifacts . You can search if you want more info, basically it caused big big problems back then, or unless you used something other than AVCHD (e.g encode to an intermediate before importing)

If you drop 1/2 the frames, to 1080p25 you cannot make a proper 1080i50 blu-ray or interlaced PAL DVD later.

I assume this is a good reason to keep hold of my 50i intermediate file?

Yes, and your originals

avisynth has wide variety of options and different choices for deinterlacers, and QTGMC itself has dozens of different settings and options)

I'm sure this stage will kick up a whole load more questions. Are there any general pointers or shall I cross this bridge when I come to it? Eg. Should I be aiming for 1080P 25 near lossless AVI? Then go on to use a Vimeo friendly preset..

QTGMC has preset settings that you should start with e.g. "faster" , "slower", etc.... It is a bobber , so you will get 50p from 50i. But in avisynth you can use SelectEven() or SelectOdd() to get 25p . It's not necessary to use a 1080p25 near or lossless AVI from your 1080i50 AE export, because avisynth is a frameserver (it can take that AE export directly, process with a script and feed that data into an encoder without a big intermediate file)

[jonfopp]

Popular lossless windows RGB compression include lagarith, ut video codec, ffv1, or even image sequences like PNG . Popular near lossless ones would be cineform (the free one is YUV 4:2:2, not YUV 4:4:4 , not RGB), DNxHD. On a Mac, Prores is used very frequently

Ok, so for my intermediate I would assume speed and size are not so important considerations, because I have no time limit and would happily buy a dedicated hard drive? I think you're implying that After Effect's codecs within its normal output module are not good enough to use. Lagarith, UT, ffv1 aren't listed, so I guess I can somehow download and run these through AE. Unless I just use a PNG sequence or similar. Would avisynth be able to process a folder of PNG interlaced fields and render out a near lossless 50P file?

AE works in RGB . But your original videos are Y'CbCr (YUV) 4:2:0 . That can pose big problems converting back & forth if not done properly

Thanks for the alert, I will do some research.

QTGMC has preset settings that you should start with e.g. "faster" , "slower", etc.... It is a bobber , so you will get 50p from 50i. But in avisynth you can use SelectEven() or SelectOdd() to get 25p . It's not necessary to use a 1080p25 near or lossless AVI from your 1080i50 AE export, because avisynth is a frameserver (it can take that AE export directly, process with a script and feed that data into an encoder without a big intermediate file)

I am a little confused but I think my method would be something like this..


Complete my After Effects composition with 50i footage interpreted as 25FPS, upper field first.
Use AE to render out a PNG sequence at 50i.
Or
Install a good lossless codec and use within AE to render out a 50i file.
Use avisynth to encode file(s) with QTGMC to 50P.
Convert to 25P for more filmic look
Render out a file for Vimeo.


Do you think there any mistakes / loopholes? Thanks for the help

[poisondeathray]

I am a little confused but I think my method would be something like this..


Complete my After Effects composition with 50i footage interpreted as 25FPS, upper field first.
Use AE to render out a PNG sequence at 50i.
Or
Install a good lossless codec and use within AE to render out a 50i file.
Use avisynth to encode file(s) with QTGMC to 50P.
Convert to 25P for more filmic look
Render out a file for Vimeo.

Do you think there any mistakes / loopholes? Thanks for the help

Yes that's the basically it



There are no "mistakes" ; but areas that you might get slightly better quality .

For example, I don't like the way AE does YUV<=>RGB conversions . It probably won't matter as much on your footage, but on higher quality (non AVCHD) footage, or graphic elements done in AE, you can see the difference

If you're going for "filmic" look, the shutter angle will be "wrong" for "filmic" because you shot at "50i" . Most consumer camcorders will auto adjust the shutter to be 1/100. Motion blur won't look appropriate as for something originally shot at 24p with 1/48 shutter . QTGMC has some parameters that can emulate motion blur as if the shutter was 1/48 (or 1/50 for 25p)

Also , you can convert to 25p directly in avisynth (It's not a separate stage or step)

[2Bdecided]

You have to convert to 25p for Vimeo whether you want a filmic look or not.

If you really want a filmic look, shoot 25p 1/50th shutter to start with. Or if your camcorder won't do that, shoot 50i with a 1/50th shutter and then use the workflow above.


If you really want a filmic look, there's a lot more to it than framerate. Whole books on the subject.
http://hddv.net/showthread.php?2941-Acheiving-quot-the-look-quot
http://hddv.net/showthread.php?43125-Single-most-contributing-factor-to-footage-lookin...cinematic-quot


Cheers,
David.

[jonfopp]

Thanks all, I may be back later.

The 'filmic' 25p reference was just because I like the dithering effect of lower frame rates, and it suits this particular project (it's not a film). I will render a 50P version as my master file, and then use that as the source of subsequent renders for online etc.


Cheers