Video Conversion / Editing Workflow Issues

[poisondeathray]

This is a Sony camcorder clip, not sure what model, not mine. But regular consumer model shooting 60i. Not the cheapest, not the most expensive consumer model.

Every info I can get on the submitted clip says 29.97i, not 60i. 1080i for AVCHD at 60i is not standard. Or maybe you're saying it's 60 FIELDS per sec instead of FRAMES p/s. This sounds rather disorganized (?) and misleading (?). It also appears to have been around the block a few times and the nm2v is noisy enough to make it look like tape.

"60i" is the lazy term for "59.94i" . Both mean the same thing as "29.97i" , just different naming conventions . Both indicate 59.94 fields per second interlace. Most old skool folk use "29.97i" because that's what it's been called for ages - but newer camcorders, software, tend to use the new naming standard

And I don't even think it's interlaced. It looks blend-deinterlaced to me.

probably an issue with your software or decoder doing something it's not supposed to

[sanlyn]

Had something to submit earlier, but I couldn't stand watching the re-rendering compression artifacts on that sample. It might have been interlaced at one point in time before the definition of that term was changed, but it'll never see interlace again. TMPGenc Plus 2.5.

[davexnet]

Hi sanlyn - your mpg is progressive. Did you deinterlace it? It looks a little cleaner on the computer monitor,
but that may be because the interlaced versions are being de-interlaced on the fly by the player.

[sanlyn]

Hi sanlyn - your mpg is progressive. Did you deinterlace it?

No. Not a good idea to deinterlace video that's no longer interlaced. I didn't adjust audio timing, either -- audio and video in the re-rendered original post are not the same duration.

[davexnet]

Are you sure it's not interlaced?
When I open in in Virtualdub, I can clearly see the two fields with the tell-tale combing.
Using the "viewfields" virtualdub filter, which splits it into two fields vertically stacked,
both the fields look clean, indicating true interlacing.

[poisondeathray]

Hi sanlyn - your mpg is progressive. Did you deinterlace it?

No. Not a good idea to deinterlace video that's no longer interlaced. I didn't adjust audio timing, either -- audio and video in the re-rendered original post are not the same duration.

Original is interlaced content, encoded interlaced

You probably have issues with your software or decoders/splitters/source filters . They might be applying a deinterlace or doing stoopid things behind your back

[davexnet]

[QUOTE=poisondeathray
You probably have issues with your software or decoders/splitters/source filters . They might be applying a deinterlace or doing stoopid things behind your back[/QUOTE]

I did something like this once. I had activated the deblocking in FFdshow's post processing
to try and clean up an old black and white UK TV show from the 60's. The deblocker worked great,
but I inadvertently left it on, and it bit me in the arse some time later.

[sanlyn]

Are you sure it's not interlaced?
When I open in in Virtualdub, I can clearly see the two fields with the tell-tale combing.
Using the "viewfields" virtualdub filter, which splits it into two fields vertically stacked,
both the fields look clean, indicating true interlacing.

They don't look clean. They don't look clean after yadif or QTGMC, either. This might have been interlaced video at one time. No more. Something's happen to it. You can deint til you're blue in the face, it'll never be right.

[poisondeathray]

^The reason is AVCSource. It has well documented issues with PAFF streams. Use something else

[_Al_]

thanks guys for samples, this is proving the pattern basically I found out also,

smart resize makes jiggle or moare the least, but it is a bit blurred,
29.970p is not good, missing frames are too obvious, noise could be acceptable but not missing frames

better avchd footage, different camcorder, it is 25i , cleaner footage, daylight:
original avchd 25i (as TS had to cut it): http://www.uploadmb.com/dw.php?id=1389543685
smart resize m2v 25i for DVD ,http://www.uploadmb.com/dw.php?id=1389547844
m2v directly from Vegas, 25i for DVD:http://www.uploadmb.com/dw.php?id=1389548470

I loaded that avchd in Sony Vegas, exported through dmfs to Avisynth smart resize there, changed color space to YUV and HCencoder encoded m2v, that is the city_smart_resize.m2v, looks better that encoded in Vegas directly, where picture kind of "shimmers"

I just added this x264 encode, interlaced , that brings up that smart resize filter a bit more, excluding HcEncoder created artifacts in that smart_resize.m2v file, so this is x264.MP4:
http://www.uploadmb.com/dw.php?id=1389552623

[poisondeathray]

thanks guys for samples, this is proving the pattern basically I found out also,

smart resize makes jiggle or moare the least, but it is a bit blurred,
29.970p is not good, missing frames are too obvious, noise could be acceptable but not missing frames

better avchd footage, different camcorder, it is 25i , cleaner footage, daylight:
original avchd 25i (as TS had to cut it): http://www.uploadmb.com/dw.php?id=1389543685
smart resize m2v 25i for DVD ,http://www.uploadmb.com/dw.php?id=1389544349
m2v directly from Vegas, 25i for DVD: http://www.uploadmb.com/dw.php?id=1389545087

I loaded that avchd in Sony Vegas, exported through dmfs to Avisynth smart resize there, changed color space to YUV and HCencoder encoded m2v, that is the city_smart_resize.m2v, looks better that encoded in Vegas directly, where picture kind of "shimmers"

It's not an "apples to apples" comparison . The problem is the vegas version was converted to NTSC specs, with blended fields inserted to make up the 25=>29.97

BTW, what "smart resize" filter is that? Avisynth ?

[_Al_]

oh, thanks I fixed it and upgraded link above, in this case vegas render looks actually better, sharper, there is no jiggle effect that much, maybe good sample for this

SmartResize by Donald Graft, v1.1 is taken from here: http://avisynth.nl/index.php/VirtualDub_I

[poisondeathray]

Time for some coffee _Al_ , I think you changed the wrong link

[_Al_]

it should be ok now

[poisondeathray]

The link for smart resize vid was changed as well, was something else changed ?

[_Al_]

No nothing changed, link for smart resize is changed because I uploaded it again and change the link, but it is the same file.
Link for Vegas is changed too, but file is correct now, 25fps interlace m2v.

[sanlyn]

^The reason is AVCSource. It has well documented issues with PAFF streams. Use something else

Attached are two frame captures from the original on frames with fast motion. I see motion blur and double-imaging, but no combing effects. Is combing supposed to be there?

DirectShowSource gave the same results as AVCSource, to the letter. To wit:

DirectShowSource(vidpath+"uploadMB_00239.MTS")

returns 854 frames (FPS 29.970)

DirectShowSource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")

returns 1705 frames

DirectShowSource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

returns 852 frames

TMPGenc Video Mastering Works 5
produced a clean 1080i Lagarith AVI output
returned 855 frames (29.970 fps)
(but resize to 480i AVI looks horrible)

TMPGenc Smart Renderer 4
Reads frames as having severe rolling shutter distortion.
Outputs a non-reencoded .TS copy with no distortion.
returned 854 frames (29.970 fps)

ffvideosource(vidpath+"uploadMB_00239.MTS")

returns 1710 frames (FPS 59.941 !!!)

ffvideosource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")

still returns 1710 frames

ffvideosource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")
SelectEven()

returns 1710 frames again!!

ffvideosource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")
SelectEven()
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

returns 855 frames - severely choppy motion and stutter

ffvideosource(vidpath+"uploadMB_00239.MTS")
AssumeTFF().QTGMC(preset="very fast")
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

Returned 1710 frames (again!)

Really sick of this one. Conclusion: someone will buy the camera on eBay.

[poisondeathray]

@sanlyn - nobody is denying that it's shitty footage , but it's still interlaced . You probably have something in your directshow filter chain that is deinterlacing it . This is one of those "directshow sucks" examples, where its prone to give you unintended results unless you keep all your filters/splitters in order . If you need help diagnosing directshow issues, start with rendering the file directly in graphedit or graphstudio

Usually when everybody else gets different results, it's got to make you wonder

Here is the same frame, but not deinterlaced (see the combing)



If you have an nvidia card, DGNVTools is very good and stable, works well without DGAVCDecode.dll's documented issues .

Or alternatively, use ffmbc/ffmpeg to decode a lossless intermediate (e.g. utvideo or ffv1), then AVISource() on that lossless intermediate

ffmpeg -i 00239.MTS -vcodec utvideo -an output_utvideo.avi

[davexnet]

What can be done about that moire effect seen in the book above bottom/center?

[poisondeathray]

What can be done about that moire effect seen in the book above bottom/center?

If you're not viewing at 1:1 moire effect be enchanced due to the resizing

To reduce moire patterns - you can use QTGMC with low sharpness settings, +/- additional antialiasers . QTGMC can be applied in progressive mode (inputtype=1 or 2) additionally as well for severe problems . It has a very strong temporal smoothing effect. If the moire only affects a certain part of the frame, you can try to apply through masks, as to not damage other "good" parts uncessarily

Sharpness tends to enhance moire and "line twittering" on interlaced footage. Many people apply low pass filters when going to DVD , and/or use softer resizing algorithms

[_Al_]

Interesting, that pattern on that book got me interested too,

I tried qtgmc before, it is all super up to the point to watch progressive result, but problems is that jiggle and quite unnatural artifacts (not only on horizontal linear parts) are introduced after re-interlacing footage, using that classic code:

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[poisondeathray]

If you guys want to test the WORST case example of interlaced resizing to DVD, try this sample originally from the sony vegas forum . The fine patterns are a bloody nightmare!

http://www.jazzythedog.com/testing/videos/AVCHD.zip

There are several threads in the vegas forum, doom9 . Basically the only algorithm that can handle it fairly well is IResize() , but of course it produces softer results . Many people use a combination of techniques (e.g you might apply IResize() only on certain prone sections, or through masks, etc..)

[_Al_]

ok, I tried that sample,
smart resize, encoded by x264: http://www.uploadmb.com/dw.php?id=1389559213
smart resize, encoded by HcEncoder: http://www.uploadmb.com/dw.php?id=1389559284
encoded in Sony Vegas: http://www.uploadmb.com/dw.php?id=1389559337

this is a sample where that NLE resize problem (Vegas or others) can be seen, where moare can "run" through image

[poisondeathray]

The discussion kind of moved on from the original topic... but fine patterns, thin lines are terrible for interlace . Often "sharpness" isn't the problem for interlaced DVD's, the problem is more often than not : oversharpness. Most studios will apply a low pass filter , or vertical blur for interlaced DVD's (although they usually won't go to the extent of masking or applying filters to different parts of the frame to limit the effect)

Here is a gif animation where one was low passed , the other was re-interlaced as per usual AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

Both were then bob deinterlaced (which emulates what the majority of progressive flat panel HDTV's would use, although some more expensive models apply other processing ) , and cropped to the region of interest (you may have to open it in a separate tab for it to play depending on the browser you use)

[poisondeathray]

Hi all,

Have a quick question for the "group. I currently capture AVCHD from a consumer grade Canon Camera which records 1080i 29.97 Upper Field first video. I edit with Premiere with intentions to go to DVD. I have stopped using AME and Encore because the downsizing to 480 quality is terrible, even with "Maximum Render Quality"option in Adobe. So I use AME to go to MPEG2 bluray format and pretty much do a straight export but I choose the option "Lower Field First". Confirm with MediaInfo that the file shows now as Lower Field First. Now I take that file into TMPGENC Video Mastering works and use it to convert the video to DVD compliant MPEG2. I choose interlaced, op option of Upper or Lower. But when TMPGENC is done, it switches back to Upper, which when I burn to DVD looks like crap with the window blind effect.


I'm stumped. I have no idea why I can't get TMPGENC to make this lower field fist.


Any ideas?

Sorry, kind of got lost in the discussion -

HD interlaced content is Top Field First (or Upper Field First) by convention 99.9999999% of the time . So if you export MPEG2 bluray, use UFF. DVD doesn't matter if it's UFF or BFF . So if you stay UFF all the way through, it will be OK

If TMPGEnc doesn't do a good job of resizing interlaced content, even deinterlacing then resizing as Khaver suggested will look better than what Adobe does by default . There are also the other options discussed in this thread in avisynth , vdub . Even the Adobe forum are aware of these issues, and even they suggest using the avisynth HD2SD script (use search if you want more info, there are video tutorials showing you how to use it) , but there are other methods as well in avisynth

[_Al_]

Most studios will apply a low pass filter , or vertical blur for interlaced DVD's (although they usually won't go to the extent of masking or applying filters to different parts of the frame to limit the effect)

Oh, I forgot about that vertical blur trick, so in Avisynth bob Yadif + Blur (0,1.5) + resize + reinterlace works good enough.
What does that mean low pass filter in video, it is hardware solution only, or can it be used in Avisynth?

[poisondeathray]

Oh, I forgot about that vertical blur trick, so in Avisynth bob Yadif + Blur (0,1.5) + resize + reinterlace works good enough.
What does that mean low pass filter in video, it is hardware solution only, or can it be used in Avisynth?

Low pass filter in video just means certain frequencies above a certain threshold are cut off, but below are allowed to pass thru . There are physical (hardware) lowpass filters, but you can also do it in software.

In this context , it's the fine lines with interlace that cause problems . The vertical blur technique is a low pass filter , so is vertical resizing . IResize() is essentially resizing vertically. The tradeoff, of course, is reduced effective resolution and perceived "sharpness" , hence why some people apply them sparingly or through masks - You want to minimize damage to areas that don't have the problems

[sanlyn]

.... they suggest using the avisynth HD2SD script (use search if you want more info, there are video tutorials showing you how to use it) , but there are other methods as well in avisynth

I gave HD2SD a try, playing with denoising parameters. Actually, I got better results with the supaa() function from Doom9, which basically resises then applies sangnom, flips the video right, applies SangNom again, flips back left, and resizes to 640x480. Supaa() gets run after deinterlace + resize to 720x480, then reinterlace. In effect, doing it that way, the thing gets deinterlaced/reinterlaced and even resized multiple times. After that, I saw little difference between that method and HD2SD.

But HD2SD has some interesting ideas and whole lot of script to contemplate. I'd look into it more deeply to see how it could be tweaked. So this schizo video sample becomes anotehr one of those learning experiences and is (actually) appreciated.

This isn't the first time I downsampled HD to smaller frames. They turned out OK. The question: what is it about that freaky sample that doesn't occur with the 1080i TV sample that looks fairly normal and doesn't pose nearly so many problems? I later made another 1080i capture that was pure interlaced instead of interlaced pulldown frames, and it look far better than this cranky bookshelf job. Something to think about.

[_Al_]

Quality of camcorder doesn't matter in a sense that re-interlacing picture again will introduce artifacts. So crappy footage or not, the same precautions must be done.

[poisondeathray]

.... they suggest using the avisynth HD2SD script (use search if you want more info, there are video tutorials showing you how to use it) , but there are other methods as well in avisynth

I gave HD2SD a try, playing with denoising parameters. Actually, I got better results with the supaa() function from Doom9, which basically resises then applies sangnom, flips the video right, applies SangNom again, flips back left, and resizes to 640x480. Supaa() gets run after deinterlace + resize to 720x480, then reinterlace. In effect, doing it that way, the thing gets deinterlaced/reinterlaced and even resized multiple times. After that, I saw little difference between that method and HD2SD.

HD2SD is much better than Adobe's default solution, and decent for overall general use. I don't like how it uses ITU par for resizing, but you can specify OutputPAR to custom parameters

The question: what is it about that freaky sample that doesn't occur with the 1080i TV sample that looks fairly normal and doesn't pose nearly so many problems? I later made another 1080i capture that was pure interlaced instead of interlaced pulldown frames, and it look far better than this cranky bookshelf job. Something to think about.

Are you referring to _Al_'s posted samples ? (not "his" just ones he found somewhere posted) The books/library vs. the outdoor horse/carriage ?

The library scene is not really "freaky" in terms of fields or interlace - yes it looks crappy but it's still interlaced. Did you get the decoding sorted out yet ? Certain types of AVCHD tend to cause decoding issues with some splitters and decoders . Your "other capture" probably wasn't AVCHD, so you probably didn't have the same issues.

Otherwise PLENTY of other differences.

Quality wise: The books scene is 60Hz , and shot with a lower bitrate with worse lighting ; the carriage scene is 50Hz, shot with 30-35% more total bitrate (so that's even more bits/frame or bits/field when you factor in the 50Hz vs 60Hz frame/field rate difference ). Motion is the biggest difference that negatively impacts compression. There is a lot more motion from the books scene, where the camera is actually moving a lot - whereas the carriage scene is shot from a focal point , and there is a lot less camera motion . Reall with long GOP compression only the differences are stored between frames. So things like small changes in the buildings between fields in the carriage scene take a lot less bits to maintain a certain level of quality . Contrast that to the books where almost the entire image has to be refreshed (no data can be referenced from adjacent fields) completely every few fields.