DVD transcoding, MeGUI says "progressive", deinterlace vs. IVTC, confused

[abolibibelot]

Hi,
I want to transcode a movie I have on DVD, it was originally a made-for-TV movie broadcast in France in 2006 (called "Des fleurs pour Algernon", based on the novel "Flowers for Algernon" by Daniel Keyes, and seriously it's one of my favorite movies ever, maybe the one that moved me the most for very personal reasons -- in my opinion a much better take on that story than the 1968 USA movie "Charly" ; I might try to subtitle it afterwards so as to share it with the english speaking crowd -- it's a shame there are no subs on this DVD, not even HI ones, so I'll have to do it from scratch). It's a PAL DVD. MediaInfo reports it as interlaced (as is standard for any MPEG2 DVD, to my understanding -- or is it ?). DGIndex reports it as interlaced. But MeGUI's AVS Script Creator module, after its "analysis" is finished, reports the "source type" as "progressive", and disables the deinterlacing options. Yet it definitely looks like interlaced video, with the usual combing effect on fast moving scenes. Here's a sample (22MB, link available for 7 days) :
http://we.tl/Kb5upijFL1
Now, if I set the "source type" manually to "Interlaced" and choose Yadif as deinterlacer (which seems to be the next best thing after QTGMC), it removes the combing effect, yet there are still some jagged edges here and there. But if I choose "Source type : Film" and "TIVTC" (or Decomb IVTC, they look just the same), the picture no longer has those jagged edges and is generally much sharper and cleaner.
So, how does it work exactly, and what is the right way to deal with it ? (I read several threads before asking this, it only added to the confusion.) Why doesn't MeGUI report this as interlaced, and if it is indeed interlaced how come a proper deinterlacer gives an inferior result to "IVTC" (which I'm still struggling to find out what it does exactly except it's different from deinterlacing) ?
http://share.pho.to/9ZNpS

Otherwise, is there anything else I should do with such a source ? And, considering I have no special compatibility requirements this time (i.e. standalone devices), what settings would significantly improve the encoding efficiency ? (Ideally I would like to get a MKV file between 1GB and 1.5GB with the best possible quality, which should be manageable as it's a 1h34min movie with not much fast action, and the source is quite clean.)

Thanks.

[LigH.de]

Most DVD studios leave their MPEG-2 video encoders set to "interlaced encoding mode" (more or less separate fields), regardless of whether the video content is progressive (recording time progresses frame by frame) or interlaced (recording time may progress field by field, even occasionally). The reason is just that interlaced content, encoded in progressive mode, would be most inefficient and look horrible.

MediaInfo and DG*Index will only analyze the bit flags in the header, thus only know which encoding mode was set up in the video encoder.

MeGUI will instead try to analyze the video content, by testing a small cropped part in different encoding modes and comparing their efficiency, but can't be really certain either. It may have "bad luck" with the chosen selection. The only certain way is to watch the video "field by field" (as a result of a Bob() function) and look for possible patterns of the temporal progress.

If it is a true PAL production, however, you can be quite certain that it won't be Telecine. The worst case, though, may be that someone blended Telecine material. Sometimes, video studios employ uneducated staff to dump prices...
__

After looking at your material:

This is in fact a progressive production, there are field pairs during the whole time. Just the pairing got messed. A part of this clip is "phase shifted".

This may not be the job for a classical deinterlacer, rather one of a good "field matcher" to fix only the pairing of combed szenes. Unfortunately, I am not experienced with this case; but I believe that TFM and TDecimate in a special parameter combination may help.

[hello_hello]

If you want to know exactly what IVTC does, have a read here:
https://en.wikipedia.org/wiki/Telecine#2:3_pulldown
http://www.afterdawn.com/glossary/term.cfm/inverse_telecine
The short explanation is film is converted to NTSC using telecine (3:2 pulldown) for 29.970fps, and IVTC reverses the process to output 23.976fps progressive. IVTC matches up the fields again and the resulting duplicate frames are removed. IVTC normally doesn't apply to PAL. Most of the time 24fps "film" is converted to PAL's 25fps simply by speeding it up.
IVTC isn't really de-interlacing as such, but it's mostly included with the options for de-interlacing.

For your sample.....
If you're choosing source type Film, it probably looks sharper for the reasons LigH.de suggested (field matching rather than de-interlacing) but it'll also be treating the source as film and removing every fifth frame (as it would for an NTSC 29.970fps to 23.976fps IVTC conversion). If you use the preview after the "Film/IVTC" de-interlacing option is applied, you'll see the total frame count is reduced which is not required and the output won't be smooth.

In this case you want the field matching without the frame removal. If you select Film as the de-interlacing method (varying field order) MeGUI will add something like the following to the script:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1).tdecimate()

tfm() does the field matching, while tdecimate() does the frame removal. Before saving the script you need to remove the tdecimate part (the part in brown text). Check with the preview button after removing it, but that should fix the output without changing the frame count or de-interlacing (if you change anything in the script creator GUI, MeGUI will add it back and you'll need to remove it again).

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)

MeGUI only analyses sections of the video and in this case it appears it's probably only the opening credits where the fields are out of order. When I analyse your sample though, MeGUI sees it as "partially interlaced", and in that case it is getting it wrong, but for PAL that looks interlaced due to the fields being out of order like yours, it probably always will.

For the record, if you select "Interlaced" and "Yadif with Bob" as the de-interlacing method in the script creator, that'll give you double frame rate output as QTGMC does by default (50fps for PAL). If you select "yadif with bob" and use the preview, and when you step through the frames one at a time it just looks like every frame is repeated (as it does for your sample) it's a fair indication it's not really interlaced. If each frame is different, then it probably is.
Of course there's always the possibility that some parts will be interlaced and other parts progressive (hence MeGUI's partially interlaced option), but generally for movies they're either one or the other.

[hello_hello]

Something else you might want to try, given you mentioned QTGMC, is running it in progressive mode after the field matching (given the video will then be progressive), like this:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)
QTGMC(InputType=1)

I find QTGMC in progressive mode can do a nice job "cleaning up" video, but like all filtering it's not always needed, so you can apply it to just the sections where it is. For example, to use it to clean up the video behind the credits in your sample, but to re-encode the rest of the video without it, you could do something like this:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_01_1 (extrait début).demuxed.d2v")

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)

Trim(0,451).QTGMC(InputType=1)\
++Trim(542,659)

crop(4, 4, -4, -4)
Spline36Resize(1012,568)

The attached sample was encoded at CRF23 to keep the file size down, but the opening credits still look better than the original DVD video.
I'd probably crop and resize it the way I did in the above script and encode using at least CRF18, or if it's something I'd like to be close to perfect, maybe CRF16 if the file size doesn't get too out of control. Use the slowest x264 speed preset you can stand (I generally stick with Slow or Slower myself) and maybe try the x264 Film tuning to help retain a little more fine detail.

[LMotlow]

Nice output, hello_hello. Good info on telecine/interlace. I have a problem with non-compliance for BD/AVCHD, but that's easily fixed. Resize to 1280x720 and use Assumefps("film") for 24fps. If audio is in the script, use "AssumeFPS("film",sync_audio=true). It makes a bigger output file, of course, unless you want PC-only. Low bitrates don't look so great blown up on TV, IMO. One can always tweak at will.

[hello_hello]

LMotlow,
What script did you use? I assume it included some denoising?
Could I have a nitpick about upscaling without converting the colours?
The colormatrix plugin is included in MeGUI's collection of Avisynth plugins. If the colour conversion option is enabled in the script creator it probably should be disabled when upscaling and the correct color conversion added to the script manually.

ColorMatrix(mode="Rec.601->Rec.709")

I'll confess the BD/AVCHD compliance thing doesn't worry me. Once a video is h264 in an MKV or MP4, any player capable of playing MKVs or MP4s shouldn't have a problem with it. As long as it conforms to High Profile, Level 4.1 or lower. I've not met an MKV capable Bluray player that won't play High Profile, Level 4.1, and the resolution shouldn't matter.

The thing that tends to put me off upscaling is the increase in bitrate that it requires for almost no gain, in my experience.
As I haven't one it for a while, I tried a few test encodes as an experiment, and this time I enabled QTGMC's denoising to remove a little more noise.

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_01_1 (extrait début).demuxed.d2v")
LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)
Trim(0,451).QTGMC(InputType=1, EzDenoise=1)\
++Trim(542,659)
crop(6, 4, -4, -4)
Spline36Resize(some 16:9 dimension)

The different resolutions produced the following file sizes (CRF16 each time).
1280x720 = 22.9MB
1024x576 = 15.5MB
960x540 = 13.8MB

Upscaling to 720p would require a drop in encoding quality to not increase the file size, which might negate any benefit to upscaling before encoding anyway. It's only a small sample but the upscaling required a 45% increase in bitrate for the same CRF value.

Although I'll confess I saw one possible benefit in upscaling to 720p. It's possibly not typical, but that was when upscaling with nnedi3 like this:

nnedi3_rpow2(rfactor=2, cshift="spline36resize", fwidth=1280, fheight=720)

For reasons I don't understand yet, that fixed some of the remaining jaggies. I'm yet to notice any bad effects from upscaling that way. Upscaling with just Spline36Resize(1280,720) didn't fix the jaggies though. I've uploaded a few screenshots to show what I mean as I found it interesting. This is how each encode displays on my TV when running fullscreen (1080p).

For me it's hard to see a benefit to upscaling before encoding in respect to quality (you need to open the images to compare them as the forum software sharpens the thumbnails). Each to their own, I guess. We all do things differently.....

960x540, 13.8MB


1024x576, 15.5MB


1280x720, 22.9MB


In the last pic, look for the arrow I added, pointing to the guttering and compare it to the previous screenshots. I don't know how obvious it'll be on a computer monitor but on my TV it's easy to see it looks a little jagged in the first two screenshots but it's smoother in the third thanks to the nnedi3 resizing. It's no big deal as the jaggies are only there for a few frames and you'd need to be looking for them to see them. They come and go as the camera moves. It's just a little thing I noticed.....

[LMotlow]

I used MDegrain2, mainly 'cause the script was pretty much the same as yours with QTGMC and I had it running another project. All I had to do was change the input file in mine. Seems MDegrain2 smoothed motion a bit more (subtle ripples in the buildings in background, opening scene), but I know it doesn't address edge blips the way QTGMC does. Mainly posted to show it can be done for BD disc. SD doesn't usually look this clean upsampled.

[abolibibelot]

Wow, I wasn't hoping it would generate such a nice and rich exchange...

@LigH.de

After looking at your material:
This is in fact a progressive production, there are field pairs during the whole time. Just the pairing got messed. A part of this clip is "phase shifted".
This may not be the job for a classical deinterlacer, rather one of a good "field matcher" to fix only the pairing of combed szenes. Unfortunately, I am not experienced with this case; but I believe that TFM and TDecimate in a special parameter combination may help.

So, if I understand correctly, true interlacing is when every field is different, and a video (as is the case here apparently) can be encoded as "interlaced" when in fact it merely has pairs of (temporally) identical fields, which is much easier to process as opposed to full fledged interlacing where every field / half frame is different ? But then what causes the combing effect, if the fields in each pair are identical ?
How do you notice the phase shifting ? Does this mean that the order of the fields (top first / bottom first) gets reversed ? How does this happen ?

@hello_hello

For your sample.....
If you're choosing source type Film, it probably looks sharper for the reasons LigH.de suggested (field matching rather than de-interlacing) but it'll also be treating the source as film and removing every fifth frame (as it would for an NTSC 29.970fps to 23.976fps IVTC conversion). If you use the preview after the "Film/IVTC" de-interlacing option is applied, you'll see the total frame count is reduced which is not required and the output won't be smooth.

Indeed, that's right. (141863 frames natively, 113490 with IVTC with default settings.)

In this case you want the field matching without the frame removal. If you select Film as the de-interlacing method (varying field order) MeGUI will add something like the following to the script:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1).tdecimate()

Yes, except it's "order=1" (apparently that's for "top field first", if I choose "varying field order" it changes to "-1").

tfm() does the field matching, while tdecimate() does the frame removal. Before saving the script you need to remove the tdecimate part (the part in brown text). Check with the preview button after removing it, but that should fix the output without changing the frame count or de-interlacing (if you change anything in the script creator GUI, MeGUI will add it back and you'll need to remove it again).

OK, thanks. (Anyway I'm using this script creator as mere help, copying the relevant lines into the actual script I intend to use. Otherwise it insists on resizing, or setting the DAR to ITU 16:9 / 1.823 when it's clearly written 1.77 on the DVD box, or adding extra processing which is probably not needed like "ColorMatrix", and it's more annoying to uncheck / verify all this each time than to create the script separately in a text editor.)
What do you mean by "that should fix the output without changing the frame count [I get that part] or de-interlacing" ? You mean as opposed to using a proper deinterlacer like Yadif ? (Since, as you explained, field matching is not deinterlacing, strictly speaking, even if the purpose is roughly the same.) By the way, what does a deinterlacer do in such a case ?

MeGUI only analyses sections of the video and in this case it appears it's probably only the opening credits where the fields are out of order. When I analyse your sample though, MeGUI sees it as "partially interlaced", and in that case it is getting it wrong, but for PAL that looks interlaced due to the fields being out of order like yours, it probably always will.

I don't know if I'm doing something wrong but even for this sample it says "Source type: progressive".

For the record, if you select "Interlaced" and "Yadif with Bob" as the de-interlacing method in the script creator, that'll give you double frame rate output as QTGMC does by default (50fps for PAL). If you select "yadif with bob" and use the preview, and when you step through the frames one at a time it just looks like every frame is repeated (as it does for your sample) it's a fair indication it's not really interlaced. If each frame is different, then it probably is.
Of course there's always the possibility that some parts will be interlaced and other parts progressive (hence MeGUI's partially interlaced option), but generally for movies they're either one or the other.

Indeed that's what it does, but If I do that with the .m2v sample instead of the whole movie .d2v index file it stops right when the main character is supposed to appear, then it keeps displaying the same frame (if I go to the end and progress backwards it alternates between a repeated frame and a grey one) ; maybe that's because I chose to open with DirectShowSource -- I hadn't seen that DGIndex had also created a second .d2v file for the sample, if I open that instead it behaves just the same.
Actually each frame is repeated (temporally it's the same) but different (with jagged edges appearing slightly shifted from one frame to the next) ; so if I understand the process correctly, "Yadif with bob" displays the even and odd lines of a given (progressive) frame as two separate frames.

Something else you might want to try, given you mentioned QTGMC, is running it in progressive mode after the field matching (given the video will then be progressive), like this:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)
QTGMC(InputType=1)
I find QTGMC in progressive mode can do a nice job "cleaning up" video, but like all filtering it's not always needed, so you can apply it to just the sections where it is. For example, to use it to clean up the video behind the credits in your sample, but to re-encode the rest of the video without it, you could do something like this:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_01_1 (extrait début).demuxed.d2v")

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1)

Trim(0,451).QTGMC(InputType=1)\
++Trim(542,659)

crop(4, 4, -4, -4)
Spline36Resize(1012,568)

You mean "++Trim(452,659)", right ? (Thus stop applying QTGMC after the camera pan in this sample.)
Interesting suggestion, but if a deinterlacer isn't required is it really worth it ? QTGMC is generally very slow, maybe less so with that setting, but still it might significantly slow down the encode. And it would be quite a lot of work to go through the whole movie and select which frames would benefit from it.

I made three test encodes (CRF=20, preset=slow) :
- without QTGMC : 10,54FPS, 5664kbps
- with QTGMC(InputType=1) : 4,22FPS, 3487kbps
- with QTGMC(InputType=1, EzDenoise=1) : 3,88FPS, 3232kbps
Well, it's much slower indeed, but it very significantly improves the compressibility, and I can see that the begining of the fast camera pan is much cleaner.
The bitrates seem already very high with CRF=20 (especially without QTGMC), with CRF=18 or 16 it might get close to the original DVD bitrate (6500-6700kbps), or even more than that.

I'd probably crop and resize it the way I did in the above script and encode using at least CRF18, or if it's something I'd like to be close to perfect, maybe CRF16 if the file size doesn't get too out of control. Use the slowest x264 speed preset you can stand (I generally stick with Slow or Slower myself) and maybe try the x264 Film tuning to help retain a little more fine detail.

Why cropping (it only has very small black borders on the left and bottom, this had probably a negligeable effect on encoding efficiency) and why not keeping the original resolution and anamorphic DAR ? Like you said later :

The thing that tends to put me off upscaling is the increase in bitrate that it requires for almost no gain, in my experience.

From what I read in many discussions on specialized forums like this one, there is no benefit at all to be gained from upscaling when reencoding a video. Isn't there a consensus about that ? On the three screenshots, I see what you mean, this edge is indeed smoother on the 720p one, but there are also added artefacts below (possibly caused by the JPEG compression) and, more noticeably, the patterns on the red rooftops on the right are different, and seem less accurate (for instance the corner seems less defined, on the building which is at the angle of two streets, about 1/4 from the right). (Again, it could be an effect of the JPEG compression.)

[manono]

But then what causes the combing effect, if the fields in each pair are identical ?
How do you notice the phase shifting ?

The fields making up each frame aren't identical when phase-shifted.

If the capital letter is the top field and the small letter the bottom field, ordinarily progressive video goes like this:

A B C D E F G
a b c d e f g 

But when they're phase-shifted the fields are off by a frame:

A B C D E F G
b c d e f g h 

And that's what produces the interlacing. You determine what's going on by separating the fields or bobbing the video and then you see the fields have identical 'partners'. Then it just becomes a question of realigning the fields correctly. Which is where TFM comes in.

[LMotlow]

I didn't take the image comparisons to much to heart. None of them are posted in their original size. QTGMC would probably have been a better denoiser for edge cleaning, but MDegrain2 was a bit faster (and was already set up) so I gave it a whirl. All denoisers ain't alike. I go for cleaning over speed, but some folks want tiny or faster. You can't have everything. Anyway, MDegrain versions are some of several denoise components used by QTGMC.

Apparently you're after PC playback or playback other than disc. That being the case, 720x480, 1280x720, and 1920x1080 and the specs pertaining to formal disc formats are academic.

Upscaling SD usually doesn't get you very much, but in this case it doesn't look bad and looks better than the original in the cases presented. If this source had been VHS most would say forget about resizing, whether up or down. Yeah, you can make a progressive 25fps DVD from the source, but by the time you clean up the artifacts the images will look softer than the upscaled versions, where filtering and upscaling smooth and disperse many smaller defects, and the original image was workable to begin with. Besides, motion on 25p DVD is annoying. But most people don't pay attention to specs nowadays and have been 18 inches or closer to tiny uncalibrated windows for so long that they don't remember what real video displays look like. That is, the product being sought is not a movie medium, not a TV medium, but a computer medium. Each is a different viewing experience technically and visually.

Whatever.....

[hello_hello]

Yes, except it's "order=1" (apparently that's for "top field first", if I choose "varying field order" it changes to "-1").

I deliberately left it as varying field order, but if TFF gives better results, by all means use it.

OK, thanks. (Anyway I'm using this script creator as mere help, copying the relevant lines into the actual script I intend to use. Otherwise it insists on resizing, or setting the DAR to ITU 16:9 / 1.823 when it's clearly written 1.77 on the DVD box, or adding extra processing which is probably not needed like "ColorMatrix", and it's more annoying to uncheck / verify all this each time than to create the script separately in a text editor.)

Not that it matters how you do it, but for future reference all that default stuff can be changed. If you click on the Config button next to the Avisynth Profile dropdown box in the script creator, you can set your preferences in there and save them as an AVS template if you like.

Also for the record, MeGUI normally defaults to ITU resizing for DVDs. All DVDs will have 1.777 or 4:3 etc written on them even if they're ITU. As you've noticed, ITU resizing isn't exactly 16:9 or 4:3. It's a little wider. It's based on the concept that of the 720x576 resolution, only 704x576 constitutes the 16:9 or 4:3 aspect ratio. The extra 8 pixels either side don't count as part of the aspect ratio (you've probably seen DVD video with substantial black bars down each side?). You can change the default Input Aspect Ratio to non-ITU (exactly 16:9 or 4:3) in MeGUI's settings via the "use ITU aspect ratio" checkbox, but either way, you're free to change the Input Aspect Ratio in the script creator for each DVD as you see fit.
The rule of thumb I go by....... pretty much all 4:3 DVDs use ITU resizing. I'm not sure I've met many exceptions to that rule, regardless of whether the whole 720 width contains picture or if there's black bars down the sides.
For 16:9 DVDs if there's substantial back bars they probably use ITU resizing, if there's only a couple of pixels of black each side they probably don't. Most newer 16:9 DVDs would be non-ITU. There's no rule as such though, but that's probably a good guide if you're in doubt, and there's no way to know for sure aside from using your eyes. You can try to find straight on shots of perfectly round objects (ie a car wheel or a clock face) and see which resizing gets them closest to being round, although the difference in resizing methods is only about 2.5%.

What do you mean by "that should fix the output without changing the frame count [I get that part] or de-interlacing" ? You mean as opposed to using a proper deinterlacer like Yadif ? (Since, as you explained, field matching is not deinterlacing, strictly speaking, even if the purpose is roughly the same.) By the way, what does a deinterlacer do in such a case ?

I meant the field matching doesn't de-interlace.
For interlaced video each field constitutes a slightly different moment in time so they can't simply be combined without interlacing artefacts where there's movement (remember each field is every second scan line in a frame). A de-interlacer attempts to combine the two fields in such a way as to make a progressive frame.
https://en.wikipedia.org/wiki/Deinterlacing#Deinterlacing_methods
In the case of your video it doesn't require de-interlacing because it's already progressive, but the fields are out of order or phase shifted. TFM() matches them up again. Most de-interlacers will blur progressive video to some degree if you de-interlace it (QTGMC tends to be an exception) so it's a good idea to avoid doing so where possible.

I don't know if I'm doing something wrong but even for this sample it says "Source type: progressive".

Because MeGUI doesn't analyse the entire video, it's probably not seeing the sections where the fields are phase shifted. I'm not 100% sure about that one.
There's an option in MeGUI's settings labelled "source detector". You can change the default analysis settings there. I've never played with them myself but you could increase the analysis percentage etc. It might make the analysis more accurate, but it'll also take longer. I generally let MeGUI do the analysis and then look for myself.
There's also another option that effects analysis for NTSC called "auto force film" (under external program configuration). You can read about it here, but I disable force film myself. I doubt MeGUI would be foolish enough to use it for PAL, but if change the force film option you need to re-index the DVD before running a new analysis (if force film is used it changes the index file).

Indeed that's what it does, but If I do that with the .m2v sample instead of the whole movie .d2v index file it stops right when the main character is supposed to appear, then it keeps displaying the same frame (if I go to the end and progress backwards it alternates between a repeated frame and a grey one) ; maybe that's because I chose to open with DirectShowSource -- I hadn't seen that DGIndex had also created a second .d2v file for the sample, if I open that instead it behaves just the same.
Actually each frame is repeated (temporally it's the same) but different (with jagged edges appearing slightly shifted from one frame to the next) ; so if I understand the process correctly, "Yadif with bob" displays the even and odd lines of a given (progressive) frame as two separate frames.

DirectShow isn't frame accurate and it'll generally turn to crap when trying to navigate back and forth. It's fine for "start to finish" encoding without any complicated filtering but that's about it. DGDecode should be okay though, so I'm not sure why it resulted in the same oddness.

When you de-interlace with Yadif it attempts to combine fields together. For example:
ABCD
abcd

A and a are combined, B and b are combined, C and c are combined etc.

For Yadif with bob:
A and a are combined, a and B are combined, B and b are combined, b and C are combined, C and c are combined etc.

So for Yadif with bob you end up with twice as many progressive frames and the result will be smoother motion. Generally, that's how it'll be de-interlaced when you watch the DVD on a progressive TV. That's how QTGMC de-interlaces, which is why if you want to use QTGMC and keep a 25fps progressive output you need to delete every second frame after QTGMC has done it's thing.

QTGMC()
SelectEven()

That's my understanding. Someone may be able to correct me if I'm wrong. Although thinking about it I'm not sure why that effectively results in repeated/duplicate frames when using Yadif with Bob on progressive video. Maybe something to do with motion compensation when de-interlacing? Anyone know?

You mean "++Trim(452,659)", right ? (Thus stop applying QTGMC after the camera pan in this sample.)
Interesting suggestion, but if a deinterlacer isn't required is it really worth it ? QTGMC is generally very slow, maybe less so with that setting, but still it might significantly slow down the encode. And it would be quite a lot of work to go through the whole movie and select which frames would benefit from it.

Yes, I meant ++Trim(452,659). I see I encoded the sample with the frame number typo too. Oh well......
I was suggesting running QTGMC in progressive mode. It's up to you whether you think it's worth it. I think it makes a huge difference to the video in the opening credits, but the actual movie might be fine without it. I've only seen the sample.
Yes, adding multiple trim() to a script can be time consuming. It depends how fussy you are and how much time you want to put into it.

The bitrates seem already very high with CRF=20 (especially without QTGMC), with CRF=18 or 16 it might get close to the original DVD bitrate (6500-6700kbps), or even more than that.

It's hard to know if such a small sample would be indicative of the bitrate required for the whole movie.

Why cropping (it only has very small black borders on the left and bottom, this had probably a negligeable effect on encoding efficiency) and why not keeping the original resolution and anamorphic DAR ?
From what I read in many discussions on specialized forums like this one, there is no benefit at all to be gained from upscaling when reencoding a video. Isn't there a consensus about that ? On the three screenshots, I see what you mean, this edge is indeed smoother on the 720p one, but there are also added artefacts below (possibly caused by the JPEG compression) and, more noticeably, the patterns on the red rooftops on the right are different, and seem less accurate (for instance the corner seems less defined, on the building which is at the angle of two streets, about 1/4 from the right). (Again, it could be an effect of the JPEG compression.)

Yeah some of that could be the jpeg compression but they're all being resized twice, once to a resolution for encoding, then again to 1080p on playback, so there's bound to be small visible differences after the playback upscaling. The 720p example I posted was upscaled with nnedi3 by doubling the resolution then resizing down to 720p with Spline36. I just thought I'd try it for a comparison. Some parts look better, some parts not, I guess.

As far as resizing goes it's personal preference. Both of the TVs in this house have built in media players and both ignore aspect ratios in MKV/MP4 files. As does one of the Bluray players. I mostly use a PC for playback myself so it's not a problem for me, but as a result I just resize everything to square pixels so I know it'll always display correctly. Ideally, if you resize that way, you'd only resize the width, so for a 16:9 PAL DVD you'd keep the 576 height and resize the width to 1024. If you crop, the same principle applies, hence my sample being 1012x568, although I quite often end up resizing the height a little anyway.
I've found by resizing to 1024x576 with Spline36Resize it can have a slight sharpening effect which I don't mind, and the encode can look a little sharper than the original DVD when running fullscreen, but technically anamorphic encoding is the ideal method. If you do crop you'd then encode what's left as-is without any resizing while setting the original pixel aspect ratio. Cropping is personal preference. I prefer to crop and remove the crud myself. Also the media players in the TVs here don't overscan, and I have my TV set not to overscan most of the time anyway, and I'd rather see a cleanly cropped picture than crud around the edges of the screen. Personal preference.....

[LigH.de]

By "true interlaced" I mean:

• The 2nd field of the 1st frame was shot # milliseconds after the 1st field of the 1st frame
• The 1st field of the 2nd frame was shot # milliseconds after the 2nd field of the 1st frame
• The 2nd field of the 2nd frame was shot # milliseconds after the 1st field of the 2nd frame
• The 1st field of the 3rd frame was shot # milliseconds after the 2nd field of the 2nd frame
• The 2nd field of the 3rd frame was shot # milliseconds after the 1st field of the 3rd frame
• The 1st field of the 4th frame was shot # milliseconds after the 2nd field of the 3rd frame
• ... and so on, a regular temporal progress field-by-field, natural combing

[1a]-[1b]-[2a]-[2b]-[3a]-[3b]-[4a]-[4b]...

Progressive means:

• 1st and 2nd field of the 1st frame were shot at the same time.
• 1st and 2nd field of the 2nd frame were shot at the same time, # ms after the 1st frame.
• 1st and 2nd field of the 3rd frame were shot at the same time, # ms after the 2nd frame.
• 1st and 2nd field of the 4th frame were shot at the same time, # ms after the 3rd frame.
• ... and so on, a regular temporal progress frame-by-frame, no visible combing

[1a=1b]--[2a=2b]--[3a=3b]--[4a=4b]...

Phase shifted means:

• The 2nd field of the 1st frame and the 1st field of the 2nd frame were shot at the same time, both # ms after the 1st field of the 1st frame
• The 2nd field of the 2nd frame and the 1st field of the 3rd frame were shot at the same time, both # ms after the 1st field of the 2nd frame
• The 2nd field of the 3rd frame and the 1st field of the 4th frame were shot at the same time, both # ms after the 1st field of the 3rd frame
• ... and so on, a regular temporal progress frame-by-frame, just the wrong pairing, making artifical combing

[1a]--[1b=2a]--[2b=3a]--[3b=4a]--[4b...

More details in "exotisches Interlacing" by scharfis_brain (German original / zipped English translation by StainlessS).

[abolibibelot]

I deliberately left it as varying field order, but if TFF gives better results, by all means use it.

How can I know for sure which one gives the best results ?

Not that it matters how you do it, but for future reference all that default stuff can be changed. If you click on the Config button next to the Avisynth Profile dropdown box in the script creator, you can set your preferences in there and save them as an AVS template if you like.

I've seen that, but annoyingly some of the customized settings I created simply disappeared... I still haven't tamed that powerful though clunky beast.

you've probably seen DVD video with substantial black bars down each side?

Oh yes ! (But this one has an incorrect aspect ratio, among other apalling issues. Or maybe it's precisely an ITU-related issue... Faces look slightly squashed horizontally.)

You can change the default Input Aspect Ratio to non-ITU (exactly 16:9 or 4:3) in MeGUI's settings via the "use ITU aspect ratio" checkbox, but either way, you're free to change the Input Aspect Ratio in the script creator for each DVD as you see fit.

Normally when using a new software I check carefully the general options and configuration settings, but this one has so many intricate features in distinct areas that I hadn't yet bothered to go in there (except the "Extra program configuration" as is required on first startup).

Because MeGUI doesn't analyse the entire video, it's probably not seeing the sections where the fields are phase shifted. I'm not 100% sure about that one. There's an option in MeGUI's settings labelled "source detector". You can change the default analysis settings there. I've never played with them myself but you could increase the analysis percentage etc. It might make the analysis more accurate, but it'll also take longer. I generally let MeGUI do the analysis and then look for myself.

I tried turning that up to 11-- uh, to 100%, took about 10-15min for the whole movie, same result. ("Force film" disabled.)
When opened directly with DGIndex (standalone .exe within MeGUI install) and selecting "Forced film" it says : "Force Film Warning / It is almost always wrong to run Force Film on an MPEG1 stream, or a stream with a frame rate other than 29.970. Your stream is MPEG2 and has a frame rate of 25.000. Do you want to do it anyway?" Which confirms it should only apply to NTSC as you say.

DirectShow isn't frame accurate and it'll generally turn to crap when trying to navigate back and forth. It's fine for "start to finish" encoding without any complicated filtering but that's about it. DGDecode should be okay though, so I'm not sure why it resulted in the same oddness.

No, here I meant that when opening the .d2v it behaves normally again, just like when I first opened the whole movie's index file. (Sorry if it wasn't clear enough.)

That's how QTGMC de-interlaces, which is why if you want to use QTGMC and keep a 25fps progressive output you need to delete every second frame after QTGMC has done it's thing.
QTGMC()
SelectEven()
That's my understanding. Someone may be able to correct me if I'm wrong. Although thinking about it I'm not sure why that effectively results in repeated/duplicate frames when using Yadif with Bob on progressive video. Maybe something to do with motion compensation when de-interlacing? Anyone know?

I've read several times that adding "SelectEven" was a bad idea as it throws away half the temporal resolution. And from the few tests I made, I was surprised to see that it slowed down the process tremendously (almost by half, as I wrote in the thread linked above), while it doesn't reduce the bitrate / filesize as much as I would have thought.

I was suggesting running QTGMC in progressive mode. It's up to you whether you think it's worth it. I think it makes a huge difference to the video in the opening credits, but the actual movie might be fine without it. I've only seen the sample.

It's hard to say for sure (and I'm not very experienced in the field of analyzing fields) but apparently there are no other areas with such a strong combing effect as in the opening. I made another sample from a later scene with a camera pan :
http://we.tl/1OP7Jl01QZ
To me it's seems already clean after field matching.

It's hard to know if such a small sample would be indicative of the bitrate required for the whole movie.

Indeed, after loading it in Bitrate Viewer, it seems that it's mostly the camera pan in the middle which accounts for the relatively high bitrate. Most of the film is composed of steady shots. I found here a nice Avisynth feature for this purpose, which selects a given number of frames every given interval :
SelectRangeEvery(10000,100) # encodes 100 frames every 10000 frames, or 4s every 400s
In this test CRF=20 is around 3700kbps without QTGMC and 2700 with QTGMC(InputType=1,EZDenoise).
It's really hard to compare the quality, though, especially with a bug in the last version of VLC media player (or is it supposed to be a feature ?) which now generates screenshots of anamorphic videos in their native resolution (same behaviour as MPC). (I could still reinstall an older version just for that purpose, but oh well...)

The 720p example I posted was upscaled with nnedi3 by doubling the resolution then resizing down to 720p with Spline36. I just thought I'd try it for a comparison. Some parts look better, some parts not, I guess.

Why not resizing directly to 720p with either one of those filters ? What are their specificity, compared with Lanczos, bicubic (which I'm more familiar with) ? As you said earlier, the main problem with upscaling is that it requires a much higher bitrate for the same quality (or a loss of quality to keep the bitrate within the same range).


-----

Edit : After some more testing I found out that using QTGMC with "slow" preset or using "slower" preset without QTGMC amounts to roughly the same encoding framerate of 3.5fps (2009 computer), and I don't want it to go slower than that. QTGMC has a much more significant effect on the average bitrate (20% vs less than 2%), so I think I'll go with that, otherwise to keep approximately the same file size I'd have to increase the CRF (bitrate with CRF=22 preset "slower" without QTGMC is about the same as CRF=20 preset "slow" with QTGMC & EzDenoise), which would be (I guess) more detrimental to the encoding quality (than using "slow" instead of "slower"). So many moving parts, it's hard to make an informed choice with so little knowledge about the inner workings of these wizardly pieces of code stacked upon one another...

Here's a file obtained using SelectRangeEvery(10000,100), QTGMC(InputType=1) (no EzDenoise, it slows down below 3fps and I'd prefer to keep some grain, even though I have no idea how strong this denoiser is -- it reduces the bitrate by an extra 7% compared to QTGMC alone), encoded with CRF=22, preset "slow" and "Tune=Film" (60s, 15MB) :
http://we.tl/0DIWmyAoRn

[hello_hello]

I deliberately left it as varying field order, but if TFF gives better results, by all means use it.

How can I know for sure which one gives the best results ?

You'd probably just have to look. To be honest, I'm not sure it'd make any difference in this case.

Not that it matters how you do it, but for future reference all that default stuff can be changed. If you click on the Config button next to the Avisynth Profile dropdown box in the script creator, you can set your preferences in there and save them as an AVS template if you like.

I've seen that, but annoyingly some of the customized settings I created simply disappeared... I still haven't tamed that powerful though clunky beast.

One of MeGUI's annoyances is that it doesn't save most changes until you close it. So if it doesn't shut down properly for some reason.....
I'm in the habit of shutting it down and restarting it manually after major changes to AVS templates or encoder configurations etc, just to be safe. I also backup the MeGUI\allprofiles folder and backing up the settings.xml file in the MeGUI folder can't hurt.

you've probably seen DVD video with substantial black bars down each side?

Oh yes ! (But this one has an incorrect aspect ratio, among other apalling issues. Or maybe it's precisely an ITU-related issue... Faces look slightly squashed horizontally.)

I've seen stuff on DVDs with all sorts of aspect ratio oddness. Some obviously wrong, for example the opening credits for episode one having a slightly different aspect ratio to the opening credits for episode two etc, when they should be the same.
Assuming though that the DVD was created "properly" it might be a case of ITU vs non-ITU resizing. Try ITU to see if it looks better.

You can change the default Input Aspect Ratio to non-ITU (exactly 16:9 or 4:3) in MeGUI's settings via the "use ITU aspect ratio" checkbox, but either way, you're free to change the Input Aspect Ratio in the script creator for each DVD as you see fit.

Normally when using a new software I check carefully the general options and configuration settings, but this one has so many intricate features in distinct areas that I hadn't yet bothered to go in there (except the "Extra program configuration" as is required on first startup).

Yeah it has a bit of a learning curve in that respect. I think having the option to enable upscaling hidden away in the script creator profile configuration and disabled by default is a little silly, because new users often don't know it's there and assume MeGUI won't let you upscale, which includes resizing a DVD "up" to square pixel dimensions.

I tried turning that up to 11-- uh, to 100%, took about 10-15min for the whole movie, same result. ("Force film" disabled.)

That's interesting, because it kind of indicates MeGUI is getting the analysis correct, despite the phase shifted fields. Whether that's what's really happening, I don't know......
I have a PAL DVD with phase shifted fields here. I asked about it myself in a thread a while back (it wasn't all that long ago). It's when I first learned video that looks interlaced isn't necessarily interlaced. If I can remember which DVD it was I'll drag it out and play around. See what MeGUI says after analysing it. If I can find it, I'll report back.
Edit: Seems I posted a sample in the thread. After analysing it MeGUI reported it as being interlaced so that probably puts an end to the theory MeGUI might have been correctly seeing your sample as progressive despite the phase shifted fields.
I'm sure I've got a couple of other old PAL DVDs that are the same. I see if I can find one later.

I've read several times that adding "SelectEven" was a bad idea as it throws away half the temporal resolution. And from the few tests I made, I was surprised to see that it slowed down the process tremendously (almost by half, as I wrote in the thread linked above), while it doesn't reduce the bitrate / filesize as much as I would have thought.

I wasn't saying you should use SelectEven(), just explaining how QTGMC works (in case you weren't aware). Most playback devices support at least High Profile, Level 4.1 these days, so up to 60fps at 720p is fine.
Using SelectEven() would normally reduce the total encoding time as without it, there's twice as many frames to encode, although the QTGMC script is probably still the bottleneck. Taking the fps numbers from the thread you linked to, and assuming you were encoding a 25i video with 1000 frames.....

With SelectEven() the output would have been 1000 progressive frames at 2.6fps.
1000/2.6 = 385 seconds encoding time.
Without SelectEven() the output would have been 2000 progressive frames at 4fps.
2000/4 = 500 seconds encoding time.

There's probably a few reasons why an encode without SelectEven() doesn't increase the file size dramatically using CRF encoding.
The x264 encoder is frame rate aware. Take a video at 24fps and encode it twice, once with AssumeFPS(25,1) in the script and again without. The 25fps encode will probably have a slightly lower bitrate even if all the encoder settings are the same.
Left to it's own devices, MeGUI automatically changes the keyint settings according to the frame rate. For a 25fps video they'd be --keyint 250 & --min-keyint 25, while for 50fps they're --keyint 500 & --min-keyint 50. The minimum and maximum distance between keyframes doesn't change in respect to duration (1 sec min & 10 sec max) but the min & max number of frames doubles for 50fps.
While there's twice as many frames to encode at 50fps, the difference between one frame and the next would be less on average at 50fps.

I was suggesting running QTGMC in progressive mode. It's up to you whether you think it's worth it. I think it makes a huge difference to the video in the opening credits, but the actual movie might be fine without it. I've only seen the sample.

It's hard to say for sure (and I'm not very experienced in the field of analyzing fields) but apparently there are no other areas with such a strong combing effect as in the opening. I made another sample from a later scene with a camera pan :
http://we.tl/1OP7Jl01QZ
To me it's seems already clean after field matching.

QTGMC might quiet down the noise a bit, but it probably won't improve it dramatically.
More often than not, when I'm using QTGMC for noise removal, I use a faster preset. Something like:
QTGMC(InputType=1, Preset="Medium", EzDenoise=1)
The logic being, if there's any noise left behind and QTGMC stabilises it too much it doesn't look natural. Faster presets stabalise less and the remaining noise still looks more like noise than when it's fairly motionless. Sometimes, faster presets require more noise removal to "balance out". It depends on the source. ie
QTGMC(InputType=1, Preset="fast", EzDenoise=1.5)
Or even if it's removing enough noise without EZdenoise, try a faster preset.
QTGMC(InputType=1, Preset="Medium")

The main side effect of QTGMC for noise removal will be the possibility of colour banding. Noise helps prevent that.

I found here a nice Avisynth feature for this purpose, which selects a given number of frames every given interval :
SelectRangeEvery(10000,100) # encodes 100 frames every 10000 frames, or 4s every 400s
In this test CRF=20 is around 3700kbps without QTGMC and 2700 with QTGMC(InputType=1,EZDenoise).
It's really hard to compare the quality, though, especially with a bug in the last version of VLC media player (or is it supposed to be a feature ?) which now generates screenshots of anamorphic videos in their native resolution (same behaviour as MPC). (I could still reinstall an older version just for that purpose, but oh well...)

The SelectRangeEvery method was often used by programs for running a compression test back in the days of 2 pass, Xvid encoding. The only program I know of that'll still run a compression test is Staxrip (the 32 bit version, at least). The more you encode, obviously the more accurate it gets.
SelectRangeEvery(10000,100) mightn't be very accurate for 50fps though, even if it's fine for 25fps. The x264 encoder tends to put a keyframe at the beginning of each scene change, then when using MeGUI by default the maximum number of frames to the next keyframe is probably 500 for 50fps (250 for 25fps). More often than not, another scene change will occur before the 500 frame limit and another keyframe will be used, but for your example you're effectively forcing a scene change and a keyframe every 100 frames, which may effect the result. Just something to be aware of if you think the result isn't accurate enough for 50fps. I recall a thread at doom9 where someone said SelectRangeEvery(1440,72) was giving pretty accurate results for 24fps video, but it also encodes a larger percentage of it.

For the VLC problem, if your monitor has a high enough resolution to display the video without resizing it, or when comparing the video running fullscreen, try Alt+PrintScreen instead and paste that into an image editor. Or can you run two instances of VLC and switch between them?
Or even better, make copies of the same script with different filtering and open each script in a different instance of MPC-HC for comparison (add some temporary resizing if you want the correct aspect ratio). MPC-HC will still try to play the video at the correct speed which can get ugly when slow filtering is used, so for comparison purposes it pays to slow everything right down when using slow filtering by sticking something like AssumeFPS(1,1) at the very end of the script. Open a script with MPC-HC, then immediately pause the player so it waits for the first frame, and from there you should be able to navigate around to compare scripts, even if it's slow.

The 720p example I posted was upscaled with nnedi3 by doubling the resolution then resizing down to 720p with Spline36. I just thought I'd try it for a comparison. Some parts look better, some parts not, I guess.

Why not resizing directly to 720p with either one of those filters ? What are their specificity, compared with Lanczos, bicubic (which I'm more familiar with) ? As you said earlier, the main problem with upscaling is that it requires a much higher bitrate for the same quality (or a loss of quality to keep the bitrate within the same range).

nnedi3 only upscales by 2x or 4x etc but it's supposed to be fairly good, and some resizers downscale with less ringing than upscaling (apparently) so I thought I'd try it. It's probably better for animation than video though. Normally if I was upscaling I'd just use Spline36Resize myself, but I never upscale by a large amount.

After some more testing I found out that using QTGMC with "slow" preset or using "slower" preset without QTGMC amounts to roughly the same encoding framerate of 3.5fps (2009 computer), and I don't want it to go slower than that. QTGMC has a much more significant effect on the average bitrate (20% vs less than 2%), so I think I'll go with that, otherwise to keep approximately the same file size I'd have to increase the CRF (bitrate with CRF=22 preset "slower" without QTGMC is about the same as CRF=20 preset "slow" with QTGMC & EzDenoise), which would be (I guess) more detrimental to the encoding quality (than using "slow" instead of "slower"). So many moving parts, it's hard to make an informed choice with so little knowledge about the inner workings of these wizardly pieces of code stacked upon one another...

All you can do is play around until you find what works/looks good for you.

Here's a file obtained using SelectRangeEvery(10000,100), QTGMC(InputType=1) (no EzDenoise, it slows down below 3fps and I'd prefer to keep some grain, even though I have no idea how strong this denoiser is -- it reduces the bitrate by an extra 7% compared to QTGMC alone), encoded with CRF=22, preset "slow" and "Tune=Film" (60s, 15MB) :
http://we.tl/0DIWmyAoRn

I can see a tiny bit of encoder blocking in places but I suspect that's in the source and not the encode. Other than that it looks okay to me, but not having seen the source, I assume it looks as good, if not better, than the original?

I wonder if you would save a little bitrate by cropping as I did and encoding at 716x568?

[abolibibelot]

@LigH.de

More details in "exotisches Interlacing" by scharfis_brain (German original / zipped English translation by StailnessS).

Thank you for these precisions, it's begining to be clear. It's just that I thought each field was displayed sequentially with the same time interval no matter what pair it belonged to (which would negate any shifting), whereas apparently each pair is displayed as if it was a complete frame (at least on computer playback). I'll have to read more on the subject.
(Oddly I can download that file but I can no longer see my own files in Mediafire... Maybe that damn Flash plugin which was disabled because of some security breach, then required to restart Firefox which I haven't done yet. Damn, there's always something...)


@hello_hello

One of MeGUI's annoyances is that it doesn't save most changes until you close it. So if it doesn't shut down properly for some reason.....
I'm in the habit of shutting it down and restarting it manually after major changes to AVS templates or encoder configurations etc, just to be safe. I also backup the MeGUI\allprofiles folder and backing up the settings.xml file in the MeGUI folder can't hurt.

Indeed, it lost settings after crashing, and then I noticed the settings newly created didn't appear where they were supposed to right away (for example in the allprofiles\x264 folder) ; now I understand it's (annoyingly indeed) designed that way. And thanks for the backup advice.

Assuming though that the DVD was created "properly" it might be a case of ITU vs non-ITU resizing. Try ITU to see if it looks better.

But if that were the case shouldn't it appear by default with the correct AR in VLC or other softwares which are supposed to respect the AR flags ?

That's interesting, because it kind of indicates MeGUI is getting the analysis correct, despite the phase shifted fields. Whether that's what's really happening, I don't know......

But you said it gave you a different result for the same input file (= the first sample I uploaded), didn't you ?

I asked about it myself in a thread a while back (it wasn't all that long ago). It's when I first learned video that looks interlaced isn't necessarily interlaced.

Well, that's where I am right now, gives me some hope, as your brain obviously didn't explode from the painstakingly acquired knowledge ! ;^p (But yeah, it wasn't long ago indeed.)

Using SelectEven() would normally reduce the total encoding time as without it, there's twice as many frames to encode, although the QTGMC script is probably still the bottleneck. Taking the fps numbers from the thread you linked to, and assuming you were encoding a 25i video with 1000 frames.....

With SelectEven() the output would have been 1000 progressive frames at 2.6fps.
1000/2.6 = 385 seconds encoding time.
Without SelectEven() the output would have been 2000 progressive frames at 4fps.
2000/4 = 500 seconds encoding time.

I don't remember the details (I kinda abandonned this project for now, the replies I got in that thread were neither encouraging, nor detailed enough for me to actually learn something useful and do further testing), but yes, that might be the explanation. I ran the encode with Avidemux and probably didn't pay attention to the "Processed frames" or "Total frames" informations. I kept the test files so looking at their timestamps (difference between creation date and modification date) it apparently took 3m06s to encode with QTGMC at default doubled framerate (1517KB, 15s160, 819kbps, 50fps) and 2m24s to encode with QTGMC(SelectEven) (1355KB, 15s160, 732kbps, 25fps).

The x264 encoder is frame rate aware. Take a video at 24fps and encode it twice, once with AssumeFPS(25,1) in the script and again without. The 25fps encode will probably have a slightly lower bitrate even if all the encoder settings are the same.

I'm not sure I understand what you mean here. How does "AssumeFPS(25,1)" affect x264's behaviour ? Maybe you mean it adapts its algorithms and, for instance, looks for similarities in a larger amount of frames than it would with a (genuine or "scripted") lower framerate ?

Left to it's own devices, MeGUI automatically changes the keyint settings according to the frame rate. For a 25fps video they'd be --keyint 250 & --min-keyint 25, while for 50fps they're --keyint 500 & --min-keyint 50. The minimum and maximum distance between keyframes doesn't change in respect to duration (1 sec min & 10 sec max) but the min & max number of frames doubles for 50fps.
While there's twice as many frames to encode at 50fps, the difference between one frame and the next would be less on average at 50fps.

Yes, that totally makes sense intuitively. And so the keyframes are the ones that get fully encoded and account for a large part of the bitrate.
So, using "AssumeFPS(50,1)" for a 25fps video would significantly improve the compression efficiency I suppose ? (But I read here that it also affects the audio, everything must be speeded up two-fold, bad idea.) Or increasing the "keyint" values ? How does that affect the resulting quality, or playability ?

QTGMC might quiet down the noise a bit, but it probably won't improve it dramatically.
More often than not, when I'm using QTGMC for noise removal, I use a faster preset. Something like:
QTGMC(InputType=1, Preset="Medium", EzDenoise=1)
The logic being, if there's any noise left behind and QTGMC stabilises it too much it doesn't look natural. Faster presets stabalise less and the remaining noise still looks more like noise than when it's fairly motionless. Sometimes, faster presets require more noise removal to "balance out". It depends on the source. ie
QTGMC(InputType=1, Preset="fast", EzDenoise=1.5)
Or even if it's removing enough noise without EZdenoise, try a faster preset.
QTGMC(InputType=1, Preset="Medium")

So, what is QTGMC's default preset in "InputType=1" (progressive) mode ? Did you see any of that unnaturally-stabilized-noise effect on the reencoded sample ? (The source has very moderate noise/grain to begin with, I'd say.) And do you know how EzDenoise compares with dedicated denoisers like SMDegrain or MCTemporalDenoise (which I tried recently following recommandations in that thread) ? For the last example, did you mean "Preset="Faster"", or did you mean "try a slower preset" ? (Obviously "medium" is slower than "fast".)

The main side effect of QTGMC for noise removal will be the possibility of colour banding. Noise helps prevent that.

Confused again... did you mean "EzDenoise helps prevent that" ?

SelectRangeEvery(10000,100) mightn't be very accurate for 50fps though, even if it's fine for 25fps.

I used that script, with "SelectRange" at the end, after field matching (so the 100 frames should mean 4 seconds) :

LoadPlugin("C:\Program Files\MeGUI_2507_x86\tools\dgindex\dgdecode.dll")
LoadPlugin("C:\Program Files\MeGUI_2507_x86\tools\avisynth_plugin\NicAudio.dll")
LoadPlugin("C:\Program Files\MeGUI_2507_x86\tools\avisynth_plugin\TIVTC.dll")
global MeGUI_darx = 16
global MeGUI_dary = 9
Vid = MPEG2Source("E:\FullDisc\DES FLEURS POUR ALGERNON\VIDEO_TS\VTS_01_1.d2v")
Aud = NicAC3Source("E:\FullDisc\DES FLEURS POUR ALGERNON\VIDEO_TS\VTS_01_1 T80 2_0ch 384Kbps DELAY 0ms.ac3")
VidFM = Vid.tfm(order=-1)
VidCleaned = VidFM.QTGMC(InputType=1)
Mix = AudioDub(VidCleaned, Aud).SelectRangeEvery(10000,100)
Return(Mix)

The x264 encoder tends to put a keyframe at the beginning of each scene change, then when using MeGUI by default the maximum number of frames to the next keyframe is probably 500 for 50fps (250 for 25fps). More often than not, another scene change will occur before the 500 frame limit and another keyframe will be used, but for your example you're effectively forcing a scene change and a keyframe every 100 frames, which may effect the result. Just something to be aware of if you think the result isn't accurate enough for 50fps.

Alright, interesting point. So it would be better to select an interval length closer to the encoder's key frame interval ? (And even then the might be scene changes within those randomly selected cuts, as it happens in my sample.)
I ran a full length encode last night and noted that the encoding rate was faster than for the sample (partly because I wasn't using the computer at all, but also probably because of that effect you mention, i.e. less key frames were required.)

For the VLC problem, if your monitor has a high enough resolution to display the video without resizing it, or when comparing the video running fullscreen, try Alt+PrintScreen instead and paste that into an image editor. Or can you run two instances of VLC and switch between them?

When I make screecaps that way (ALT+PrintScreen) the video display appears completely black (using XnView, same result with MSPaint). It works with MPCHC but with this one I still don't get how to accurately set the aspect ratio and window size for anamorphic videos (granted it's a 2008 version, maybe it's been improved in that regard). Now let's try SMPlayer : same as VLC, it only captures the GUI's frame with a black rectangle where the video should be (and it doesn't seem to allow screen grabbing by itself, there's a button shaped as a camera but greyed out).
Trying to compare two videos playing at the same time is way trickier, maybe it's doable with more experience, but for now I can only see very obvious differences that way.

Or even better, make copies of the same script with different filtering and open each script in a different instance of MPC-HC for comparison (add some temporary resizing if you want the correct aspect ratio). MPC-HC will still try to play the video at the correct speed which can get ugly when slow filtering is used, so for comparison purposes it pays to slow everything right down when using slow filtering by sticking something like AssumeFPS(1,1) at the very end of the script. Open a script with MPC-HC, then immediately pause the player so it waits for the first frame, and from there you should be able to navigate around to compare scripts, even if it's slow.

Interesting suggestion, if a little more convoluted, I'll try. (But as I wrote above, I can't get accurate frame size and aspect ratio with MPCHC.)

I can see a tiny bit of encoder blocking in places but I suspect that's in the source and not the encode. Other than that it looks okay to me, but not having seen the source, I assume it looks as good, if not better, than the original?

Unfortunately MeGUI doesn't seem to permit a direct stream copy, while VirtualDubMod produced a 889MB file when saving to MKV with "Direct stream copy", using the same AVS script (according to MediaInfo it saved the video to ATI YVU12 4:2:0 Planar). With Avidemux + AVSProxy : it refused to save to MPEG-PS, I chose MPEG video, the resulting file is also 889MB but unreadable (and not recognized by MediaInfo). Of course Avisynth serves decoded frames and not the native video stream, silly me. So I loaded the VOBs and remade the edits manually in Avidemux and finally managed to save a proper native MPEG2 sample. Here, if you care to verify those blocking issues (56MB) :
http://we.tl/7qICb10kN7
(Or please explain what and where I should look.)

I wonder if you would save a little bitrate by cropping as I did and encoding at 716x568?

Hard to say without testing, it would no longer be "mod16", which is said to have a detrimental effect on the encoding efficiency. (But 1920x1080 isn't mod16 either, yet the powers-that-be chose that resolution as a standard...)

[hello_hello]

Assuming though that the DVD was created "properly" it might be a case of ITU vs non-ITU resizing. Try ITU to see if it looks better.

But if that were the case shouldn't it appear by default with the correct AR in VLC or other softwares which are supposed to respect the AR flags ?

All software, or by far the majority of it, displays DVD video as exactly 4:3 or 16:9. There's no way for the software to know if it's ITU or not. As far as I know any aspect ratio flags simply say 4:3 or 16:9. That's why it's so much fun.
You can decide for yourself when using MPC-HC. A tap on the "6" key will stretch the width to something pretty close to ITU (the aspect ratio is displayed in the status bar). The "5" key resets it.

That's interesting, because it kind of indicates MeGUI is getting the analysis correct, despite the phase shifted fields. Whether that's what's really happening, I don't know......

But you said it gave you a different result for the same input file (= the first sample I uploaded), didn't you ?

Yeah, thinking about it, it sees the sample as partially interlaced, so why it doesn't see the whole video as the same, I'm not sure.

The x264 encoder is frame rate aware. Take a video at 24fps and encode it twice, once with AssumeFPS(25,1) in the script and again without. The 25fps encode will probably have a slightly lower bitrate even if all the encoder settings are the same.

I'm not sure I understand what you mean here. How does "AssumeFPS(25,1)" affect x264's behaviour ? Maybe you mean it adapts its algorithms and, for instance, looks for similarities in a larger amount of frames than it would with a (genuine or "scripted") lower framerate ?

AssumeFPS(25,1) just changes the frame rate from 24fps to 25fps (for my previous example) by speeding it up a little. The number of frames doesn't change. The encoder knows what the frame rate is. Have a look under "standard error stream" in MeGUI's log file. I don't really understand it myself, but CRF encoding tries to maintain a constant visual quality for a given CRF value, so I assume if the frames are going by a little faster then they can be slightly lower quality without you noticing. Probably in areas where there's motion. That's just my assumption as to how it works, but I'm sure the same video will have a slightly lower bitrate at 25fps than 24fps for a given CRF value. Not a huge difference, but there is a difference.
I just tried it with your original sample, no filtering except tfm(). CRF18, slow preset, tune film. 13.8MB at 25fps, 14.2MB at 24fps.

So, using "AssumeFPS(50,1)" for a 25fps video would significantly improve the compression efficiency I suppose ? (But I read here that it also affects the audio, everything must be speeded up two-fold, bad idea.) Or increasing the "keyint" values ? How does that affect the resulting quality, or playability ?

AssumeFPS changes the speed of the video, so no, you wouldn't want to do that. Changing the keyint settings does effect compressibility and you can change them independantly of the frame rate. In fact the x264 defaults are --keyint 250, --min-keyint 25 for all frame rates. I haven't done a lot of testing, but for 100% Bluray compliance you need to use --keyint 24 and --min-keyint 1. If you use those keyint settings for a given CRF value, then run the same encode again with --keyint 240 and --min-keyint 24 (same frame rate), the file size can decrease by something like 10%.
The closer the keyframes are together the faster a device can navigate through a video, but --keyint shouldn't effect normal playback unless a device has particular restrictions. I just let MeGUI set keyint according to the frame rate and I've not had any playback problems.

So, what is QTGMC's default preset in "InputType=1" (progressive) mode ? Did you see any of that unnaturally-stabilized-noise effect on the reencoded sample ? (The source has very moderate noise/grain to begin with, I'd say.) And do you know how EzDenoise compares with dedicated denoisers like SMDegrain or MCTemporalDenoise (which I tried recently following recommandations in that thread) ? For the last example, did you mean "Preset="Faster"", or did you mean "try a slower preset" ? (Obviously "medium" is slower than "fast".)

I'm pretty sure the default preset is slow, the same as de-interlacing mode, so I meant a faster preset for less stabilisation. ie medium.
I didn't see any unnaturally stabilised noise in your sample as it was mostly gone.
I generally prefer QTGMC for denoising to SMDegrain or other scripts (I've barely used MCTemporalDenoise myself) for lower quality video as it stabalises the picture too. I've encoded a few videos where QTGMC caused banding and SMDegrain didn't for reasons I don't understand, so I went with SMDegrain.
Here's some old examples for you. I created this thread back when I first started experimenting with QTGMC for denoising. Have a look at the samples attached to this post. A denoising torture test. Every time I look I prefer a different one, but it's always a choice between one of the QTGMC encodes and TemporalDegrain. The three QTGMC encodes were all done with the slow preset at the time, but if you look at the QTGMC encode without any extra denoising you'll probably see what I'm referring to when it comes to "stable noise". TemporalDegrain no doubt removed the most noise but it didn't hide the single frame blemishes or stabilise as much as QTGMC. The "shimmering" from the original de-interlacing is reduced more by QTGMC (watch the building frame in the background about 10 seconds in).
Or try the samples attached to the opening post. The original video is included. If you re-encode it with QTGMC in progressive mode without EzDenoise you'll probably see the faster presets stabilise the noise less and I find it looks a bit more natural when it's not completely removed. I'd probably do the QTGMC encodes a little differently to the samples attached to the post now, but it's probably still a good sample for experimenting with noise removal. Have a play around.

This is my latest "go to" script for noise removal, especially for HD, but it's quite slow (MP_Pipeline can give a little speed boost). The battle between removing noise and blurring is never ending.

MP_Pipeline("""
LoadPlugin("C:\Program Files\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\video.lwi")
QTGMC(InputType=1, Preset="medium", EzDenoise=1.5)
### prefetch: 16, 0
### ###
""")
LSFMod(Strength=75)
gradfun3()

The main side effect of QTGMC for noise removal will be the possibility of colour banding. Noise helps prevent that.

Confused again... did you mean "EzDenoise helps prevent that" ?

No, any denoising will increase the chances of banding, so it's more likely to happen with EzDenoise as noise itself helps prevent banding. The higher the bitrate (or the lower the CRF value) the less likely there'll be banding, and using the dither plugin to add dithering (noise you can't see) can help prevent it too. Hence gradfun3() at the end of the above script.

So it would be better to select an interval length closer to the encoder's key frame interval ? (And even then the might be scene changes within those randomly selected cuts, as it happens in my sample.)
I ran a full length encode last night and noted that the encoding rate was faster than for the sample (partly because I wasn't using the computer at all, but also probably because of that effect you mention, i.e. less key frames were required.)

I haven't played around enough to know what would give the most accurate results, Probably a third or half would be fine because the default max keyint is about 10 seconds but most scene changes happen more frequently than that. I just mentioned it in case the range you're using doesn't prove to be accurate. I rarely worry about that sort of thing myself as I tend to just pick a CRF value and let the file size be whatever it needs to be.

When I make screecaps that way (ALT+PrintScreen) the video display appears completely black (using XnView, same result with MSPaint). It works with MPCHC but with this one I still don't get how to accurately set the aspect ratio and window size for anamorphic videos (granted it's a 2008 version, maybe it's been improved in that regard). Now let's try SMPlayer : same as VLC, it only captures the GUI's frame with a black rectangle where the video should be (and it doesn't seem to allow screen grabbing by itself, there's a button shaped as a camera but greyed out).
Trying to compare two videos playing at the same time is way trickier, maybe it's doable with more experience, but for now I can only see very obvious differences that way.

Printscreen probably only works with some renderers. I've only used MPC-HC for so long.....
I don't compare video while it's running much, but it's easy enough to open two scripts with two instances of MPC-HC, pause them both, maximise them, and use the Navigate menu to have both display a specific frame. Then you can switch between pictures without needing to go to the trouble of saving pictures. MPC-HC won't display anamorphic scripts with the correct aspect ratio, or save anamorphic screenshots with the correct aspect ratio, but the "View/Video Frame/ Override Aspect Ratio" menu lets you force a standard aspect ratio such as 4:3 or 16:9. Or put some temporary resizing in the scripts....

So I loaded the VOBs and remade the edits manually in Avidemux and finally managed to save a proper native MPEG2 sample. Here, if you care to verify those blocking issues (56MB) :
http://we.tl/7qICb10kN7
(Or please explain what and where I should look.)

I might have made you go to that trouble for nothing, because I remembered yesterday a bit of blocking standing out in one scene but today I can't see it or remember where it was. Sometimes things stand out, but when you look again, they don't.
It might have been something like this where there's a bit of movement and the guys shoulder is starting to look a bit "blocky" and I just happened to notice it when the video was running fullscreen on my TV. As it was in these screenshots, and I'm sitting fairly close to it, so it's probably getting close to nitpicking. Sometimes things like that jump out, sometimes not. It could be the denoising as much as the bitrate.

Original:


Encode:

I wonder if you would save a little bitrate by cropping as I did and encoding at 716x568?

Hard to say without testing, it would no longer be "mod16", which is said to have a detrimental effect on the encoding efficiency. (But 1920x1080 isn't mod16 either, yet the powers-that-be chose that resolution as a standard...)

The mod16 thing makes almost no difference for x264 encoding. I've run comparison encodes and it's just not worth thinking about. I'm yet to find a hardware player where non-mod16 dimensions will cause problems (although mod2 widths can sometimes be an issue for DirectShow decoding), but I try to be paranoid and stick to mod4 widths and that's about it. Other than that I encode using whatever dimensions I like. Even much iTunes video is mod4 or mod2.