QTGMC/IVTC issues on AVISynth/AvspMod

[Fraugster]

Hi! This is my second post about a project I've been having issues with lately. I've been working on sprucing up the Archival of Evangelion DVD release, which is the original aired episodes unaltered for those who didn't know. I just set AVISynth back up again and I have been having issues with QTGMC. On the placebo preset everything is literally dark.


[Attachment 49718 - Click to enlarge]

This is what it is supposed to look like:


[Attachment 49719 - Click to enlarge]

On that note, the other issue I am having with AVISynth is encoding. I cannot for the life of me figure out how to export my scripts! At a very basic level I'd like to export to an uncompressed AVI and encode it using VirtualDub or Premiere. But the other half of me would like to be able to have my encodes and if I ever needed to re-encode it for some reason I'll have the AVSs and the AVIs laying around ready for me to do so. If anyone has some insight to help me that would be phenomenal!

[jagabo]

There is some kind of problem (maybe 10bit/8bit conversion errors) in the noise processing at "very slow" and "placebo" (this may be limited to the 64 bit version, and may be a compatibility issue with one of the support filters). If you set NoiseProcess=0 you won't get the dark picture. Conversely, if you enable NoiseProcess with faster presets you will get a dark picture.

To encode your video you can use any encoder that supports AviSynth script. For example, open the script in VirtualDub with File -> Open Video File, select a codec, and encode away. I usually use the x264 command line encoder.

[Fraugster]

There is some kind of problem (maybe 10bit/8bit conversion errors) in the noise processing at "very slow" and "placebo" (this may be limited to the 64 bit version, and may be a compatibility issue with one of the support filters). If you set NoiseProcess=0 you won't get the dark picture. Conversely, if you enable NoiseProcess with faster presets you will get a dark picture.

To encode your video you can use any encoder that supports AviSynth script. For example, open the script in VirtualDub with File -> Open Video File, select a codec, and encode away. I usually use the x264 command line encoder.

Thank you very much kind stranger! That fixed the dark video. But I've been noticing some issues with ghosting like this:


[Attachment 49722 - Click to enlarge]

My current script is as follows:

AVISource("Ep 1.avi", audio=true)
AssumeTFF()
QTGMC(Preset="Placebo", NoiseProcess=0)
SelectEven()
TDecimate(Mode=1)
AssumeFPS(23.976)
Crop(12, 4, 698, 0, align=False)
LanczosResize(640,480)

[jagabo]

Are you sure that's not in the original video? Use Bob() instead of QTGMC() and see if it shows the same blending. Upload a sample of your source (not filtered or reencoded).

[Fraugster]

Are you sure that's not in the original video? Use Bob() instead of QTGMC() and see if it shows the same blending. Upload a sample of your source (not filtered or reencoded).

I'm fairly certain that its a ghost and not a fade. I tried bob, same issue still. I'm comparing to the blu-ray version but the blu-ray is from a different master so it could've been changed. But there are plenty of other times in the video ghosting still occurs. My method for ripping the DVD was to use MakeMKV on the ISOs I downloaded. MakeMKV just removes the DVD restrictions and makes the DVD title into an MKV without any data loss. I then went to virtualdub and trimmed each title down to individual episodes and exported those as uncompressed AVI.

Here is my sample, this is from my uncomp AVI of episode 1:
Ep 1 Sample.avi

[jagabo]

For some reason downloading from Videohelp is very slow today. I've only seen a few seconds of your source so far and there's very little motion in what I've seen. But it appears to be a PAL-to-NTSC conversion with field blending. Usually, the best way to deal with this is QTGMC() followed by SRestore().

But you're doing this all wrong. As I mentioned in your other thread you should be using DgIndex with the original VOB files. Mark the start and end of the episode and save an index file (.d2v) and demuxed audio (usually .AC3). Then use Mpeg2Source() to open the index file and filter from there. You can also use DgIndex to demux a short sample of the video as an .M2V file. That will give you a much smaller video and there will be no possibility of any changes from your conversion to AVI (VirtualDub has problems with interlaced MPEG 2 sources).

[Fraugster]

For some reason downloading from Videohelp is very slow today. I've only seen a few seconds of your source so far and there's very little motion in what I've seen. But it appears to be a PAL-to-NTSC conversion with field blending. Usually, the best way to deal with this is QTGMC() followed by SRestore().

But you're doing this all wrong. As I mentioned in your other thread you should be using DgIndex with the original VOB files. Mark the start and end of the episode and save an index file (.d2v) and demuxed audio (usually .AC3). Then use Mpeg2Source() to open the index file and filter from there. You can also use DgIndex to demux a short sample of the video as an .M2V file. That will give you a much smaller video and there will be no possibility of any changes from your conversion to AVI (VirtualDub has problems with interlaced MPEG 2 sources).

You got it! I'm installing DVDFab and DgIndex. I'm assuming I need both correct? DVDFab to take the VOBs off and use DgIndex to mark in and out and make index files for each episode? Then throw that into AVISynth as a Mpeg2Source? Once I get that all done will you need another sample, I'm assuming youd like the .m2v sample from DgIndex? Also I haven't said this yet but thank you for taking the time out of your day to help me, it really means a lot, and I appreciate the help immensely.

[jagabo]

You don't need DVD Fab (or DVD Decrypter) if you are starting with an ISO image or a home made disc. It's only needed for decrypting commercial discs. If you have an already decrypted VIDEO_TS folder you can just open the VOB files from there. If you have an ISO image you should mount the ISO as a virtual dvd drive (Win10 can do this natively, earlier versions of Windows need something like WinCDEmu (open source virtual CD/DVD/BD emulator).

It's easiest to use DgIndex to mark individual episodes. It will save the index file and corresponding audio will be saved. You must leave the VOB files where they are because the actual video still comes from there.

I have the full clip you uploaded now. It looks even worse than I originally thought. It may be an NTSC-to-PAL conversion made from an NTSC-to-PAL convesion, both with field blending. Or possibly someone just applied a very strong temporal filter to reduce noise. I need a smooth panning shot to say for sure.

Your script will look something like:

Mpeg2Source("filename.d2v", CPU2="ooooxx") # note d2v, not vob or mpg, "ooooxx" = apply deringing filter
QTGMC()
SRestore(frate=25.0)

[Fraugster]

You don't need DVD Fab (or DVD Decrypter) if you are starting with an ISO image or a home made disc. It's only needed for decrypting commercial discs. If you have an already decrypted VIDEO_TS folder you can just open the VOB files from there. If you have an ISO image you should mount the ISO as a virtual dvd drive (Win10 can do this natively, earlier versions of Windows need something like WinCDEmu (open source virtual CD/DVD/BD emulator).

It's easiest to use DgIndex to mark individual episodes. It will save the index file and corresponding audio will be saved. You must leave the VOB files where they are because the actual video still comes from there.

I have the full clip you uploaded now. It looks even worse than I originally thought. It may be an NTSC-to-PAL conversion made from an NTSC-to-PAL convesion, both with field blending. Or possibly someone just applied a very strong temporal filter to reduce noise. I need a smooth panning shot to say for sure.

Your script will look something like:

Mpeg2Source("filename.d2v", CPU2="ooooxx") # note d2v, not vob or mpg, "ooooxx" = apply deringing filter
QTGMC()
SRestore(frate=25.0)

I can get you a pan sample if you'd like once I get DgIndex up and running. There's a spot in episode 1 of just that. A few actually. May I ask why restore to 25 frames and not 23.976?

[manono]

May I ask why restore to 25 frames and not 23.976?

Because a PAL master was used for the NTSC DVDs. You can slow it to film speed afterwards using AssumeFPS(23.976) and you'll have to slow the audio as well.

You can probably decrypt the episodes directly from the DVD using DVD Decrypter, rather than decrypting the whole thing and separating out the episodes in DGIndex. I don't have the DVD so I'm not entirely sure how it's set up, but first change DVD Decrypter from the default File Mode to IFO Mode in the 'Mode' tab. If the individual episodes don't then show up, go into Tools->Settings->IFO Mode->File Splitting->By VOB ID.

I worked on this series a very long time ago and remember it as being a godawful mess. The NTSC DVDs are horrible and probably shouldn't be used as a source.

[Fraugster]

May I ask why restore to 25 frames and not 23.976?

Because a PAL master was used for the NTSC DVDs. You can slow it to film speed afterwards using AssumeFPS(23.976) and you'll have to slow the audio as well.

You can probably decrypt the episodes directly from the DVD using DVD Decrypter, rather than decrypting the whole thing and separating out the episodes in DGIndex. I don't have the DVD so I'm not entirely sure how it's set up, but first change DVD Decrypter from the default File Mode to IFO Mode in the 'Mode' tab. If the individual episodes don't then show up, go into Tools->Settings->IFO Mode->File Splitting->By VOB ID.

I worked on this series a very long time ago and remember it as being a godawful mess. The NTSC DVDs are horrible and probably shouldn't be used as a source.

The archive is the originally aired masters. I think they've been rescanned for this release. I tried your script with QTGMC and srestore to 25 to no avail on fixing the ghosting issues

[jagabo]

The archive is the originally aired masters. I think they've been rescanned for this release.

That doesn't mean anything. It's very common for analog PAL video to be converted to NTSC for release in NTSC countries. And vice versa. Sometimes you get NTSC->PAL->NTSC conversions because the original NTSC tapes were no longer available. There are just so many ways old shows are screwed up like this. DVDs are often made from whatever surviving tapes they can find.

There's much more wrong with your video than just a PAL->NTSC conversion. Since you haven't uploaded an m2v with a panning shot I haven't bothered to look any deeper.

[Fraugster]

The archive is the originally aired masters. I think they've been rescanned for this release.

That doesn't mean anything. It's very common for analog PAL video to be converted to NTSC for release in NTSC countries. And vice versa. Sometimes you get NTSC->PAL->NTSC conversions because the original NTSC tapes were no longer available. There are just so many ways old shows are screwed up like this. DVDs are often made from whatever surviving tapes they can find.

There's much more wrong with your video than just a PAL->NTSC conversion. Since you haven't uploaded an m2v with a panning shot I haven't bothered to look any deeper.

Here ya go! Sample.m2v

[manono]

It turns out to have been field-blended from a film source. Try this:

Yadif(Mode=1)###or your favorite bobber, such as QTGMC
Srestore(frate=23.976)

Only the worst companies pull this kind of crap. They have a good master and then proceed to ruin it.

[Fraugster]

It turns out to have been field-blended from a film source. Try this:

Yadif(Mode=1)###or your favorite bobber, such as QTGMC
Srestore(frate=23.976)

Only the worst companies pull this kind of crap. They have a good master and then proceed to ruin it.

Yadif nor QTGMC with srestore works. Still blending. Im thinking I need something that will throw away frames and not blend them. Yadif makes less blends but there still is a lot

[manono]

Works here. I used Yadif only because it's fast:

MPEG2Source("test.d2v")
Yadif(Mode=1)
Srestore(frate=23.976)

Scene change problems don't count.

[Fraugster]

Works here. I used Yadif only because it's fast:

MPEG2Source("test.d2v")
Yadif(Mode=1)
Srestore(frate=23.976)

Scene change problems don't count.

I'm using yadifmod2, is there any significant difference? What is the proper terminology for the "ghosting" of the residual frame?

Like this one:

[Attachment 49734 - Click to enlarge]

Is this as good as it can ever look with those 'residual' frames? What do you mean by 'scene change'? Like a jump cut?

[manono]

Is this as good as it can ever look with those 'residual' frames?

What are you talking about? The 'color outside the lines'? The color shifted to the left 2 pixels or so? If so, that's a different issue entirely and can be fixed, more or less.

The archive is the originally aired masters. I think they've been rescanned for this release.

If you bought these expecting some sort of a restoration, you were sadly mistaken. It looks to me they were made from an analog and probably VHS source. They are pure garbage.

[Fraugster]

Is this as good as it can ever look with those 'residual' frames?

What are you talking about? The 'color outside the lines'? The color shifted to the left 2 pixels or so? If so, that's a different issue entirely and can be fixed, more or less.

The archive is the originally aired masters. I think they've been rescanned for this release.

If you bought these expecting some sort of a restoration, you were sadly mistaken. It looks to me they were made from an analog and probably VHS source. They are pure garbage.

Look around the bird in the image I sent. See whats left? The wings spread is the frame before its pretty faint but its almost like a yellowish greenish look. I wasn't gonna 'restore' it so to say but spruce it up. Deinterlace, crop and resize, then mux the original VHS subs over it. I was hoping for better quality from QTGMC. If I reduce the SrchClipPP to 0 theres not so many blended frames.

[manono]

I don't see any residual from the previous frame. Maybe someone else will, but an unblender won't remove it. Maybe something else will. QTGMC isn't always the best bobber to use on field-blended anime. You asked about YadifMod earlier, and I use that quite a lot. Something like:

yadifmod(mode=1, edeint=nnedi3(field=-2))
Srestore(frate=23.976)

Unless that's already how you use it, you'll need Nnedi3.

[jagabo]

QTGMC isn't creating those blends. They are in your source. Run SeparateFields(), which simply separates the two fields and you'll see the same blending. And that blending is visible in the luma for at least four consecutive fields, at least eight fields in the chroma! That is why I suggested earlier that someone ran a very strong temporal noise reduction filter on the video in the past (before the DVDs were made).

This video is just terribly produced. There's film bounce at most shot changes (splices in the film caused the film to bounce in the projector as the film passed through). Often the film was still moving as the frame was digitized, leading to rolling shutter distortions. There are the field blends from a frame rate conversion and further blending from a temporal noise reducer. At shot changes the bottom half of the field blends in first, then the top half of the next field (this might be intentional).

[Fraugster]

I don't see any residual from the previous frame. Maybe someone else will, but an unblender won't remove it. Maybe something else will. QTGMC isn't always the best bobber to use on field-blended anime. You asked about YadifMod earlier, and I use that quite a lot. Something like:

yadifmod(mode=1, edeint=nnedi3(field=-2))
Srestore(frate=23.976)

Unless that's already how you use it, you'll need Nnedi3.

wow! that one fixed a lot of the most extreme blending! whats different about this and just yadifmod(mode=1)? The residual part I'm referring to is you can see the wings from the previous frame overlayed on the image I sent. Is there something I can do about the deinterlacing artifacts along edges?

EDIT: Here's another example I found

[Attachment 49736 - Click to enlarge]

[Fraugster]

QTGMC isn't creating those blends. They are in your source. Run SeparateFields(), which simply separates the two fields and you'll see the same blending. And that blending is visible in the luma for at least four consecutive fields, at least eight fields in the chroma! That is why I suggested earlier that someone ran a very strong temporal noise reduction filter on the video in the past (before the DVDs were made).

This video is just terribly produced. There's film bounce at most shot changes (splices in the film caused the film to bounce in the projector as the film passed through). Often the film was still moving as the frame was digitized, leading to rolling shutter distortions. There are the field blends from a frame rate conversion and further blending from a temporal noise reducer. At shot changes the bottom half of the field blends in first, then the top half of the field.

Yea this show when first airing had horrible budget restrictions. The Platinum versions which were made in 2003 went back to the drawing board and remastered everything. Thats the version most people have seen. But the subs on the original version are great and EVAgeek user made digital versions of the VHS subs that I wanna sync to this version because they're made for the Perfect Collection which is also the originally aired version. The perfect collection I could not find and this version is technically the newest version available. The video is destroyed to all hell but I can fix it up to at least be presentable and hostable on my Plex.

[SaurusX]

Japanese telecine jobs from the 90's and previous are notoriously blendy. Animation and live action. This is one area that they were definitely not cutting edge.

[Fraugster]

Japanese telecine jobs from the 90's and previous are notoriously blendy. Animation and live action. This is one area that they were definitely not cutting edge.

Would I have any luck with AnimeIVTC? I havent been able to get it working so I've been avoiding it.

[jagabo]

Would I have any luck with AnimeIVTC?

No, your source is too messed up. You will never remove all the blending.

[Fraugster]

Would I have any luck with AnimeIVTC?

No, your source is too messed up. You will never remove all the blending.

I figured. Well that wraps it up then. Thank you all for your help I sincerely really really really appreciate it. To have people as motivated by video as I am is a great feeling and I can't express enough how thankful I am for all your help!

[jagabo]

I did work out a way to remove a little of the light residual temporal blending. By subtracting a little bit of the previous frame you can reduce the blending.

Mpeg2Source("Sample.d2v", CPU2="ooooxx", Info=3) 
yadifmod2(mode=1, edeint=nnedi3(field=-2))
Overlay(last, Loop(2,0,0).ColorYUV(gain_y=-220, cont_u=-220, cont_v=-220), mode="subtract").ColorYUV(gain_y=40, cont_u=40, cont_v=40)
SRestore(23.976)

Because of the severe blending in the original video QTGMC ends up making it worse (as you noticed before). Though Yadif is generally a poorer deinterlacer in this particular case it avoids exacerbating the blending.

[Fraugster]

I did work out a way to remove a little of the light residual temporal blending. By subtracting a little bit of the previous frame you can reduce the blending.

Mpeg2Source("Sample.d2v", CPU2="ooooxx", Info=3) 
yadifmod2(mode=1, edeint=nnedi3(field=-2))
Overlay(last, Loop(2,0,0).ColorYUV(gain_y=-220, cont_u=-220, cont_v=-220), mode="subtract").ColorYUV(gain_y=40, cont_u=40, cont_v=40)
SRestore(23.976)

Because of the severe blending in the original video QTGMC ends up making it worse (as you noticed before). Though Yadif is generally a poorer deinterlacer in this particular case it avoids exacerbating the blending.

Ooh okay. I've been having success with QTMGC in the very fast setting as it uses yadif with the benefits of QTGMC. I'll try the overlay and see how that improves the image.

[jagabo]

I did work out a way to remove a little of the light residual temporal blending. By subtracting a little bit of the previous frame you can reduce the blending.

Mpeg2Source("Sample.d2v", CPU2="ooooxx", Info=3) 
yadifmod2(mode=1, edeint=nnedi3(field=-2))
Overlay(last, Loop(2,0,0).ColorYUV(gain_y=-220, cont_u=-220, cont_v=-220), mode="subtract").ColorYUV(gain_y=40, cont_u=40, cont_v=40)
SRestore(23.976)

Because of the severe blending in the original video QTGMC ends up making it worse (as you noticed before). Though Yadif is generally a poorer deinterlacer in this particular case it avoids exacerbating the blending.

Ooh okay. I've been having success with QTMGC in the very fast setting as it uses yadif with the benefits of QTGMC. I'll try the overlay and see how that improves the image.

You may have to adjust the ColorYUV settings for the blend removal if you change the prior filters.