Trying to RIP Anime DVDs to Episodes. Problems with Avisynth Filters.

First of all good morning/evening to all and I hope you are having a good day.

I occasionally have looked for help in VideoHelp, but this is the first time I am actually actively asking for help, so I am really sorry in case I forgot to attach some needed information. Also I am sorry if I am actually giving too much information, but I really do not know what to include. This is my first time trying to more manually encode, rather than go for something like Vafe, that has premade filters.

I have installed the latest KLite Mega Codec pack, Kawaii Codec Pack (I use the Media player from klite for general videos and the one from Kawaii when just watching shows in fullscreen, for it's optimized exclusive mode and madVR), Avisynth 2.6 with many scripts and plugins, which at least the ones I use when I test, I try to make sure that are the latest.


The DVD are NTSC with a variable bitrate @ 30FPS and I ripped it with MakeMKV. These are two parts cut from the MKV file I got from it, and I cut them off using Avidemux; Sample 1 and Sample 2, where Part one is a 2 minute part with bright scenes and the part 2 is a 1 minute part with dark scenes.

From there I used the latest HandBrakeCLI to make it from variable to constant, since keeping it variable gave me a lot of headaches when I was trying toe detelecine it with avisynth. Afar from that, I am cutting Opening/Endings off, and making it constant 30FPS, actually helped me with cutting them with high precision.

This is the commands I run HandBrakeCLI with:

Code:

---

"C:\Encoding\Ordered Chapter Creator\HandBrakeCLI_64bit.exe" -i "C:\Encoding\Ordered Chapter Creator\title00.mkv" -o "C:\Encoding\Ordered Chapter Creator\1lossless.mkv" -m -E copy --audio-fallback ac3 -B 0 -s 1 --subtitle-default -e x264 --x264-tune animation -r 29.97 --cfr -q 8 -x ref=1:weightp=1:subq=2:rc-lookahead=10:trellis=0:8x8dct=0:keyint=2:min-keyint=1 --strict-anamorphic --crop 2:0:4:4 --audio 1 --subtitle 1 --markers="C:\Encoding\Ordered Chapter Creator\chapters_file.csv"

---

Here are the Sample 1 and Sample 2 after using the code above.

I have to admit that the red part of it, I do not know what it does, but rather found it premade, although the whole script I found premade and I adjusted it to my needs. The audio matches the source, the FPS forced to be constant and full, quality higher than the source and I am cropping out black and bad pixels on the edges of the original. Actually just before asking on here I added "--detelecine --deinterlace --decomb" and it really worked well on both 29.75 and 23.976 FPS when I tried them, with only the lines becoming a bit bad, with the 23.976 FPS one being a bit worse.

But to not confuse, the video I am working with Avisynth, came from the original source + the code above with handbrake. I did not use the detelecine/deinterlace/decomb functions of it, because I thought if the video is as close to the original, it would be better for avisynth to work with it.


After trial and error, I just can't seem to get the right way to interlace the video that I got from Handbrake. The main reason I want handbrake, is because it can convert the video in such a form, that cutting it on it's chapters and removing the opening and ending is easy. Also cleaning some parts that seem to have extra frames from other parts.

If you click on the images below, it will open an imgur page with a couple of them for reference, since imgur has the best UI imho.

My best take with TFM and TDecitame, that gave me the best result from different tries. No matter what, there will be always a part that some interlacing seems to exist.

Code:

---

ffvideosource("C:\Encoding\Tests\test1.mkv")
AddBorders(0, 0, 0, 2)
deint = TDeint(mode=2, mtnmode=3, blim=100)
TFM(cthresh=4, slow=2, clip2=deint)
savedHints = last
MergeHints(hintClip=savedHints)
TDecimate(mode=0)
Crop(0, 0, -0, -2)
Spline36Resize(850, 478)

---

https://forum.videohelp.com/images/imgfiles/iMIY0iSm.png
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

With AnimeIVCT mode 1 it actually gives a lot of ghost coloring. Sorry if this is not the right term. As in a color ghost of a previous or next frame/pose, appears for a split second over the next. Also, a lot of times, lines in motion get a lot of jittery, but it feels that happens from sharpening, although I do not know much about why it would happen. Also due to killcomb, which was the best way I found to kill the extra interlacing that was left like with the TFM-TDecimate combo, makes the video slightly more blurry. Although I am guessing I should worry for that later.

Code:

---

ffvideosource("C:\Encoding\Tests\test1.mkv")
AddBorders(0, 0, 0, 2)
AnimeIVTC(mode=1, aa=4, precision=3, dark=1, killcomb=3)
Crop(0, 0, 0, -2, align=False)
Spline36Resize(850, 478)

---

https://forum.videohelp.com/images/imgfiles/RmqNdZ7m.png
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

With AnimeIVCT mode 4, that keeps the FPS as it is, the motion jittering of lines is gone, but the video looks more blurry than mode one and since killcomb=3 does not work, the need of an extra IVCT filter, so the random interlaced frames wont appear, so I used ReMatch. Now though except the random color ghosts the other two have, some of them seem to be interlaced.

Code:

---

ffvideosource("C:\Encoding\Tests\test1.mkv")
AddBorders(0, 0, 0, 2)
ReMatch()
AnimeIVTC(mode=4, omode=2, aa=4, precision=3, dark=1)
Crop(0, 0, 0, -2, align=False)
Spline36Resize(850, 478)

---

https://forum.videohelp.com/images/imgfiles/5blcUdgm.png
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Now like I mentioned. Using detelecine/deinterlace/decomb with HandBrakeCLI actually seems to prove useful, but I did not do it, because I thought it would be better to leave the source as intact as possible. I just am not sure since forcing it to 23.976 FPS made the video at times slow down at random scenes and on both, the lines seem to be blocky and not smooth, like when avisynth is actually doing the job. In this link I have the same image compared between the ways I used IVCT.

https://forum.videohelp.com/images/imgfiles/pDuRSwem.png

I guess it would be better if I somehow could convert the original source through meGui, but that line that I said I do not know what it does,

Code:

---

-x ref=1:weightp=1:subq=2:rc-lookahead=10:trellis=0:8x8dct=0:keyint=2:min-keyint=1

---

is something I could try to adjust on meGui and maybe convert it to raw through there, applying the correct filters? I guess going from having everything premade and jsut choosing options, to trying to go more manual, was not something that could happen easy.

I am sorry if the info were not condensed and if I confused you, but honestly I am a bit lost myself. Thanks for any help in advance and I hope you all will have an awesome week.

I just looked quickly at the first sample. I think your problems in avisynth might be from using ffvideosource. It's less reliable for mpeg2 sources, especially in MKV. It IVTC's cleanly with DGIndex (extract to elementary stream with mkvextract, then use DGIndex) . For example, I don't see the palm issues as in your screenshot, even with pp=0 (deinterlacing off when combing detected)

Those encoding settings in red are lower compression (faster to encode, lower quality), but with a keyint of 2 (max GOP size), means IPIPIP , basically very low compression, higher filesizes required for a certain level of quality. I guess that's what you're going for with crf 8 anyways

1. DON'T make MKVs before working on it (and especially if providing samples).
2. Use DGindex on the VOBs.
3. It IVTC's perfectly using a standard TFM/TDecimate (only looked at the first sample).

If this is all about cutting it the way you want, then DVDDecrypter can decrypt by chapters so you can leave out the opening/ending from each episode. Or use DVD Shrink on the DVD decrypted to the hard drive to get the parts you want.

And if using AviSynth don't use Handbrake, and vice-versa. I couldn't make any sense out of any of that part.

Thank you for the fast replies. I will try to not test in both ffvideosource and DirectShowSource when it comes to avsPmod from today XD. I never knew about DGindex, so I am going to try it now and do it straight from the Vobs as suggested. Although MakeMKV does not convert the source as I know. It just copies the VOBs in a large MKV. EItherway, I am going to try now straight from the DVD.

Those settings I found while I was looking for a way to make the video have more I-Frames, so I could cut it with more precision on every chapter. This is the reason I use Handbrake, to make the original source have more I-frames, making sure the chapters will not have parts of another chapter, because there were not I-frames for MKVMerge to cut on.

I did not use DVDDecrypter, but I used DVDFab Decrypter to do that. All the episodes would come cut in weird ways, having in the beginning or the end, extra or missing seconds. The only reason I ended up using handbrake to add more I-Frames, is because I found that on a site that was suggesting it as a solution. This is really my first encode from the source, so I have been trying different things before I ended up coming here for help.

As for DVD Shrink, sadly when I tried it, it crashed, but I am guessing the DVDFab Decrypter does the same job, no? If anything I could use MakeMKV to extract the raw and then use mkvmerge with -split and the chapters I want. I still do not know if MakeMKV just extracts, but the speed it works is literally, like it copies the file from my DVDs to the HDD.

DVDDecrypter, not DVDFab. Set it for IFO Mode (not the default File Mode) and select the chapters you want.

Quote:

---

I ended up using handbrake to add more I-Frames, is because I found that on a site that was suggesting it as a solution.

---

It's not a solution and wastes time and degrades the video quality with a full reencoode. AviSynth's Trim command can cut where ever you like, though, as can DGIndex when you make the initial D2V Project File.

Quote:

---

Originally Posted by weltall

Thank you for the fast replies. I will try to not test in both ffvideosource and DirectShowSource when it comes to avsPmod from today XD. I never knew about DGindex, so I am going to try it now and do it straight from the Vobs as suggested. Although MakeMKV does not convert the source as I know. It just copies the VOBs in a large MKV. EItherway, I am going to try now straight from the DVD.

---

If you've already ripped the DVDs as MKVs, you can open the MKVs with TSMuxer and remux them as TS files. DGIndex will be able to open and index TS files, and MeGUI will index TS files with DGIndex. It's better to rip the DVD "correctly" in the first place though. MakeMKV should be able to rip DVD titles as individual MKVs. I'm not sure if it can rip by chapter though. Or preferably use DVDDecrypter or DVDShrink.
There's a "force film" option in MeGUI's settings for DGIndex that forces the source to be decoded as 23.976fps if most of it is deemed to be "film". It's set to 95% by default, but I disable the "force film" option completely myself.

Quote:

---

The DVD are NTSC with a variable bitrate @ 30FPS and I ripped it with MakeMKV. These are two parts cut from the MKV file I got from it, and I cut them off using Avidemux; Sample 1 and Sample 2, where Part one is a 2 minute part with bright scenes and the part 2 is a 1 minute part with dark scenes.
From there I used the latest HandBrakeCLI to make it from variable to constant, since keeping it variable gave me a lot of headaches when I was trying toe detelecine it with avisynth. Afar from that, I am cutting Opening/Endings off, and making it constant 30FPS, actually helped me with cutting them with high precision.

---

It might be that you're just mixing up the terminology, but variable bitrate and variable frame rate are two completely different things.

Quote:

---

Originally Posted by weltall

Those settings I found while I was looking for a way to make the video have more I-Frames, so I could cut it with more precision on every chapter. This is the reason I use Handbrake, to make the original source have more I-frames, making sure the chapters will not have parts of another chapter, because there were not I-frames for MKVMerge to cut on.

---

While it's better to rip the DVD according to how you want to encode it with DVDDecrypter or DVDShrink (ie by chapter or title etc) if it's not ripped that way you can use the AVS Cutter under MeGUI's tools menu to specify start and end frames for encoding. You can add multiple start and end points to a script.
Create a script with MeGUI as you normally would (including any filtering and IVTC etc), and if you want the source divided into sections when you encode it, make an appropriate number of copies of the script. Open each copy of the script script with the AVS Cutter and specify the start and end frames. There's a video preview to make it easy. The AVS Cutter will also save a cuts file for loading into the audio section so the audio can be re-encoded to match, or there's an Audio Cutter under the Tools menu for cutting audio without re-encoding.
Doing it that way you don't have to split the source video into sections for encoding, so there's no I-frame issues. You can use the AVS Cutter to specify any end start and end frames you like.

Thank you so much manono. Honestly I thought DVDFab was just a newer version of DVD Decrypter. I never imagined DVD decrypter can do such a great job.

DGIndex seems to be reencoding the file too though, but I am guessing it is of higher quality? If anything I tried DGIndex and got the file directly from the source, but I am quite confused with the resulted file. I tried it on the default Skal Algorithm and tried it on IEEE-1180 Reference one too. I also tried it on forced film too, just in case, even if the result was video. They seem to have become 25 FPS and the player says the duration changed to 1 hour and 22 minutes, even though Media Info says it is still 1 hour and 33 minutes. For some reason AvsPMod shows 1 hour and 46 minutes. I am guessing the wrong times are because the format is not fully supported by media player and AvsPMod? Also weirdly, both the player and AvsPMod show it is heavily telecined. Actually the weird part is that both show a different image. (I am not talking about the aspect ratio difference.)

This is with MPC
https://forum.videohelp.com/images/imgfiles/HLHcoLpm.png

And this one from AvsPMod
https://forum.videohelp.com/images/imgfiles/y4nYwdbm.png

@hello_hello
Honestly, this will be the 3rd time I will be starting over and twice now I have reached the point of splitting the episodes and getting them ready, only to realize that something went wrong. I guess I am learning the hard way. The sure is I definitely want to do it right, since I love the anime and I want to be as OCD as possible, and starting over is the only option.

I might try to make DVDShrink work again. I am sure I am doing something wrong. Also I am definitely confusing the two terms. I did mean FPS XD Sorry for that. As for what I want to do, it is a whole mess for sure. I first want the episodes as whole episodes, so I can attach to them .ass subtitles that I have, being better than the vob ones they come with, which are per episode. After that I want to split them apart to pieces, so I can remove the Openings/Endings. I have think of some painful ways to do so, but if anything I will do it if nothign else can be done.

I actually have a little video showing step by step how to split episodes with AVS Cutter. Looks a bit confusing, but I am sure if I do it once, it will work. So I am guessing I need to accept it, start over, mess up and ask for more help XD

I really appreciate all the help and the time and effort with the detailed answers you are all giving me :) I guess I need to start with DVD Decrypter :)

Quote:

---

Originally Posted by weltall

DGIndex seems to be reencoding the file too though...

---

DGindex is incapable of reencoding anything. Its main function is to demux the audio and to make a D2V project file for use in an AviSynth script with the DGDecode.dll and MPEG2Source. It's all explained thoroughly in the docs included with the DGMPGDec package.

You can choose to demux the video as well (Save Project and Demux Video) but it's an unreencoded M2V straight from the DVD VOBs. Ordinarily you wouldn't need the demuxed M2V as the D2V is enough to do the frameserving into the encoder of your choice.

Quote:

---

I tried it on the default Skal Algorithm...

---

The default one is fine.

Quote:

---

I also tried it on forced film too, just in case, even if the result was video.

---

You can't use Forced Film with your source. You have to use 'Honor Pulldown Flags'. Forced Film is for soft telecine and yours is hard telecine. Again, read the docs. If you were to run the Preview (File->Preview) you'll see your source is Interlaced Video (hard telecine, in this case).

Quote:

---

They seem to have become 25 FPS

---

If you use Forced Film it'll become 23.976fps and full of interlacing. Otherwise, the default Honor Pulldown Flags keeps it at 29.97fps ready to be IVTC'd in your script. At no point will it become 25fps unless you make it so using a command in the script such as AssumeFPS(25). At 23.976fps or 29.97fps the length remains the same unless you change it with another line in your script. You can open your script in VDub and go File->File Information to find out such things as the framerate and the video length.

To be honest I'm not 100% clear as to what your process is.
I referred to MeGUI in my previous post because you mentioned it in your opening post so I assumed you were familiar with it.

DGIndex doesn't re-encode. Just like ffms2 it just decodes the source video. It only supports MPEG-1 or MPEG-2 in certain types of formats or containers such as VOB files, MPEG, TS or M2TS. It doesn't support MKV or MP4. It can apply some processing such as de-blocking but it's basically a decoder. The algorithm used doesn't really matter. I think that's more CPU-speed related.

Quote:

---

I also tried it on forced film too, just in case, even if the result was video.

---

You shouldn't apply force film to video or you'll make a mess. If it's detected as film by DGIndex or force film is applied because it's film it'll be decoded as 23.976fps progressive and no IVTC is required. If DGIndex decides it's video it'll be decoded as 29.970fps, and if that's the case you need to apply IVTC to reverse the pulldown process and restore the original 23.976fps progressive film. You can't do both.... apply IVTC in a script if DGIndex decided it's film or if force film was applied.

Part of your earlier problem appears to be you were encoding a source with 3:2 pulldown as progressive video using Handbrake, then trying to remove the pulldown from the Handbrake version where the interlacing artifacts had been encoded.

Quote:

---

They seem to have become 25 FPS and the player says the duration changed to 1 hour and 22 minutes, even though Media Info says it is still 1 hour and 33 minutes. For some reason AvsPMod shows 1 hour and 46 minutes. I am guessing the wrong times are because the format is not fully supported by media player and AvsPMod? Also weirdly, both the player and AvsPMod show it is heavily telecined. Actually the weird part is that both show a different image. (I am not talking about the aspect ratio difference.)

---

I don't really understand that, although if you're opening vob files and checking the duration using a media player it's safer to assume it'll display the wrong duration than to assume it's correct..... for vob files. If you open the same script in MPC-HC or AvsPmod they should agree on duration though.

Start off nice and simple. Remux your ripped MKV as a TS file with TSMuxer and index the TS file with DGIndex, or index the first vob file in a set with DGIndex if you've re-ripped the DVD that way. Don't use force film.

DGDecode_mpeg2source("D:\Sample2.d2v")
tfm(order=1).tdecimate(mode=1)
Spline36Resize(854,480)

That should be all you need for a nice progressive 23.976fps output if your second sample is anything to go by. If it looks okay you could try replacing tfm and tdecimate with AnimeIVTC(mode=1) if it also includes other filtering for improving animation, but I've never used AnimeIVTC so I don't know what it does exactly. A quick look seems to indicate mode=1 does standard 3:2 pulldown removal (IVTC).

What's the logic behind AddBorders(0,0,0,2) and Crop(0,0,0,-2) in your original scripts? I'm not saying it's bad, just that I don't understand.

Edit: I see manono beat me to much of it while I was writing the post. Oh well...... :)

Ah my mistake. Since I did not know DGIndex, I thought I would save the project and demux the video. I will have to read the instructions of the DGIndex. I did though peak on both of them, but I did not read too carefully to be honest. But my mistake was definitely that I thought remuxing it was part of it. It was why I got confused a bit when hello_hello mentioned the meGui will index the files with DGIndex's help.

I only use the windows MP for batch file commands, to play stealthily sounds. Afar from that I have not used it for many years, since MPC is lighter and better. But both MPC and AvsPMod did not agree on the time and both were wrong, but again, I was doing it wrong using the demuxed file from DGIndex, since I should have been using it in a different way ^^;; do forgive my incompetence.

I think it finally dawned on me that DGIndex is only for ... indexing.. (uh I feel stupid) because of your example with AvsPMod hello_hello XD. Ok, I will be using the d2v file. Once more, sorry guys for thinking I should demux with DGIndex.

I think it is time to start again with tests and filters and the addborders+crop is because like I said the video I was working with, was the precropped video I had from handbrake. I had it crop 2 pixels from the height and some of the other denoise, sharper and dehalo filters I have been using (including AnimeIVCT) wanted the height to be multiple of 4. So I just added the extra 2 just so I wont get that problem.

Thank you all for the help and both you manono and hello_hello, for the detailed replies. I am glad I asked in here. Now back to ripping and splitting them until the time that I will have to encode them and finally this time I will rip them the right way! :D

PS: It does not matter if he posted before you hello_hello, I still appreciate the effort you put. If anything, reading some things twice is definitely better!