Suggestions For Optimizing SD Animation Encoding

[LouieChuckyMerry]

Delete the red bits after indexing the .m2v files with DGIndex:

DGDecode_mpeg2source("YourSourceFile'sPath.d2v", cpu=4, info=3)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-64bit[Portable]\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)

[drojman]

So what I ended up doing for the minute is COMPLETELY wiping everything, AviSynth and MEGUI as I realised that I had created a whole mess - apparently MEGUI's portable version of AviSynth and the standalone can cause some sort of weird interplay - it gets entered in the registry and as such can try and cross polinate plugins with each other. I'm sure someone more familiar with it can figure it out but I figured it would be better to start anew.

Plus I've decided to re-do Futurama as the old versions I have for some reason were 710x480, not sure why!

With all that said (and again, thank you so much for all your help) if I am to use your version with a completely clean slate, what's going to be the best way? I realise...I've never actually done more than one episode at a time with AviSynth since I spent so long testing the same episode over and over. Do you have to duplicate the script but name each different source file? As you can tell, I have no real clue what I'm doing.

[LouieChuckyMerry]

In what form are your source files?

[drojman]

All I have now is DVDs processed through MakeMKV to MKV files. Otherwise they are untouched.

Both NTSC dvds too.

[LouieChuckyMerry]

OK. Make a new folder in a place convenient for you and call it something like "EncodingTemp", whatever you want. Then place each .mkv episode in its own thoughtfully named subfolder, like "S1.E1", "S2.E2", etc., then batch demux each .mkv's video file with MKVcleaver. Now you'll have .m2v files. Get that going and I'll upload my stuff then PM you. Ahhh, these are indispensible:

AVSMeter (forum thread)

Avisynth Info Tool (forum thread)

Universal Avisynth Installer (forum thread)

Maybe try to wrap your head around some of this before you begin encoding .

[LouieChuckyMerry]

I sent you a PM.

[drojman]

Thanks again, I at least was able to use Powershell to shift all the episodes into an individual folder so that didn't take as long as I thought.

On the move with MKVcleaver now!

All done, that was actually quite painless!

[LouieChuckyMerry]

Thanks again, I at least was able to use Powershell to shift all the episodes into an individual folder so that didn't take as long as I thought.

On the move with MKVcleaver now!

All done, that was actually quite painless!

_____________

Did you mean each episode is in its own folder? That's almost mandatory to avoid chaos.

[drojman]

Yes, all in their own folder quite nicely. Powershell is excellent for that sort of thing thankfully.

Hope the hangover is getting better

[LouieChuckyMerry]

It is, thanks . Couldn't resist, ha ha. Did you download my MeGUI and plugins? And use Groucho2004's awesome tools to sort out your AviSynth installation (my MeGUI doesn't use the included AviSynth+)?

[drojman]

I've downloaded everything I think I need now, watching the Futurama rips back, the combing on them is so bad

[LouieChuckyMerry]

I've the NTSC and PAL DVD's of the classic first four seasons and both sets are horrible quality. After LOTS of help the final script on the Futurama thread produces results as good as The Simpsons script. Good luck with your projects, I'll answer any questions I can.

[drojman]

How did your MEGUI interplay with the seperate AviSynth? That's where my errors creep in. It seems like it gets confused between the two and tried to run a script that doesn't exist in install, despite which setting in MEGUI I use in regards to an external or internal use of AviSynth.

And a more general question, I understand the idea behind cropping the 16 pixels, in reality without the crop and allowing the picture to remain as 720x480 what real word difference would there be, if any?

[LouieChuckyMerry]

Honestly, I can't ever remember any issues with the AviSynth included with MeGUI because when I began using it that feature didn't exist so I had to have AviSynth installed. My knowledge is quite limited with regards to actually making AviSynth functional on a computer; the AviSynth thread at doom9.org is a good place for help. Also, those tools by Groucho2004 should be some help.

The reason each side is cropped is because they're not clean, but you can judge for yourself. And, you could always crop the sides but resize to 720x480 anyway; I cropped the 8 pixels each side then upscaled to 960x720 and it's not in the least noticeable.

[drojman]

Ok...let me see if I have this somewhat correct at least in terms of the process itself:

Seperated .m2v files in their own folder which will individually be process by MEGUI, and treated as their own thing.

Once the processing is done and the files are cleaned up, do I demux all audio from the other non - edited MKVs and then combine them?

[LouieChuckyMerry]

My work flow for TV series is to have a main folder with subfolders for each season that themselves have subfolders for each episode. In each episode subfolder I have the subtitles with the name of the episode, like "S1E1-SimpsonsRoastingOnAnOpenFire.srt" and the source file "S1.E1-OriginalVideo,OriginalAudio(s).mkv". I then batch demux the video and audio(s) files with MKVcleaver and rename the audio file(s) logically. Then, for .m2v video (DVD's) I batch index with MeGUI and the default DGIndex, batch encode the episodes after running as many tests as necessary to determine a reasonable CRF, then, using MKVToolNix I'll remux the encoded video with the original audio(s), using the subtitle with the proper episode name as the first file loaded into MKVToolNix (but not ticked) so that the resulting .mkv file is also then named properly (I'll upload you my portable MKVToolNix with useful settings). Just a moment...

[drojman]

Would this apply if I have already renamed the files correctly with Filebot?

So what I did was:

1. Organise each MKV into is own folder, while also being correctly named.

2. Demuxed it so I have the audio (english and commentary), subtitles and video file. They are labelled (for example) Track1_The Series has Landed)

What I planned to do is once the videos are processed is re-feed them back through Filebot, I think from my experience enough of the name is intact that Filebot would be able to recognise and rename them.

[LouieChuckyMerry]

I've never used Filebot, but if it works for you that's great, and the rest seems like a good plan .

[drojman]

Great, I'm hoping to do it that way. Thank you for the files though as I will keep them handy for anything Filebot won't recognise which is definitely possible.

Now on to the actual hard bit and figuring out how to apply MeGUI/Avisynth to all the M2V files at once. What I've done for the minute is make a copy of all files so I can at least test without ruining anything

[LouieChuckyMerry]

I always have two separate sets of all my source files because I'm an Eye-Die-Ott . I use Everything to search the main folder, then type ".m2v" so that only the video files are shown, then drag & drop them to MeGUI one by one and batch index them. Then I again use Everything to search the main folder, typing ".avs", so I can then drag & drop them one by one and batch encode them. I use Everything for, er, almost everything when dealing with TV series encoding.

[drojman]

I have made a clone of the Futurama files each step of the way. I think I might have about 200GB used just for these first 4 DVDs!

That Everything tool is incredibly useful and just what I need, thank you!

I know have AviSynth setup and verified using the tools you sent me...I think. I copied the plugins folder to the AviSynth version I installed too, which I hope will work/was the right thing to do.

In terms of the scripting now, I see there's a few included, should I be using them/any of the others you've used. I've sort of decided to go with Futurama first as I'm more familiar with it, so I figured I'd take a bit at that first.

I'm really sorry to be a pain, and again I realise I'm asking a lot. Any chance you could show me your work flow for actually processing the files in MEGUI? I think I can see what I'm doing, but I'd be lying if I said I was sure.

[LouieChuckyMerry]

I have made a clone of the Futurama files each step of the way. I think I might have about 200GB used just for these first 4 DVDs!

For the first four seasons I have 140GB x 2 and the DVD's for extra backup, ha ha.

That Everything tool is incredibly useful and just what I need, thank you!

I find it indispensable.

I know have AviSynth setup and verified using the tools you sent me...I think. I copied the plugins folder to the AviSynth version I installed too, which I hope will work/was the right thing to do.

You can use the AVS Info Tool to check things out.

In terms of the scripting now, I see there's a few included, should I be using them/any of the others you've used. I've sort of decided to go with Futurama first as I'm more familiar with it, so I figured I'd take a bit at that first.

The profile you want is "Tools/AVS Script Creator/AviSynth profile/zFuturama[Upscale][NTSC]". Go there and select it then close the small window; this makes it so that profile will be the default choice when indexing. Before you dive in, take a bit of time with just a couple episodes to get the hang of things. Trust me on that; I rerererere-encoded both The Simpsons and Futurama because I was too excited. Literally.

I'm really sorry to be a pain, and again I realise I'm asking a lot. Any chance you could show me your work flow for actually processing the files in MEGUI? I think I can see what I'm doing, but I'd be lying if I said I was sure.

Please explain? No worries, I simply am confused. Back in about thirty minutes...

[drojman]

I used that to check and it warns me that I have a duplicate but reading the Doom9 forum that's just a warning and acceptable.

Currently making a backup now of everything that I've split so I can test it, will probaby split down into a smaller group again for further testing to see exactly what's what.

Sorry I'm not really being super clear, I'm scanning through the docs now. Basically I think the only thing I have left to do is basically figure out how to apply AviSynths script to a big batch of files and process them one by one.

By the way, with your processed files do you overwrite your input files or create new? I'm thinking for later when remuxing together it's going to get complicated ensuring I combine the correct rencoded file and not the original ;

Out of interest MEGUI seems to offer a Demuxing option, to split audio, am I being stupid in seeing this though?

[LouieChuckyMerry]

I used that to check and it warns me that I have a duplicate but reading the Doom9 forum that's just a warning and acceptable.

You could run a simple test by indexing a single .m2v file with the Futurama profile, fixing the two lines in the resulting .avs file mentioned earlier, then see if the script loads OK or crashes MeGUI.

Currently making a backup now of everything that I've split so I can test it, will probaby split down into a smaller group again for further testing to see exactly what's what.

Wha?

Sorry I'm not really being super clear, I'm scanning through the docs now. Basically I think the only thing I have left to do is basically figure out how to apply AviSynths script to a big batch of files and process them one by one.

Once we've determined whether or not your AviSynth-MeGUI is up and running, we can deal with this.

By the way, with your processed files do you overwrite your input files or create new? I'm thinking for later when remuxing together it's going to get complicated ensuring I combine the correct rencoded file and not the original ;

I don't quite understand the bit in italics. To avoid confusion, I have my sources on my laptop's HDD storage drive, MeGUI on a RAMDisc (not necessary; I just thought "Why not?" at the time I set things up), and have the encoded video (MeGUI output) created in a "ReEncTemp" folder on my laptops SSD C:\ drive. Seems to me having the sources and the encodes on different drives is a good thing so one disc isn't doing everything (but perhaps I'm being silly).

[drojman]

OK sorry, I think I've had a few lightbulb moments! Well not lightbulb, someone in the distance maybe lit a candle or something.

I couldn't figure out the interplay between the indexing and script creation, but I now do. It's index first, then edit the AVS file. I had the crash you were experiencing so I still need to go and fix that (daunting!) but that's something that is certainly doable. I'm sorry if I don't use the correct terminology along the way, but I do think I better understand the tool now. I'm going to have a go and re-encoding one now and will post back results, and now I finally understand why I have to split the MKVs too!

Ignore my ramblings about storage. I was/am trying to figure out the best way to make it easier for my brain to manage. Basically I was musing how best to have MEGUI output a video file in a way that's going to make it easy to remux with my split audio files. If that makes any sense.

Edit: Even after editing out the offending comments in my .avs file MEGUI crashes when I load the script...

[LouieChuckyMerry]

I couldn't figure out the interplay between the indexing and script creation, but I now do. It's index first, then edit the AVS file. I had the crash you were experiencing so I still need to go and fix that (daunting!) but that's something that is certainly doable. I'm sorry if I don't use the correct terminology along the way, but I do think I better understand the tool now. I'm going to have a go and re-encoding one now and will post back results, and now I finally understand why I have to split the MKVs too!

To save time, if MeGUI doesn't crash when you load a script than it'll run fine; also, as you'll note at the bottom of all my profile scripts is:

# # Trim()
# SelectRangeEvery(1000,66)

Any line in an .avs script with a hash (or two or ten or...) in front of it won't be read; BUT, if you remove the hash(es) it will. The "Trim()" function, at its simplest, allows for sections of frames (digital video is like digital audio, just a series of samples; for digital video there are "GOP"'s, or Groups Of Pictures, literally a series of still images); AviSynth is a frame server, it feeds these pictures (frames) one at a time to, er, technical term unknown, but I reckon you get the idea. So, with "Trim()" you can test a short section of the source easily; for example, "Trim(1111,2222)" would only encode frames 1111 through 2222. Great for checking things out. "SelectRange()" literally selects a range of frames to process; my "SelectRange(66,1000)" selects 66 frames every 1000 frames, or 6.6%. This is what I use to check the quality-bitrate-whatever of a script so I don't have to run an entire encode to view the results; ie, I'll look at a two-hour movie source, determine how strong I want to set the denoising (SMDegrain) then run a SelectRange(66,1000) at a well-considered CRF so I can poke through the whole film--66 frames every 1000 to the end of the movie--and then check the visual quality and bitrate then make any necessary adjustments to the script-CRF setting before encoding the entire film. On my rig a full two-hour long movie would take ~18 hours to encode but a "SelectRange(66,1000)" would only take about an hour +/-. I hope that wasn't too confusing.

Ignore my ramblings about storage. I was/am trying to figure out the best way to make it easier for my brain to manage. Basically I was musing how best to have MEGUI output a video file in a way that's going to make it easy to remux with my split audio files. If that makes any sense.

As above, I think keeping the sources separate from the outputs is mandatory for chaos control (if not hard drive abuse). When muxing your encoded video with the original audio I find it easiest to simply have the source folder open, open MKVToolNix, drag the subtitle to MKVToolNix to establish the output file name (but it won't be part of the .mkv file because MKVToolNix is set to have the first file drag-dropped be the default output file's name but not tick {include in the output} any subtitle files, then add the audio(s) from the already-open source folder, then right-click in the "File name" box (upper-left), then select the video file from the dialogue box (adjust Window's dialogue box such that your output folder is visible in the dialogue box to save navigation time; I'll shoot an image).

[drojman]

So I think I've hit a snag with my AviSynth+ install - whenever I try and index the files using that profile, and then hit save, MEGUI crashes. I switch to the 32bit version of AviSynth and the indexing completes perfectly, but when I run the script to actually process the video, I get an error because some of the plugins I have are 64 bit. Time to try and fix that.

I don't supposed you know which version you AviSynth you're using?

Sorry just saw that you replied, will read...

[LouieChuckyMerry]

So I think I've hit a snag with my AviSynth+ install - whenever I try and index the files using that profile, and then hit save, MEGUI crashes. I switch to the 32bit version of AviSynth and the indexing completes perfectly, but when I run the script to actually process the video, I get an error because some of the plugins I have are 64 bit. Time to try and fix that.

Unfortunately it'll do that until the two lines are fixed, which is far from functional when you have to restart MeGUI for every frackin' episode (I did seek help; no answers yet). I'm using AviSynth+ 3.7.2 (r3661, 3.7, x86_64). Did you fix the offensive two lines in the script? Ahhh, you could index all the .m2v files with your 32bit version (not "+", I think that's why the two lines cause trouble; hey!, does the 64bit verion of AviSynth work; that would make it so you could index and encode without switching versions), batch fix the two lines with Notepad++ (or whatever you use in place of Notepad), then run the encodes with AviSynth+. Not ideal but better...

[drojman]

Unfortunately I think this is a seperate issue, MEGUI isn't even letting me complete the process - it will crash immediately aftering I've indexed the file and then whenever I try and save the AVS script. Even if I edit the AVS file after the crash - it does appear to successfully save - I still get a crash whenever I even selected it in MEGUI.

However, even though I've been using the universal installer I am only on 3.6.0 which is far behind the version you are on. I will swap to that and see what happens. If not I'm not sure what to do for the minute.

Crashing even with 3.7.2+...

[LouieChuckyMerry]

Just for S's & G's, load your .mkv source (not the .m2v) into MeGUI and, with the default L-SMASH indexer, select the profile "SMD-1080p[Film]".