Gray flashing frames in H.264 Conversions - Bad Avisynth Script?

[GMaq]

Hi,
I am using the following script to convert MPEG-1 VCD files to H.264:

DirectShowSource("C:\Documents and Settings\Glen\Desktop\Edited Video\Music Videos\CCR - Born On the Bayou LIVE.mpg")
Crop(2,40,-2,-40)
Tweak(0,1.5,0,1.1)
LanczosResize(640, 360)
ConverttoYV12()
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Convolution3DYV12.dll")
Convolution3D(preset="vhsBQ")

The script loads and finishes fine in both SUPER and ssj2_goha's 3GP Converter Hack, however when played back in VLC or QT there are several gray frames at the very beginning, and at regular pulses throughout the first minute or so of footage, after that they seem to settle down and go away, Is my script bad? This problem is slightly less apparent in SUPER, but annoying nonetheless, Otherwise I'm pleased with the quality, Help as always appreciated, I'm a noob at Avisynth so suggestions are welcomed.

[manono]

Hi-

Is it like that when you open the script in VDub and scroll through the frames? I suspect so. Make a D2V project file using DGIndex, and then use MPEG2Source instead of DirectShowSource. See if that gives you better video.

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Convolution3DYV12.dll")
MPEG2Source("C:\Documents and Settings\Glen\Desktop\Edited Video\Music Videos\CCR - Born On the Bayou LIVE.mpg")
Crop(2,40,-2,-40)
Tweak(0,1.5,0,1.1)
LanczosResize(640, 360)
ConverttoYV12()
Convolution3D(preset="vhsBQ")

http://www.neuron2.net/dgmpgdec/QuickStart.html

[Soopafresh]

Yup, probably Directshowsource related. The version of DSS that comes with 2.57 is supposed to fix the grey frame issue. Or use manono's method via Mpeg2source and Dgindex (works with mpeg1 as well). That's a faster decoder than DSS.

[GMaq]

Thanks Manono,
I am using AVSP to create the scripts, when I play the scripts in WMP there are no Gray frames.
Will DG Index allow me to keep the source audio with the video?
The encoders I'm using don't allow me to encode the video and audio separately and although I've done all sorts of Muxing and Demuxing with regular MPEG-4, these files are for an iPod and have to be muxed a certain way, Thanks Again

[manono]

Hi-

Using MPEG2Source, you're encoding video alone. I only encode video and always mux in the audio afterwards. It's possible to stick the audio back into the script, but wouldn't recommend it. I much prefer to use a dedicated audio program, such as BeSweet, to handle audio. I find it hard to believe that your muxers won't allow a separate audio stream to be muxed in. But that said, yes, you can add the audio back in:

http://avisynth.org/AudioDub
http://avisynth.org/ResampleAudio

[GMaq]

@Soopafresh, Thanks for the tip but I am using Avisynth 2.57 already, I'll have to check if there are updates,

@Manono
I've used BeSweet and dbPowerAmp in the past with great results and I could use MeGUI, but it is hard to get decent info on what x264 settings to use for Hi-Res 640x480 H.264, then you have to use a special version of mp4Box and face east with two Toad's legs in your left back pocket at sunrise to get it to work with iTunes, That's why I've been using these 2 drag and drop encoders to get the iPod/iTunes compliance, Hopefully there will be further improvements in DirectShowSource, It's funny if I just drag and drop the source mpg. files they encode fine, but these are old clips that need some help, and my usual route of VirtualDub MPEG-2 doesn't support mp4 or H.264 in an m4v container as I'm sure you well know. Thanks again for you time and help

[Soopafresh]

You're overthinking things, GMaq. You can get great looking and completely compliant ipod compatible files without much grief at all. I posted a script, but can't seem find the link any longer, so here it is uploaded again.

http://www.bestsharing.com/files/khMe3217115/ipod__enc__v3.zip.html

http://mirror05.x264.nl/Sharktooth/force.php?file=./x264/x264-CLI_r620.7z

Just for testing purposes, feed it an xvid AVI that you've named Input.avi . I'll post the m4a encoder settings and the mp4box mux commands. Betcha a penny it'll work beautifully.

Here's an example. Should play quite nicely on your Ipod.



muxed.mp4

[GMaq]

Hi,
Thanks Soop!! You're on!

So all this CLI stuff is done inside of AviSynth? While I'm vulnerable I might as well be stupid! I've been pretty dependant on GUI's so you may have to speak slowly, it's all pretty new to me.

[Soopafresh]

Don't hesitate to ask any questions as you're trying this out. This stuff is like building IKEA furniture. The first one takes forever, but once you've figured it out, the other ones take no time at all. Don't get flipped out over Avisynth. You only need to know the basics. BTW, check out the MP4 posted above for an example of a completed file.

You're going to also need http://nic.dnsalias.com/NicMP4Box.zip (the muxer)
and
http://www.nero.com/nerodigital/eng/Nero_Digital_Audio.html (the audio encoder)

-------------------------------------------The VIDEO Part--------------------------------------------
When you unzip the ipod__enc.zip file, you'll find two script files - Let's take a look at both of them.

ipod.avs -- the avisynth script . These settings assume you're feeding the x264 encoder an AVI file (Xvid, Divx, etc) that is 16:9 aspect ratio, hence the 640,352 resizing settings on the last line. The name of the file were wrapping inside of the avisynth script is called input.avi . Pretty simple. The FPS lines normally aren't needed unless your source file is some non standard frame rate. The ipod will accept 23.976-30fps.

setmemorymax(512) 

avisource("input.avi")
ConvertFPS(29.97)
assumefps(29.97) 
bicubicresize(640,352)

converttoyv12()

Ipod_Enc.bat - the batch file you'll run to transcode the video to ipod compatible settings
The bitrate setting should be the only part to concern yourself with, but these settings are pretty well optimized. We're doing a 2 pass encode, btw.

@echo off


 "x264.exe" --pass 1 --bitrate 900 --stats ".stats" --level 3 --keyint 240 --min-keyint 24 --no-fast-pskip --nf --no-cabac --subme 6 --analyse p8x8,b8x8,i4x4,p4x4 --vbv-maxrate 10000 --qcomp 0 --scenecut 50 --me umh --progress --no-psnr --output "ipod_vid.m4v" "ipod.avs"

"x264.exe" --pass 2 --bitrate 900 --stats ".stats" --level 3 --keyint 240 --min-keyint 24 --no-fast-pskip --nf --no-cabac --subme 6 --analyse p8x8,b8x8,i4x4,p4x4 --vbv-maxrate 10000 --qcomp 0 --scenecut 50 --me umh --progress --no-psnr --output "ipod_vid.m4v" "ipod.avs"

pause

:end

----------------------------------------------------------------------------------------------------

You should have a folder with the following files in it for the video transcoding part of things


Input.AVI (your test xvid file to transcode)
x264.exe
ipod_enc.bat
Ipod.avs

Double click on Ipod_enc.bat to start the fun.

Notes - X264 can run multithreaded with around a 25% performance increase, if your PC supports it. We'll get to that part in a bit.

[GMaq]

Soopafresh,
1. Thanks for the silly-slap! 2. Thanks for everything else, this is better than a private tutor! I guess I know what I'm doing this evening, I'll give it a try and see how it goes and let you know, Your time and patience are greatly appreciated. GMaq

[Soopafresh]

Happy to help out

[rockstar]

pasted from my edit on the other thread before i saw your reply....this could be a vista issue (have to run it for work reasons - bah)

hmmm - when i run the batch i get

avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'ipod.avs'
avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'ipod.avs'
Press any key to continue . . .

the file i tested was a divx file - what are the valid inputs? how can i expand this to do a vob? can i convert from other formats?

[Soopafresh]

rockstar -- You do have Avisynth installed, right ?

Your file should be fine, but let's start by verifying the input.avi file is actually good.

Download this: https://forum.videohelp.com/images/guides/p1649665/avi_gain.rar

Unzip to it's own folder, place your input.avi file in the same folder and run the .bat file.

You'll want to play the fixed-input.avi after the process finishes


You'll soon learn how to input .VOB files into your script Let's all start with a basic example - an AVI file, then we'll get fancy schmacy with VOBs. Cool thing is, you'll have perfect A/V sync after you're through with this little tutorial. I don't want to go too fast. These are all building blocks to get you the best possible quality ipod trasncodes.

[GMaq]

Soopafresh,
Nice muxed clip, disturbing, but nice, I think I'm on board with most of this, I've gotten almost this far on the video side with MeGUI before, what confuses me is the muxing, I have also been using the Nero Audio encoder for some time so that's cool too, So I have my .m4v Video and my AAC Audio, what are the magic words to tell Nicmp4box and bring peace and harmony to the universe? I realize the purpose of our Xvid input file is a fail-safe to test and try, When I bring my MPEG-1 file to the party I should use DG Index instead of DirectShow Source right??

[Soopafresh]

Gmaq -
Open up the Nicmp4box zip file and the syntax is right there in the documentation. Can't miss it

NicMP4Box -add audio.m4a -add video.mp4 Muxed.mp4

I believe you must encode to M4A rather than AAC . There's a subtle difference if I'm not mistaken. The nero audio encoders automatically determine that by the target file extension you provide.


Quick Dgindex run through - Because you'll be entering some info into your ipod.avs script, you'll make your stay on this earth a lot easier by downloading this: http://www.mlin.net/files/PathCopyEx.msi Now, whenever you right-click on a file in explorer, you'll see a new option that says "Copy path to clipboard". Lemme tell ya, you'll be glad you use it.

Dgindex is 2 files - Dgindex.exe which you use to open your Mpeg files with, and DGdecode.dll, which YOU MUST place into your C:\Program Files\AviSynth 2.5\plugins\ folder

1) Install Pathcopyex.msi
2) Copy the dgdecode.dll into C:\Program Files\AviSynth 2.5\plugins\
3) Open Dgindex.exe File--> Open ---> (your.mpg)
4) Audio-->Output Method-->Demux All Tracks
5) Video-->FieldOperation-->Honor Pulldown Flags
6) File--->Save Project
7) It'll prompt you to save the .D2V file. Save it in the same folder as the Ipod.avs file.

mpeg2source("path_to_your_.D2V")
#Right click on d2v and copy path to clipboard to enter above
ConverttoYV12()
Crop(2,40,-2,-40)
Spline36Resize(640, 360)
Convolution3D(preset="vhsBQ")
Tweak(0,1.5,0,1.1)

As long as you copy any of your image enhancement or noise reduction plugins in the avisynth plugins folder, you won't have to worry about LoadPlugin("....") in your scripts.
Sorry about the code formatting above. You might as well delete the ipod.avs file and replace it with your script above, which you should save as Ipod.avs

Note - even though your source file is Mpeg1, the Dgindex app still requires the command
Mpeg2Source

Also, Convolution3D is kind of an old plugin. If you post a <2MB segment of your source file up here or on some shared file hosting place, I'll be glad to recommend a better denoiser.

[GMaq]

Thanks again Soop, You Rock!!! I'll mess around with it this weekend, lots to digest!

[Soopafresh]

Like I mentioned, snip a bit of your source file and post it somewhere.

[Soopafresh]

Rockstar - I figured out your message. Came up with a much enhanced version of the script, which also creates the audio and muxes the final product. Sorry for your inconvenience.


Here you go. Included an Input.avi just so we both have a point of reference. Test it out, please.

http://www.bestsharing.com/files/khMe3217115/ipod__enc__v3.zip.html

You can actually place a bunch of Xvid or Divx AVI files into the folder and the batch process will now convert them all into ipod compatible MP4 . Note: The AVI input files CANNOT have spaces in the name:

Bad: My Divx File.AVI
Good: My_Divx_File.AVI

Up next - a version for vob transcoding...


Mods - Sorry for posting 2 messages in a row....

[GMaq]

WoooooHoooo!
OK Soopafresh, You know how the old "People don't know what they like, they like what know" thing works, Well I took most of your advice, and applied it to MeGUI, which I do know, Using NicMp4Box in MeGUI and creating a x264 profile based on the one you showed me, as well as using Mpeg2Source in my script I was able to create an iPod compliant file, that plays back great. Finally!!! Thanks again, because this whole exercise taught me a lot about what's going on under the hood of these programs. One more question though, is there a command line I can add to NeroAACEncoder to normalize the audio?? I have posted the MeGUI x264 profile I created in case anyone else wants to try it. I am diggin' this forum large!!!

P.S. I still need some advice on Avisynth noise reducers, I'll post a sample clip soon...

x264%20ipod%20hi-res%202-pass.xml

[Soopafresh]

Right on bro, whatever makes it easier for you to get to the final result

Add the following to your script

ConvertAudioToFloat()
Ensurevbrmp3sync()
Normalize()

That'll boost the audio levels, IF megui actually processes the audio through avisynth.

If NOT, then it exceedingly easy to up the volume using one of three methods:

On a WAV souce:

Commandline - Normalize (download the updated Ipod_enc_v3 script above for the app and the syntax)

or

GUI - the levelator (great, but strong) http://www.gigavox.com/levelator/

on an AAC source

http://altosdesign.com/aacgain/

[GMaq]

Soopafresh,
MeGUI handles the audio separately and gives you a choice of AAC/M4a Encoders, then sends it to mp4box to be muxed, there is an "adjust volume automatically" box to tick but it doesn't do much, I'll check out the links, Can you recommend a good host so I can send you some clips for denoising advice?

[Soopafresh]

www.bestsharing.com - Just a 15-20 second sample will do.

[GMaq]

Hi Again,
Here's the file, thanks for taking time to look at it, It is MESSED up, This is how I got it, in PAL VCD MPEG-1, It is rare footage and very hard to find. It suffers from a noisy multi-generation source, obvious horrendous frame drops during capture, messy cropping and to top it all off MPEG-1 compression artifacts galore, even NeatVideo in VirtualDub did very little to improve it, Convolution 3D did help it somewhat, but I am interested in learning more about Avisynth Denoisers, Thanks again, You better start putting this on a tab!!

http://www.bestsharing.com/files/qsk1M217586/CCR-Clip.mpv.html

[Soopafresh]

Okey dokey. Lemme take a look. This will work, fret not.

Edit - This thing looks like it was originally NTSC dimensions and 16x9

Requires MVtools http://avisynth.org.ru/mvtools/mvtools-v1.6.2.zip . Copy dll into plugins folder.

SetMemoryMax(512)

mpeg2source("   ")

Converttoyv12()

Crop(0,24,0,24)

Pal2Ntsc()

selectevery(4,0,3)

spline36resize(320,176)


function PAL2NTSC(clip video)
{
backward_vec = video.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
forward_vec = video.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = video.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return video.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=60000,den=1001,idx=1,idx2=2)
}

Here's a sample of both, one at 25fps and the other at 29.97. Yeah, I know 23.976 should be used for film

http://www.bestsharing.com/files/4C61AWz217645/CC_ARE.zip.html

BTW, That looks like the Royal Albert Hall concert. There's probably some better sources available.

Here's a pic that looks about 1 generation better.

http://ccr-tribute.sverige.net/images/gallery/rah01.jpg

Finally, below is probably about the best you'll be able to get with this source. It uses "Super Resolution" to double the size of the video clip, then the resulting video is fed back into Avisynth and downsized. Uses an app called Videoenhancer to double the size of the image, then uses Virtualdub filters to deblock. NOT fast. 30 day full trial if you want to test it out. http://www.thedeemon.com/VideoEnhancer/ Here's a short clip of what it looks like:

ipod_vid_001.mp4

It's a bit better, but arguably not worth the time. You can kinda do the same with

Crop(0,24,0,24)
Spline36Resize(704,576)
Sharpen(0.6)
Spline36Resize(320,176)

Just go with the above script, and perhaps your tweak() settings. Problem with noise filtering is that it has already been done (by somebody many, many years ago), and what's left is smudge. Nothing really to clean.

[GMaq]

Soopafresh,
Nice work!! The improvement in motion is nothing short of incredible!! I viewed them double size and the MPEG-1 macroblocks are really suppressed, And you're right it is Albert Hall, I actually have a 20 Year old VHS copy with different camera angles, perhaps filmed on a different night, Unfortunately it was bounced about 6 times more than this, it's beyond filtering of any kind. I thought I'd won the lottery when this copy came along!! It's all over YouTube but by the time I revive it from FLV I'm no farther ahead. A couple of questions, can we go any larger with the resolution? I use my iPod 99% of the time for TV Out, I realize upsizing crap gives you bigger crap, Some other MPEG-1 stuff I've done has come out all right, Even if we could maintain the width of 352x192 or 352x208, I realize I'll just have to try it out and see for myself.
I must really thank you VERY MUCH for all of your efforts and time, Life goes by pretty fast in this day and age and people are generally maxxed out and busy beyond belief!! So being one of those people myself it makes me appreciate even more what a "Good Samaritan" you've been.

LONG LIVE SOOPAFRESH !!!!!

[Soopafresh]

Thanks for your kind words I guess the next largest size would be 480x272. It'll start looking pretty bad if you go much larger, but try the following.

Download eedi2 (this is a higher quality upsizer)

http://bengal.missouri.edu/~kes25c/EEDI2v092.zip

SetMemoryMax(512)

mpeg2source("   ") 

Converttoyv12()

Crop(0,24,0,24)

eedi2().turnleft().eedi2().turnright()#upsizes x2 in width and height & next line we downsize

spline36resize(640,352) #try 480,272 if the larger res looks too bad

Pal2Ntsc()

selectevery(4,0,3)




function PAL2NTSC(clip video)
{
backward_vec = video.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
forward_vec = video.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = video.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return video.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=60000,den=1001,idx=1,idx2=2)
}

[GMaq]

Soopafresh,
Now I've done it! Here is the result of running and encoding the script copied from the post above on the original CCR Clip.mpv I sent you. It's gone a little Picasso on me!? Take a look at your (in)convenience, Thanks yet again.

http://www.bestsharing.com/files/a6Msds217776/SoopTest.mp4.html

[Soopafresh]

Hmmm....

Whoops, it's the crop command Forgot the - on the bottom

Crop(0,24,0,-24)

Or Try

eedi2().turnleft().eedi2().turnright()#upsizes x2 in width and height & next line we downsize

Crop(0,48,0,-48) # double sized, double the crop

spline36resize(640,352)

[GMaq]

OK,
That looks much better! I should've noticed that crop thing myself, I don't know if it's the script or the x264 in-house deblocking but even upsized the macroblocks are gone! There are some artifacts in some higher motion scenes later in the (full) clip that look like they may be coming from the frame looping and blending that MV Tools is doing, I'm going to save the original script and have a look at the MV tools readme and play around with it. That iPod enc v3 really works well too, everything in one folder! One (hopefully) final question: I found an updated DirectshowSource.dll to the one that comes in Avisynth 2.5.7, it seems to work better without the Gray frames, On stuff I just want to do a quick n' dirty encode on do those Audio commands in a previous post work on any Directshow Audio source or only on .WAV sources? And the mp3vbrsync would only need to be in the script if the Audio was vbr mp3 right??

[Soopafresh]

Sure, you can try DirectshowSource on the audio. The Kevin Bacon of Avisynth plugins.
DSS is pretty cool because it covers just about anything that can be played by your specific Windows installation and codecs installed. So if your codecs and splitters are good, DSS will do the job. The operative term is if . I don't use it too much, but sometimes there's no other option.

Additional audio options you can use (modify to your liking - look at the Wav batch file):

https://forum.videohelp.com/viewtopic.php?t=320614

Most of the time, I'll process the audio outside of the Avisynth script, then mux the audio with the newly converted video. But that's just my own preference.

EnsureVbrMp3sync is supposedly a magic bullet for all vbr audio and doesn't harm things regardless.

All functions which change the frame rate have the tendency to create artifacts during fast moving scenes, hence the MVtools motion compensated method of changing the fps. That's an amazing plugin. Check out the documentation on additional things it can do. The MVdegrain2 function is a tremendous noise reducer for old grainy film. Too bad it is so slow.
Neatvideo uses the same motion compensating technology to denoise.


As an alternate to the Pal2NTSC function, you can try MotionProtectedFPS. Includes a dll and function that you'll need to copy into your plugins folder. See motion.zip below.

add to your script

import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avs")

replace

Pal2ntsc()

with

MotionProtectedFPS(30) #or try 29.97 > Can't remember this one

or

BlendFPS(29.97)


motion.zip