HCenc encoding problem (2nd pass)?

[ecc]

I was using HCEncGUI to encode a film using an AviSynth script that I'd used before to repal a PAL-NTSC converted disc and then slow down the resulting progressive 24.975 to 23.976. I've had success with this before on a 100 minute film. This particular project at NTSC speed ran 130 minutes. On the first pass (which took nearly 4 hours) everything seemed okay but as the second pass started, it looked okay but then began to break up into a heavily pixellated mess. I stopped the encoding and took some screencaps with PowerDVD. After the point where I stopped, it looks okay (I'm guessing thats from the previous encode).

Does HCEncGUI have a problem encoding 2+ hour films (at single layer bitrate - the original m2v was also below 4 GB) or is it the speed of my computer (the 100 minute project and another 90 minute film I used this .avs script with took about five hours total for 2 pass encoding)?

Here are the caps (resized to 480 x 360):











BTW, here is my .avs script but I don't think there's anything wrong with it:

loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\repal.dll")
mpeg2source("d:\dvd\video_ts\VTS_01_1.demuxed.d2v" )
tdeint(mode=1)
repal()
assumefps(23.976)
converttoyv12()

[guns1inger]

I have encoded longer footage without issue. Also, low bitrates is generally something HCenc is good at.

I don't see a loadplugin call for DGDecode.dll, so I assume you have a version installed in your plugins folder. Does it match the version that comes with the version of DGIndex you used to created the .d2v project with ? If it is not the same version you can get unpredictable behavior.

[ecc]

HCEncGUI has DGDecode.dll 1.4.8.0. My DGDecode folder has that version along with several others. 1.4.8.0 is also the version in the Avisynth folder.

[guns1inger]

The question you didn't really answer is

Is the version of DGDecode.dll being called implicitly (because it you don't call it explicitly) the same as the one that was used to create the .d2v file ?

Yes - No - Don't Know ?

I would use LoadPlugin and point it explicitly at the right .dll and try again.

[ecc]

As mentioned before the DGDecode.dll version in DGDecode is 1.4.8.0. The file corresponding to that in HCEncGUI's folder is called DGDecode13.dll. Is that the one I should load since its version number is the one in DGDecode?

[Alex_ander]

Unlikely the problem is in mismatched .d2v/DGDecode.dll: in that case error message from DGDecode.dll via AviSynth would appear in any script viewer (assuming the script was not given to HC without a preview). DGDecode.dll even checks differences between different betas.
I think it is not important which version was installed with HC (for encoding directly from .d2v) since the one loaded by AviSynth is used here.

[manono]

I think it is not important which version was installed with HC (for encoding directly from .d2v) since the one loaded by AviSynth is used here.

You'd think so, but how can you explain his problem otherwise? I've never heard of an HCEnc encode producing garbage like that, not when the script looks OK.

I just looked in my new HCEnc 022 folder and found 7 different DGDecode.dlls. Pretty weird, since there's no DGIndex in there. I have no idea what they're all for. I'm with guns1inger. Make sure you have the DGDecode.dll that goes with the DGIndex you're using. If you don't know if you have a matched pair, then get the latest DGMPGEnc package, make a fresh D2V, and load the DGDecode.dll in your script after sticking it in your plugins folder:

loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\repal.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\dgdecode.dll")
mpeg2source("d:\dvd\video_ts\VTS_01_1.demuxed.d2v" )
tdeint(mode=1)
repal()
assumefps(23.976)
converttoyv12()

I don't know if that alone will get rid of all the garbage in your encode, but you might as well learn to do things right. You shouldn't need the ConvertToYV12() in the script, because if your source is a DVD, they're all YV12 to begin with.

[Alex_ander]

I just looked in my new HCEnc 022 folder and found 7 different DGDecode.dlls. Pretty weird, since there's no DGIndex in there. I have no idea what they're all for.

A possible explanation: when encoding from .d2v directly, probably HC first reads version # from .d2v file (written by DGIndex for making its error messages), then loads the proper DGDecode version from its collection of renamed stable versions released between HC builds. I don't think those dll's are used when encoding from avs scripts.

[manono]

Yeah, that makes sense. Thanks.

[Alex_ander]

I've just gotten a confirmation to this by loading both ways one of my recent projects. For d2v HC wrote a message that DGDecode16 successfully loaded. Comparing file sizes, it's beta 10 of v.1.50 that I currently use. At loading avs no mentioning of DGDecode from HC, just a message that AviSynth initialized successfully.

[AlanHK]

After the point where I stopped, it looks okay (I'm guessing thats from the previous encode)

HCEnc does not make any video on the first pass, if you look at the m2v file it stays at zero size until the second pass. HCEnc collects statistics in a temporary file in its program folder during the first pass which it uses in the second pass when it produces the m2v.

[ecc]

A possible explanation: when encoding from .d2v directly, probably HC first reads version # from .d2v file (written by DGIndex for making its error messages), then loads the proper DGDecode version from its collection of renamed stable versions released between HC builds. I don't think those dll's are used when encoding from avs scripts.

Where does it say the version in the .d2v? At the top it says DGIndexProjectFile13 followed by the stats and then the video analysis (or whatever d00 1 0 0 0 0 92 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2 etc. are).

One thing I forgot to mention (which I took to be normal based on another thread) is that I almost never get a correct d2v file during the demux. Opening it up reveals that it references the VOB files rather than the m2v hence the input file error. Changing the # of files and the file name to the m2v file does not work so I have to create a new d2v file with the F4 command in DGIndex. Does that have any bearing on the situation?

[buzzqw]

try reducing the variable or error, try with this script

loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\repal.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\dgdecode.dll")
mpeg2source("d:\dvd\video_ts\VTS_01_1.demuxed.d2v" )
tdeint(mode=1)
trim(0,1500)

then with this

loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\repal.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\dgdecode.dll")
mpeg2source("d:\dvd\video_ts\VTS_01_1.demuxed.d2v" )
tdeint(mode=1)
trim(0,1500)

and last with this

loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\repal.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\dgdecode.dll")
mpeg2source("d:\dvd\video_ts\VTS_01_1.demuxed.d2v" )
tdeint(mode=1)
assumefps(23.976)
converttoyv12()
trim(0,1500)

maybe a problem between hc and some filter .. like repal.. or assumefps

BHH

[ecc]

maybe a problem between hc and some filter .. like repal.. or assumefps

Unless its something that's popped up recently as the result of some change, I've never had a problem with repal before since before this I used it to repal NTSC video and assumefps(25). From that I got 720x480 25fps video that I then used DGPulldown with 24.975 to 29.97 without error.

The problem of HCEnc tagging on additional footage after the encode started with assumefps(23.976) - its annoying but I could just trim that off the end (the very first frame of the wrong video occurs just after the very last frame of black at the end of the proper encode) but this blocking and distortion is new.

Since it took nearly four hours, I wonder if other computer processes like the screensaver (blank), the paging file usage, and Cacheman's attempts to recover memory which already slowed down the fps processing could have tripped up HCEnc's own processes (I think Cacheman suggests suspending its operations when performing some lengthy tasks). Does this sound likely?

[Alex_ander]

Where does it say the version in the .d2v? At the top it says DGIndexProjectFile13...

here: 'DGIndexProjectFile13'; in my test .d2v had 'DGIndexProjectFile16' (matches my v.1.5.0 b10), I guess file version of DGdecode can be the same for different betas in case version2version changes only touch DGIndex.

[ecc]

The DGDecode Info button in HCEncGUI 0.22 (which I just downloaded) finds all of the DGDecode.dll's in the program's DGDecode folder (6,10,11,12,13,15,16) which includes DGDecode13.dll (version 1.4.8.0) which is the version number of DGDecode.dll (no 13) in DGIndex and AviSynth 2.5.

HCEncGUI 0.21 which I have been using has the same valid .dlls as .22 but it also has an invalid version (#8 no version number included) too. Since the d2v mentions version 13, I'd assume this is what HCEncGUI would initialize from its own folder. Any chance the one invalid .dll might be tripping things up? I don't know why it would.

I've been using DGIndex.exe which I guess is an older version of DGMPGDec. Is that newer version less likely to result in an invalid .d2v in the first place?

EDIT: Never mind, the dgmpgdec zip I've downloaded has DGIndex as its .exe file, so I'm assuming the name change is just that (though it does have .dll 1.4.9.0 which is not included in either version of HCEncGUI - would I have to copy it into there to use the newer version of DGIndex?)

[Alex_ander]

...would I have to copy it into there to use the newer version of DGIndex?)

Nope:
a)You are encoding from avs script and HC doesn't use internal DGDecode files at that (those dll's are only used at encoding from d2v directly). You only have to use proper file version with the script (load plugin or put it in 'plugins' folder).
a)Even in d2v mode HC wouldn't find in its directory a file named DGDecode.dll (named without file version # or even named using any new number), tested by loading a valid d2v project and disabling DGDecode16.dll. However for encoding directly from d2v you can use a newer (than HC) version of DGDecode if you rename it using one of existing numbers and replace file in DGDecode directory. But that's not what you want now .

[hank315]

A possible explanation: when encoding from .d2v directly, probably HC first reads version # from .d2v file (written by DGIndex for making its error messages), then loads the proper DGDecode version from its collection of renamed stable versions released between HC builds. I don't think those dll's are used when encoding from avs scripts.

Yes, that's the way it's done, for d2v input HC reads the first line from the d2v file to identify the DGIndex project file version and then copy/rename the proper DGDecodeXX.dll to DGDecode.dll.
When the encode is done DGDecode.dll is trashed.
The DGDecode16.dll in the HC package is from dgmpgdec150 beta 11.

Using Avisynth input with mpeg2source(), Avisynth will generate an error if there's a DGIndex/DGDecode mismatch.

[ecc]

Tried it again with a different film and HCEncGUI 0.22 and noticed the error message this time. Here is the log:


-----------------------------------------
| HCenc - MPEG2 encoder - rel. 0.22.0.0 |
-----------------------------------------

MPEG profile@level: MP@ML
input: c:\documents and settings\**** *******\desktop\velutto.avs
output: D:\BW EM\VIDEO_TS\VTS_01_1.demuxed.m2v

--------------------
| encoder settings |
--------------------

profile: FAST
frames: 0 141125
framerate: 23.976
aspect ratio: 16:9
bitrate Kb/s: 4423
max. bitrate Kb/s: 9608
pulldown: no
closed gops: no
VBV check: yes
scene change det.: no
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 9
scan method: auto
bias: 0
chapter frames: 0
time code: 0 0 0 0
CPU: SSE3
priority: idle
SMP active: no
matrix: MPEG
luminance gain: no

------------------
| source stats |
------------------

nr. of frames in source: 141126
width*height: 720x480
fps: 23.976
nr. of frames to encode: 141126
frames to encode: 0 - 141125

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 3:08:09 (11289.39 s)
fps: 12.5

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 4423 kb/s
est. outfile length: 3178025 kB
intermediate encoding time: 0.8 s

---------------------
| encoding - pass 2 |
---------------------

*** ERROR, source mismatch in pass 2 starting at frame: 695

[Alex_ander]

Maybe something with output path: are you trying to rewrite video demuxed by DGIndex?

[45tripp]

There's a 'reload avisynth' option under the settings 3 tab,
try setting that.

gl

[ecc]

That didn't work either. I got the same 2nd pass source mismatch error and garbage after the 3 1/2 hour first encode.

[ecc]

The problem seems to be the combination of repal and assumefps(23.976). I'm converting a PAL DVD resized to 720x480 with assumefps(23.976) with the timing of subs and audio adjusted and its in the second pass and there have been no problems so far. Before I tried the repal/assumefps(23.976) combo, I had been doing repal()/assumefps(25) - since the repal framerate is 24.975 though I use that framerate as the input in DGPulldown - with no problem.

The first repal()/23.976 conversions I did looked great but tagged on a strange recap of the contents of the last VOB - from which the m2v was made - at the end that I could just lop off but recent ones had that source mismatch error on the second pass.

Is it just a problem not previously known about repal or is it a problem of the order I put the scripts in? I've read that you crop first and then deinterlace but are there any rules about where to place assumefps (other than somewhere after repal)?

[manono]

I don't understand why you have the problem, and don't really think it's the AssumeFPS(23.976) in the script, but you can leave off the AssumeFPS(23.976), encode for 25fps as you did successfully before, and apply pulldown afterwards using DGPulldown set for its default 23.976->29.97. That'll serve the same purpose as the AssumeFPS(23.976fps) in the script.

Also, if you do that, you can (and probably should, if you are particular about the final file size) raise the bitrates by the amount of (original bitrate x (24.975/23.976)). That is, by raising the average and max bitrates by a factor of 1.0417, you'll come out with results equal to those had you encoded for 23.976fps to begin with.

I don't usually use HCEnc, but do what I described fairly often when using CCE and encoding silent films at non-compliant framerates.

[ecc]

apply pulldown afterwards using DGPulldown set for its default 23.976->29.97. That'll serve the same purpose as the AssumeFPS(23.976fps) in the script.

I didn't know you could do that. I was about to ask if there was either a way to do the framerate seperately or to do the other steps first and then run the modified m2v through HCEncGUI again with just the assumefps. I'll try that.

I don't usually use HCEnc, but do what I described fairly often when using CCE and encoding silent films at non-compliant framerates.

It's interesting you should mention that. Does that still result in a progressive picture with a silent film? A review of the new PAL disc of Nosferatu mentions that its interlaced because of its non-compliant framerate.

EDIT:

It definitely is the AssumeFPS(23.976)! The PAL encode I just finished has a larger file size than the bitrate calculator says it should and when I try get to the end of the film in DGIndex something goes wrong and the program freezes so I can't even cut what's at the end of it.

I even opened the m2v in a newer version of DGIndex and it freezes and says "caught an exception during decoding" and then continue yes/no but it won't continue and it too freezes and crashes.

I'm thinking I'll have to delete it, re-rip it, and encode just the deinterlacing and resizing and do the fps in DGPulldown. Hopefully dropping the assumefps from the script will mean that it encodes faster than the six hour total of this apparently wasted encode.

Is the Assumefps problem with avisynth or with HCencGUI? I'd like to know even though it seems like I can combine two steps in one by doing 23.976 to 29.97 in DGPulldown.

[manono]

Does that still result in a progressive picture with a silent film? A review of the new PAL disc of Nosferatu mentions that its interlaced because of its non-compliant framerate.

Pulldown can only be applied to a progressive source, so if it's progressive to begin with, it'll be progressive afterwards. All silent films on DVD, except some encoded for 23.976fps progressive for NTSC and those encoded for 25fps progressive for PAL, are interlaced. We can do things in software that the DVD production houses can't do in hardware. I have heard that a few PAL silent films on DVD use stretch printing and are progressive, but have yet to see any myself. Stretch printing is when they encode duplicate frames into the video. For example, if the film is 20fps and is being encoded for PAL DVD, they might add in one duplicate frame every 4 frames to bring the framerate up from 20 to 25fps. This isn't the ideal way to increase the framerate, as it makes the output play noticeably jerky. Much better would be to encode for 20fps (25fps actually, I guess) and then apply pulldown for 20->25fps. That way you get added duplicate fields upon playback, and it plays much more smoothly than when you have duplicate frames. You can encode as progressive down to 2/3 the output framerate. For PAL this is 16.67fps and for NTSC this is 19.98fps. Below those framerates you have to add in duplicate frames. But that's true even for retail DVDs. And, of course, if the source is interlaced, as is the new Nosfaratu DVD you mentioned, you have to make it progressive before encoding, with the use of the proper field matching/decimation, as is possible with the use of TIVTC or some other AviSynth IVTC.

Is the Assumefps problem with avisynth or with HCencGUI? I'd like to know even though it seems like I can combine two steps in one by doing 23.976 to 29.97 in DGPulldown.

I wouldn't know. I think I remember using HCEnc a couple of times without a problem when doing a PAL2NTSC conversion from 25fps to 23.976fps. It didn't follow the RePAL filter that time, though.

[ecc]

Any ideas on how to remove 2:2 pulldown? The DVD of the PAL film I was converting is interlaced. In the previous script I just deinterlaced it - Tdeint(mode=0) - but I haven't found anything about avisynth scripts that remove 2:2 pulldown (DScaler seems to the only program that removes it but that's a player).

[Alex_ander]

If it is field shifted from progressive, something like this with Decomb plugin could help:
Telecide(order=1,guide=2)

[ecc]

I have no idea what I'm doing wrong. Now I've ended up with a nearly three hour file out of a 110 minute film. Same decoding error in DGIndex. For some reason, about an hours worth off footage gets tagged onto the file after the film ends and that was not how the original m2v looked. Dropping the assumefps made it encode much faster but I'm still getting the same error (minus the source mismatch).

Here's my script:

loadplugin("c:\program files\avisynth 2.5\plugins\dgdecode.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\decomb.dll")
loadplugin("c:\program files\avisynth 2.5\plugins\tdeint\tdeint.dll")
LoadPlugin("C:\program files\directvobsub\VSFilter.dll")
mpeg2source("d:\one\video_ts\vts_01_1.demuxed.d2v" )
Telecide(1,2)
tdeint(order=0)
lanczos4resize(720,480)
TextSub("d:\one\video_ts\vts_01_0.srt")
converttoyv12()

Once again, I don't think there's a problem with the script because the running time is correct when the avs is loaded into HCEncGUI. Its somewhere in the encoding that it all goes wrong. Why is it repeating that last section and making the entire file unusable?

[manono]

Where do you come up with these scripts? What's Telecide(1,2)? Is Guide=1 and Postprocessing=2? If so, and you have a PAL source and are trying to realign the fields, Guide should be 2. Since Telecide already has Postprocessing on by default to deinterlace any interlaced frames that might slip through, why are you adding the deinterlacer TDeint afterwards. It degrades the video, slows the encoding, and shouldn't be there. Why are you burning the subs into the video? Why not make the subs selectable like normal people? And finally, why the ConvertToYV12? As I said earlier, if from a DVD and if for HCEnc, it's already YV12 and you don't need that line.

Back to the Telecide line for a moment. Have you determined for sure that the fields are only phase shifted, as opposed to them being interlaced for another reason? Make your script like this temporarily:

mpeg2source("d:\one\video_ts\vts_01_1.demuxed.d2v" )
SeparateFields()

Open it in VDub(Mod), scroll to a place with movement and advance a frame at a time. If each pic is repeated once, then you have phase shifted fields. If you see that each field is different, or that many are blended/ghosted, double imaged, then something else is at work. And if the field order is really BFF, then you should probably have AssumeBFF() just before the Telecide line. This is all explained in the included docs.

I don't know why the extra bits are being encoded. If you open the script in VDub(Mod), check in File->File Information and length is correct, and if, as you said, the length is correct in HCEnc, then my guess is that you have some setting incorrect in HCEnc, perhaps the framerate.