Avisynth-QTGMC render gradually slowing down

[camwow13]

Hello! Sorry if this is a really noob question. I spent forever searching this problem out but I don't think I'm able to articulate it properly or just something weird has happened to me.

I'm rendering some DV Typ-2 files to h.264. They're sourced from some old Video8 tapes over FireWire and a Digital8 camcorder.
I'm using the same workflow I use with my VHS imports. original>Avisynth>QTGMC>ffmpeg>render. I'm de-interlacing the file to 60fps and upscaling to 720 with some sharpening for use on streaming sites. Essentially following this guide here. My install is identical to his.

My problem is that the render is slowing down. It starts at 120+fps and gradually drops to less than 10fps as the render goes on. The renders complete and look great, but there's nothing different about the end of the file compared to the beginning that would warrant such a huge drop. I'm using Avisynth+ with multithreading and everything seems to be working great.... but then it just slows down. I've read that I need to set a memory limit sometimes, but those threads are all really old and I've seen just as many saying that's no big deal now?? I've also adjusted Edithreads and Prefetch and that doesn't make any difference in this case.

Important stuff:

Avisynth Script:

SetFilterMTMode("QTGMC", 2)
FFmpegSource2("oldvideo.avi", atrack=1)
ConvertToYV12()
AssumeBFF()
QTGMC(Preset="Slow", Edithreads=4)
BilinearResize(720,540)
Crop(10,0,-22,-6)
Spline64Resize(960,720)
LimitedSharpenFaster()
Prefetch(20)

ffmpeg script

ffmpeg -i "oldvideo.avs" -ss 00:00:02 -t 01:03:49 -c:v libx264 -preset medium -crf 19 -tune grain -profile:v high -level 4.2 -movflags +faststart -c:a aac -b:a 256k "oldvideo.mp4"

I'm probably just doing something dumb here, Thanks in advance!

[jagabo]

I don't see anything obviously wrong with the script. Have you tried using AviSource() instead of FFmpegSource()? What kind of CPU usage are you seeing? Is memory usage increasing as the encoding progresses? How much? 32bit or 64bit?

[camwow13]

I don't see anything obviously wrong with the script. Have you tried using AviSource() instead of FFmpegSource()? What kind of CPU usage are you seeing? Is memory usage increasing as the encoding progresses? How much? 32bit or 64bit?

I haven't tried AviSource yet. I'll keep that in mind!
CPU usage starts out at around 100% on all 12 cores. The render I'm doing now has dropped to 8 fps and is using 23%.
Memory usage under processes is only around 1.6 gigs, but I'm showing over 22 gigs being used with no other programs running to account for that so I'm assuming it's the render. I didn't check usage at the start but it's been stuck at 22 gigs for a while now. I should probably watch the memory usage from a fresh render.
I'm using 64 bit

Thanks!

[jagabo]

I ran your script on an NTSC DV AVI clip on my old quad core i5 2500K. I specified 36 threads in the ffmpeg command line. I'm seeing ffmpeg run at about at about 20 fps, using about 1.5 GB of memory. After 10 minutes or so there's no change in speed, though memory usage has crept up to 1.6 GB. How long is it before you start seeing a slowdown?

By the way, the ConvertToYV12() in your script should be ConvertToYV12(interlaced=true).

[hello_hello]

That's been a very intermittent problem for quite a while. I don't think anybody has discovered the cause.
https://forum.doom9.org/showthread.php?p=1742591#post1742591

The good news is when your current encode has finished, the slowdown problem will probably go away for quite a while. Maybe even forever. It's a mystery.

As odd as it may seem, trimming the first 100 or so frames from the beginning of the script often prevents it happening. At least it sometimes did for me. I have no idea why as it makes no sense.

Encode the first 100 or so frames by adding Trim to the end of the script ie.Trim(0,99), and then edit it to encode the rest when the first part is done. Trim(100,0).
For x264, add --stitchable to the command line so you can append the two sections later.

Fortunately I haven't experienced a slowdown in a long time, but I recall a few encodes where I aborted the encoding job, split the encoded video on a key-frame near the end, added Trim() to the script to start encoding from that key-frame, wait for the slowdown, abort the encoding, rinse and repeat....

[camwow13]

I ran your script on an NTSC DV AVI clip on my old quad core i5 2500K. I specified 36 threads in the ffmpeg command line. I'm seeing ffmpeg run at about at about 20 fps, using about 1.5 GB of memory. After 10 minutes or so there's no change in speed, though memory usage has crept up to 1.6 GB. How long is it before you start seeing a slowdown?

By the way, the ConvertToYV12() in your script should be ConvertToYV12(interlaced=true).

Usually took around 15 minutes for it to start. Thanks for the tip on the YV12, didn't know that. It's been working fine without it, will this improve something?

Running more tests with memory limits and the other tips shared here.

[jagabo]

By the way, the ConvertToYV12() in your script should be ConvertToYV12(interlaced=true).

Usually took around 15 minutes for it to start. Thanks for the tip on the YV12, didn't know that. It's been working fine without it, will this improve something?

Without specifying interlaced=true you will get chroma blending/ghosting. The issue will be visible with large motions of colored objects. Especially obvious with cartoons.

[camwow13]

By the way, the ConvertToYV12() in your script should be ConvertToYV12(interlaced=true).

Usually took around 15 minutes for it to start. Thanks for the tip on the YV12, didn't know that. It's been working fine without it, will this improve something?

Without specifying interlaced=true you will get chroma blending/ghosting. The issue will be visible with large motions of colored objects. Especially obvious with cartoons.

Makes sense, fortunately haven't been dealing a lot with colors so can't see any serious bleeding issues, but I'll definitely keep that enabled from here on out. Thanks!

[camwow13]

That's been a very intermittent problem for quite a while. I don't think anybody has discovered the cause.
https://forum.doom9.org/showthread.php?p=1742591#post1742591

The good news is when your current encode has finished, the slowdown problem will probably go away for quite a while. Maybe even forever. It's a mystery.

As odd as it may seem, trimming the first 100 or so frames from the beginning of the script often prevents it happening. At least it sometimes did for me. I have no idea why as it makes no sense.

Encode the first 100 or so frames by adding Trim to the end of the script ie.Trim(0,99), and then edit it to encode the rest when the first part is done. Trim(100,0).
For x264, add --stitchable to the command line so you can append the two sections later.

Fortunately I haven't experienced a slowdown in a long time, but I recall a few encodes where I aborted the encoding job, split the encoded video on a key-frame near the end, added Trim() to the script to start encoding from that key-frame, wait for the slowdown, abort the encoding, rinse and repeat....

Humm, well you're right. The issue hasn't cropped up again, either because I fixed it or it's just random like you said Lol. The problem was repeatable on that one video file though. Anyway I have set a ram limit for around 2 gigs and that's kept my RAM from ballooning to over 22 gigs. Renders run at a nice even 120fps now.

If it becomes an issue again I'll definitely try the Trim() option. The solution to a weird problem like this is always something just as weird haha.

[jagabo]

fortunately haven't been dealing a lot with colors so can't see any serious bleeding issues, but I'll definitely keep that enabled from here on out. Thanks!

Here's what it looks like: https://forum.videohelp.com/threads/389089-Color-Ghosting Note how the green color of the wings is blended with the background (were the wings were in the other field).

[camwow13]

fortunately haven't been dealing a lot with colors so can't see any serious bleeding issues, but I'll definitely keep that enabled from here on out. Thanks!

Here's what it looks like: https://forum.videohelp.com/threads/389089-Color-Ghosting Note how the green color of the wings is blended with the background (were the wings were in the other field).

That's really interesting! Never saw like that or that bad in my videos so I guess I got lucky.