VideoHelp Forum

+ Reply to Thread
Page 1 of 4
1 2 3 ... LastLast
Results 1 to 30 of 94
Thread
  1. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    I have a customer with 15 Video8 tapes and wants them done before Christmas, QTGMC in fast setting averages 10 fps, a 2 hrs video takes about 12 hours to de-interlace not including the time to capture and encode, Even if I do one tape a day, say start one process when I go to work and another process when I go to bed, that's over 15 days. What other settings can be used in QTGMC to make it go close to half real time without sacrificing too much quality?
    Quote Quote  
  2. Video Restorer lordsmurf's Avatar
    Join Date
    Jun 2003
    Location
    dFAQ.us/lordsmurf
    Search Comp PM
    QTGMC, preset=faster, batch encode in Hybrid.
    QTGMC is faster than realtime on my i7-6700K. Deinterlace + encode only 15 videos would be an overnight affair.
    Quote Quote  
  3. With NTSC SD video I get around 290 fps deinterlacing with QTGMC(preset="fast") with a script like:

    Code:
    AviSource("DV.avi") 
    QTGMC(preset="fast")
    Prefetch(8)
    Even running single threaded I get around 75 fps. That's with 64 bit AviSynth+ on an 8 core 16 thread CPU. And it's just the AVI decoding and QGTMC() (File -> Run Video Analysis Pass in VirtualDub2). IT does not include the encoding or any other filters. Encode that script with x264 CLI at the slow preset I get 180+ fps.

    If you aren't getting 100 percent CPU usage with a single encoding you can run two or more at the same time to increase your overall throughput.
    Quote Quote  
  4. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    I think time for a new computer:
    I have an all in one AMD A6-7310 APU 2GHz 8GB RAM and Win 10 64 bit, Not the greatest I think. What's weird is CPU is only 50% and RAM at around 45% and I'm not running anything in the background.

    I have a very fast laptop but I don't use it for this kind of work.

    The way I'm using QTGMC is by dropping the script file in vdub2, Is there a better way?
    Last edited by dellsam34; 6th Dec 2020 at 18:54. Reason: Added a question
    Quote Quote  
  5. Originally Posted by dellsam34 View Post
    I have an all in one AMD A6-7310 APU 2GHz 8GB RAM and Win 10 64 bit, Not the greatest I think.
    Still, I'd expect better than 10 fps.

    Originally Posted by dellsam34 View Post
    What's weird is CPU is only 50% and RAM at around 45% and I'm not running anything in the background.
    Are you running a multithreaded build of AviSynth? Are you using any other filtering? What encoder and what settings? 32 bit or 64 bit? What kind of source? For example, uncompressed video may be limited by disk speed, not processing speed.

    Originally Posted by dellsam34 View Post
    The way I'm using QTGMC is by dropping the script file in vdub2, Is there a better way?
    That's how I measured the QTGMC speed above. Open the AVS file with AviSynth and select File -> Run Video Analysis Pass. That reads frames linearly as fast as AviSynth can process them. Are you using any AviSyth filters?

    You can track down the cause of the slowness by eliminating QTGMC and any other filters. Run the analysis pass. How fast is it? That tells you how fast you can read the source video. Add one filter back, compare speed. Add another filter, etc.
    Quote Quote  
  6. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    All what I am using is:
    AviSource("path\file.avi")
    QTGMC(Preset="fast")
    No additional filters
    I don't know what avisynth I have, All what I know is I had a nightmare getting all the dependencies to get it to work, How do I run the analysis pass?
    Quote Quote  
  7. Create a script with only:

    Code:
    Version()
    Open it in VirtualDub2. What does it say?

    Or try adding

    Code:
    prefetch(4)
    to the end of your script (like in my example above). If you get an error you are not using AviSynth+. The older AviSynth doesn't support that command and isn't multithreaded unless you go out of your way to find and install a multithreaded version. If the script is still working see if it processes any faster.
    Quote Quote  
  8. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    It says AviSynth+ 3.5 (r3043, master, x86_64)
    What prefetch(4) does?
    Quote Quote  
  9. Originally Posted by dellsam34 View Post
    It says AviSynth+ 3.5 (r3043, master, x86_64)
    Good.

    Originally Posted by dellsam34 View Post
    What prefetch(4) does?
    Basically it enables multithreading with 4 threads. 2 might be enough for you. Try 2, 3, and 4 and see which gives the most throughput.
    Quote Quote  
  10. another option is using vapoursynth, when using QTGMC() with just simple preset like fast, it is running 100% of CPU,

    so you can try Hybrid and choose vapoursynth, if having script without fancy filters that are setup for avisynth only
    Quote Quote  
  11. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    With prefetch(4) I'm topping 19 fps, still not practical for the project.

    Image Attached Thumbnails Click image for larger version

Name:	Untitled.jpg
Views:	539
Size:	95.4 KB
ID:	56119  

    Last edited by dellsam34; 6th Dec 2020 at 23:10.
    Quote Quote  
  12. Originally Posted by dellsam34 View Post
    With prefetch(4) I'm topping 19 fps, still not practical for the project.
    How Important is deinterlacing quality for this project? Try some of the faster settings and see if you can live with one of them:

    # The "Preset" used selects sensible settings for a given encoding speed. Choose a preset from:
    # "Placebo", "Very Slow", "Slower", "Slow", "Medium", "Fast", "Faster", "Very Fast", "Super Fast", "Ultra Fast" & "Draft"
    Quote Quote  
  13. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    Well, They want the highest quality possible because they don't have a way to store the original large files, So once encoded that's it no more going back, I tried to convince them to buy a hard drive but they don't want to deal with large files. I may have to tell them that I will not be able to get this done in the time frame they wanted.
    Quote Quote  
  14. borrow/hijack/recruit your friends / neighbors/ family/ 5th cousin's computers

    Tell them it will keep the house warm in the winter

    If one of them has a fast GPU, there is also a GPU cuda version of QTGMC that runs much faster (avisynth cuda , KTGMC)
    Quote Quote  
  15. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    With "faster" I'm getting close to 25 fps, I might be able to get by this way. For encoding I will just use Hybrid like LS suggested.
    Quote Quote  
  16. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    If I have to get an all in one PC what should I be looking for in terms of a CPU that can process a 2hr file within minutes instead of hours?
    Quote Quote  
  17. Originally Posted by dellsam34 View Post
    Well, They want the highest quality possible because they don't have a way to store the original large files, So once encoded that's it no more going back, I tried to convince them to buy a hard drive but they don't want to deal with large files. I may have to tell them that I will not be able to get this done in the time frame they wanted.
    Why not just encode it as interlaced? x264 does it very efficiently. I understand the purpose is for some kind of 'archiving'. No need to deinterlace.
    Quote Quote  
  18. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    De-interlacing is part of the deal, they have the camcorder connected to the TV and they didn't like the interlacing effects, So I told them there is a software that can remove that. I know there is a lot of ways to get them done "legacy box" style but I don't feel comfortable doing that, I would rather not take the project than feel guilty about it.
    Quote Quote  
  19. Just to jump in, it looks like you've only got one hdd in the system - admittedly a SSD - but would he see any improvement with more? Also, have you checked that your Win10 is running with the AHCI drivers for your Hard disk controllers? (device manager, some mention of 'AHCI' in the drivers of hard drive controllers section. There are also some registry entries you can check, but I can't remember what they are off the top of my head.) Also, assuming you're running some internet security, have you excluded any temp folders and destination folders that your setup is using from a possibly overzealous security program. Whitelist your encoding program as well, if you can, just for good measure.) And, obviously, disconnect from internet whilst encoding.

    I'm guessing you've probably got all this covered, but I would be interested to know what the thoughts are on multiple SSDs for encoding. Good luck.
    "Well, my days of not taking you seriously are certainly coming to a middle."
    Quote Quote  
  20. Originally Posted by dellsam34 View Post
    With prefetch(4) I'm topping 19 fps
    3 threads might give a frame or two more than 4. Did you try it?
    Quote Quote  
  21. Originally Posted by TimA-C View Post
    Just to jump in, it looks like you've only got one hdd in the system - admittedly a SSD - but would he see any improvement with more?
    Extremely unlikely.
    Quote Quote  
  22. And what codec and color format is your source?

    Another thing to check: what color format is coming out of AviSource?

    Code:
    AviSource("path\file.avi")
    Info()
    If it's not YV12 try forcing it:

    Code:
    AviSource("path\file.avi", )
    QTGMC(preset="fast", pixel_type="YV12")
    That may give slightly faster processing.
    Quote Quote  
  23. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    Originally Posted by TimA-C View Post
    Just to jump in, it looks like you've only got one hdd in the system - admittedly a SSD - but would he see any improvement with more? Also, have you checked that your Win10 is running with the AHCI drivers for your Hard disk controllers? (device manager, some mention of 'AHCI' in the drivers of hard drive controllers section. There are also some registry entries you can check, but I can't remember what they are off the top of my head.) Also, assuming you're running some internet security, have you excluded any temp folders and destination folders that your setup is using from a possibly overzealous security program. Whitelist your encoding program as well, if you can, just for good measure.) And, obviously, disconnect from internet whilst encoding.

    I'm guessing you've probably got all this covered, but I would be interested to know what the thoughts are on multiple SSDs for encoding. Good luck.
    No, The bottle neck is not the hard drive as you can see from the previous screen shot, it is hovering around 5% usage, It's the processor.


    Jagabo, I will try setting 3 when I get home.
    Quote Quote  
  24. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    With 3 cores only it's the same thing about 25 fps give or take, I think I have to live with this speed until I get a new computer.
    Quote Quote  
  25. Try using Yadif(mode=1, order=0). It's about twice as fast as QTGMC(preset="faster"). Though at this point other parts of the process may be limiting your total throughput. You might have to use order=1 instead of order=0, depending on the field order of your source.

    Yadif isn't as good as QTGMC but depending on the amount of motion and sharpness of your source you may not notice much difference. And you're losing a lot of the benefits of QTGMC by using preset="faster" anyway.
    Quote Quote  
  26. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    It's not just QTGMC, anything I process through vdub2, A crop followed by a resize with Lanczos3 averages 9 fps.

    So when I capture using MediaExpress in 8 bit I then run a quick pass with vdub2 and save as HuffYUV to be able to use the filters because they refuse to accept the raw captured files. Here is a HuffYUV sample I work from:
    Image Attached Files
    Quote Quote  
  27. crop and resize in the script ? or in vdub? Probably faster in the script

    superfast vs ultrafast is where the big dropoff occurs in terms of quality for QTGMC . That's where NNEDI3 gets switched out for yadif for parts of the EDImode subroutine . ie. superfast is the quickest preset you should use, otherwise use something else for QTGMC
    Quote Quote  
  28. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    No, using vdub built in filters, I don't usually crop and resize unless the customer insists on removing the edges, If I have to resize I will always resize to 1080x1440 for better results and that can take longer to process.

    Here is the above sample on youtube after being processed:
    https://youtu.be/HEsS3xmaUeA
    Quote Quote  
  29. Originally Posted by dellsam34 View Post
    It's not just QTGMC, anything I process through vdub2, A crop followed by a resize with Lanczos3 averages 9 fps.
    Does this 9 fps include encoding to h.264 or some other codec?

    The fps number you quote should be from the "run video analysis" option in the File menu. This will apply all the VDub filters, but will not encode. This lets you remove the time it takes to encode. It is possible that if you are using a really CPU-intensive codec (e.g., h.265) you might get some slow results, especially with 4K material. However, a simple crop, even for high-res material should run at or near real-time.
    Quote Quote  
  30. Capturing Memories dellsam34's Avatar
    Join Date
    Jan 2016
    Location
    Member Since 2005, Re-joined in 2016
    Search PM
    No, I haven't gotten to encoding yet, This is just de-interlacing at about 25 fps now thankfully. And cropping/resizing combined filters using vdub2 I'm getting about 9 fps. Again these are huge captured SD files in the order of 150GB minimum.
    Quote Quote  



Similar Threads