Staxrip deinterlace - extreme file sizes

[Traderbam]

Hi, I am looking for some advice on a scalable approach for deinterlacing files captured from Hi8 to lossless HuffYUV avi files. For an hour of captured footage this is around 40Gb which is fine. The problem is around storage requirements for processing this avi into a ProRes MOV file for editing. Based on the process below, I get a 100GB+ MOV file (not including the huge temporary project files that Staxrip generates)

Using Staxrip I am doing a QTGMC deinterlace with these settings (ProRes quality is set to Normal rather than HQ)

AssumeTFF()
QTGMC(preset="Medium", InputType=0, sourceMatch=3, sharpness=0.2, tr2=3, ediThreads=8)
Crop(0,8,0,0)
AddBorders(0,4,0,4)
Spline64Resize(%target_width%, %target_height%)

Is there anything in the QTGMC settings that I could modify to reduce size of the deinterlaced file? I don't want to remove the resize. Final target will most likely be MP4 or MOV via H265 codec.

Thanks

[poisondeathray]

What were you resizing it to ? Must be something larger - I wouldn't expect Huffyuv => QTGMC double rate detinterlace => Prores (Normal) to increase the filesize that much

Less sharp scaling kernal , higher denoising, less sharpening in QTGMC will reduce filesize slightly, but I doubt it would offset whatever width,height I suspect you are upscaling to .

[Traderbam]

Sorry missed that detail - resizing to 1440 x 1080.

Here are the details of the generated MOV. Anything suspect here? Perhaps I can try a lower quality of ProRes - LT perhaps?

For the script itself, is that set of parameters for QTGMC fairly standard or could that be overkill? They are pretty close to the defaults that Staxrip enforces for that input type.

G: MPEG-4, 105 GiB, 1 h 13 min, 205 Mb/s
V: ProRes, 422, 1440x1080, 50 FPS, 204 Mb/s
A: PCM, 1 536 kb/s, 2ch, 48.0 kHz, Default

Format                          : MPEG-4
Format profile                  : QuickTime
Codec ID                        : qt   0000.02 (qt  )
File size                       : 105 GiB
Duration                        : 1h 13mn
Overall bit rate mode           : VBR
Overall bit rate                : 205 Mbps
Writing application             : Lavf59.20.101

Video

ID                              : 1
Format                          : ProRes
Format version                  : Version 0
Format profile                  : 422
Codec ID                        : apcn
Duration                        : 1h 13mn
Bit rate mode                   : VBR
Bit rate                        : 204 Mbps
Width                           : 1440 pixel
Height                          : 1080 pixel
Display aspect ratio            : 4:3
Frame rate mode                 : CFR
Frame rate                      : 50.000 fps
Color space                     : YUV
Chroma subsampling              : 4:2:2
Scan type                       : Progressive
Bits/(Pixel*Frame)              : 2.621
Stream size                     : 104 GiB (99%)
Writing library                 : fmpg

Audio

ID                              : 2
Format                          : PCM
Format settings                 : Little / Signed
Codec ID                        : sowt
Duration                        : 1h 13mn
Bit rate mode                   : CBR
Bit rate                        : 1536 Kbps
Channel(s)                      : 2 channel
Channel layout                  : L R
Sampling rate                   : 48.0 KHz
Bit depth                       : 16 bit
Stream size                     : 803 MiB (1%)
Default                         : Yes
Alternate group                 : 1

[poisondeathray]

Yes you can lose lower quality LT , but why bother to capture lossless in the first place ? The main culprit is the upscaling

Why don't you upscale at the last step , after editing ? Editing lower resolution files is faster, takes less storage

What kinds of things are you doing at the "for editing" stage ?

[Traderbam]

Process will be to import into Resolve for editing/cuts/titles and minimal tweaks like shadows/highlights/temp etc.

My thinking was that I want to preserve the best quality up until the final delivery format to H265 for YouTube. Would it be better to do a simple deinterlace to ProRes and leave the resize to be handled at render time via Resolve? Will it support a Spline64 approach? All the script samples I looked at seemed to have deinterlace, crop, borders etc in the same sequence as upscale.

[poisondeathray]

Process will be to import into Resolve for editing/cuts/titles and minimal tweaks like shadows/highlights/temp etc.

My thinking was that I want to preserve the best quality up until the final delivery format to H265 for YouTube. Would it be better to do a simple deinterlace to ProRes and leave the resize to be handled at render time via Resolve? Will it support a Spline64 approach? All the script samples I looked at seemed to have deinterlace, crop, borders etc in the same sequence as upscale.

There is nothing special about Spline64 - it's a sharper resizer kernel , but that's it. It's prone to ringing artifacts like any sharp kernel. You can use other resizers and sharpen to get similar or better results. Or upscale with other methods

If you had some super complex upscaling method , I could see encoding to an intermediate (because processing time takes so long) , but not Spline64 . There is no way I would pre upscale with Spline64 to edit, there are more negatives than benefits

But titles you mentioned - things like graphics, overlays are something you might need to do at the higher resolution, because if you upscale titles, graphics, it will look like crap. The other edits/cuts/small color tweaks do not need to be done at higher resolution. For me, I would plan the project. If I was "married" to Spline64, then I'd use it on only the sections that I really need it, otherwise you slow down editing, use extra storage space, for nothing

[Traderbam]

I'm definitely not married to any process or approach. Other than the deinterlace itself, and needing a format that I can import into Resolve then I'm flexible.

I removed the upscale and tried again with source file - the resulting file is only about 1GB larger than source.

I will have some titles and overlays and don't generally have a problem editing the larger file size but keen to learn about better options for upscaling. Is there a recommended resource I can read up on around the alternatives to spline64? Majority of my source will be Hi8 with some VHS to follow.

Anything look "odd" in the QTGMC parameters that I have specified?

Appreciate any guidance here.

[Traderbam]

Just to add to this - if upscaling to something like 1440x1080 is it important to consider a resharpen AFTER upscale or is this not necessary?

My workflow will be something like...

AssumeTFF()
QTGMC(Preset="Fast")

Crop(16,0,-32,-12) #Cropping right hand side distortion on Sony's
AddBorders(16,6,16,6) #Add borders to make dimensions 704x576
LanczosResize(%target_width%,%target_height%)

Any comments/suggestions? Thanks

[poisondeathray]

I will have some titles and overlays and don't generally have a problem editing the larger file size but keen to learn about better options for upscaling. Is there a recommended resource I can read up on around the alternatives to spline64? Majority of my source will be Hi8 with some VHS to follow.

VHS, Hi8 sources typically do not upscale very well . There are newer machine learning scalers, but they tend to not work well on VHS, Hi8 sources - there are many other issues to deal with, and you upscale artifacts

A proper capture is probably the most important step, then addressing the various issues like noise . Many have oversharpening artifacts and ringing to begin with, and using a sharp resizer just makes the artifacts worse. You have to adjust to your source

There are many comparisons posted on typical scaling kernels, you have to search for them. Some have interactive graphs, slider comparisons. But they are typically done on clean sources and test patterns to identify the trade offs in blurring, ringing, sharpening, aliasing, artifacts . VHS, Hi8 is very different

if upscaling to something like 1440x1080 is it important to consider a resharpen AFTER upscale or is this not necessary?

Depends on the source, the scaling used

Typically you will have decide on the tradeoffs. Sharpening artifacts (if you didn't remove them), ringing, noise accentuation. It's something you have to preview and try out

Many people don't bother upscaling VHS and Hi8 sources, or it depends on the reasons for upscaling . It boils down to what benefit are you gaining over the player or display performing the upscaling ? Otherwise it's a waste of bandwidth and filesize. Some displays upscale well, some do not. Another valid reason for upscaling is youtube, to preserve 50p/59.94p - you need minimum 720p

[Traderbam]

I will have some titles and overlays and don't generally have a problem editing the larger file size but keen to learn about better options for upscaling. Is there a recommended resource I can read up on around the alternatives to spline64? Majority of my source will be Hi8 with some VHS to follow.

VHS, Hi8 sources typically do not upscale very well . There are newer machine learning scalers, but they tend to not work well on VHS, Hi8 sources - there are many other issues to deal with, and you upscale artifacts

A proper capture is probably the most important step, then addressing the various issues like noise . Many have oversharpening artifacts and ringing to begin with, and using a sharp resizer just makes the artifacts worse. You have to adjust to your source

There are many comparisons posted on typical scaling kernels, you have to search for them. Some have interactive graphs, slider comparisons. But they are typically done on clean sources and test patterns to identify the trade offs in blurring, ringing, sharpening, aliasing, artifacts . VHS, Hi8 is very different

if upscaling to something like 1440x1080 is it important to consider a resharpen AFTER upscale or is this not necessary?

Depends on the source, the scaling used

Typically you will have decide on the tradeoffs. Sharpening artifacts (if you didn't remove them), ringing, noise accentuation. It's something you have to preview and try out

Many people don't bother upscaling VHS and Hi8 sources, or it depends on the reasons for upscaling . It boils down to what benefit are you gaining over the player or display performing the upscaling ? Otherwise it's a waste of bandwidth and filesize. Some displays upscale well, some do not. Another valid reason for upscaling is youtube, to preserve 50p/59.94p - you need minimum 720p

Yes YouTube is one of my main considerations. 720 or ideally 1080 should get "less bad" treatment by the YT reencoding. For anything else like sharing clips with family/friends I can maintain the source aspect ratio and in some cases avoid the need to deinterlace at all.

[poisondeathray]

A fairly conservative machine learning upscaler is NNEDI3_rpow2 - it's older but much less prone to cause weird distortions compared to the newer generation of machine learning scalers. The main benefit is less aliasing, jaggy artifacts for the upscale compared to Lanczos, or Spline64 . People have used this for many years upscaling for YT, including VHS sources. Resolve doesn't have it, you'd have to either prescale for import (and deal with the filesize issues), or export from resolve then post scale for the final distribution format

[Traderbam]

Thanks will research NNDEDI3. Is that available as a module/plugin I can use in an Avisynth script?

[poisondeathray]

Thanks will research NNDEDI3. Is that available as a module/plugin I can use in an Avisynth script?

Yes, avisynth or vapoursynth

[Traderbam]

I've swapped out my LanczosResize for the NNEDI3_rpow2 approach but tbh I didn't notice any difference in the resulting file. Few more questions please to help clarify my thinking

Crop(16,0,-32,-12) #Cropping right hand side distortion on Sony's
AddBorders(16,6,16,6) #Add borders to make dimensions 704x576

I have "bad" sections to either side of the frames so I need to do quite an extensive crop. When adding borders back in, given online/youtube is the final destination in most cases, do I need to perform the 720x576 to 704x576 adjustment, or can I just pad my crop with borders back to the 720x576?

Does either of these dimensions matter for my upscale to 1440x1080 e.g. if this is my target, should I be resizing from 720x576, 704x576 or does it not matter?

nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
aWarpSharp(depth=5)
Sharpen(0.2)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1440, fheight=1080)
aWarpSharp(depth=5)
Sharpen(0.2)

I've read elsewhere that it works well to do the upscale in steps as per above with sharpening at each stage. In other places I've read that upscale should be done in a single step to avoid throwing away quality at each step. Any thoughts on the best approach?

[lollo]

I've read elsewhere that it works well to do the upscale in steps as per above with sharpening at each stage. In other places I've read that upscale should be done in a single step to avoid throwing away quality at each step. Any thoughts on the best approach?

In general, for "film" transferts (see videofred's scripts) and animation (see some of the jagabo scripts) a sharpening at different stages is more effective; for VHS or similar captures, 1 single sharpening, before or after the upscale according to the source is better.

In any case, always experiment with your own source the different techniques, because YMMV!

[poisondeathray]

I've swapped out my LanczosResize for the NNEDI3_rpow2 approach but tbh I didn't notice any difference in the resulting file.

That could be "normal" for some types of sources, especially lower quality sources - The effective resolution of the source is so low, the lines and details are are so "thick" and fuzzy/blurry - that the aliasing artifacts that you get with lanczos / spline kernels are less noticable.

Crop(16,0,-32,-12) #Cropping right hand side distortion on Sony's
AddBorders(16,6,16,6) #Add borders to make dimensions 704x576

I have "bad" sections to either side of the frames so I need to do quite an extensive crop. When adding borders back in, given online/youtube is the final destination in most cases, do I need to perform the 720x576 to 704x576 adjustment, or can I just pad my crop with borders back to the 720x576?

Up to you , test to see what you like

If you have other sections treated differently - and the padding or AR changes suddenly, some people might notice, some might not . I think more people would notice bars than a small AR change . Very few people will notice small AR differences. Some people don't like pillarbars at all, and don't mind small AR difference. Some people need perfect AR . But if there are family members in the video, tread carefully if you make them "fatter" :P , that should influence your decision

If you decide to add bars, technically you should do it at the end after upscaling , because you contaminate the edges when you resample with bars . Many resampling kernels look at a 3x3 radius , or 5x5 radius, so you are getting black pixels averaged into the calculation . Most people will not notice the difference

[Traderbam]

Useful info. Thanks. I think I will simplify the process to find something that works for as many of my Hi8/VHS sources as possible. I don't think I want to get into testing a process for every single tape/file otherwise I will never get anywhere!

I will likely stick with

QTGMC deinterlace "fast"
Crop bad regions left/right
Add Borders to maintain 720x576
Resize with Lanczos to 1440x1080 for YouTube

I will be storing my Lossless HuffYUV avi files anyway so can always come back to things later if really essential but sounds like the above will work for my needs.