Youtube Workflow

[Sunk]

I plan to upload some of my captures to Youtube akin to several members like lollo and dellsam34. What's the general workflow in Avisynth to make a capture to be mangled by YT's encoder as little as possible? I know deinterlacing is one of the processes, but I've also seen some people like dellsam34 upscale their videos. How does one achieve that in VDub?

[lollo]

I use an AviSynth flow (partially described in my YT videos):

- deinterlace or inverse telecine or fields matching or nothing for "progressive" material
- denoise (you may want to skip this)
- change color matrix before upscaling
- upscale to 1440x1080 [to reduce YT degradation while compressing]
- sharpen (you may want to skip this)
- export as lossless 4:2:2 avi loading the AviSynth script in VirtualDub
- upload to YT

you may want to mask head switching point at some point in time.

If you have a loooong capture and do not wish to produce a laaaarge lossless avi file, which also takes loooong to upload, you can feed the AviSynth script to ffmpeg and encode to h264 with low crf (16). I experimented little differences doing this on the final YT video (but prefere the lossless approach if possible)

Using VirtualDub only the operations are the same, but the filters available for the tasks are lower quality. If you do not wish to use AviSynth, give Hybrid a try.

[Sunk]

- change color matrix before upscaling

By that I assume you mean "ConvertToYV<digit>"?

[SaurusX]

- change color matrix before upscaling

By that I assume you mean "ConvertToYV<digit>"?

No. lollo means to use ColorMatrix(mode="Rec.601->Rec.709"). This ensures that the decoder will present the colors correctly after the video has been resized to 1080 vertical.

[lollo]

Yes, SaurusX

A complete script here:

video_org=AviSource("<your capture>)

# cropping 
	crop_left=8	
	crop_top=12
	crop_right=16
	crop_bottom=10
video_org_crop=video_org.crop(crop_left,crop_top,-crop_right,-crop_bottom)

### de-interlacing
deinterlaced=video_org_crop.AssumeTFF().QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

### convert to YV16
deinterlaced_yv16=deinterlaced.convertToYV16()

### denoising
denoised_yv16=deinterlaced_yv16.TemporalDegrain2(degrainTR=3)

### convert to YUY2
denoised=denoised_yv16.convertToYUY2()

### change color matrix
denoised_cm=denoised.Colormatrix("Rec.601->Rec.709")

### add borders
denoised_cm_borders=denoised_cm.addborders((crop_left+crop_right)/2,(crop_top+crop_bottom)/2,(crop_left+crop_right)/2,(crop_top+crop_bottom)/2)

### upscale
upscaled=video_org.nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize", fwidth=1440, fheight=1080)

### convert to YV12
upscaled_yv12=upscaled.convertToYV12()

### sharpening
sharpened_yv12=upscaled_yv12.LSFmod(defaults="slow")

### convert to YUY2 with chroma from YUY2 color space
sharpened=sharpened_yv12.convertToYUY2().MergeChroma(upscaled)

return(sharpened)

No level correction, no correction to 704x576 proper PAR, no color correction, no defect removal, just basic restoration


edit: if you want to stick to Rec601 spec for DAR, just remove 8 pixels from left and right replacing

addborders((crop_left+crop_right)/2,(crop_top+crop_bottom)/2,(crop_left+crop_right)/2,(crop_top+crop_bottom)/2)

with

addborders((crop_left+crop_right)/2-8,(crop_top+crop_bottom)/2,(crop_left+crop_right)/2-8,(crop_top+crop_bottom)/2)

[pandy]

btw there is no sense in hq compression for YT as YT will recompress your uploaded video to won standards - use sufficiently high bitrate with fast encoding to get reasonable sized upload file (uploading uncompressed or losslessy compressed video is waste of time, energy and bandwidth).

[lollo]

btw there is no sense in hq compression for YT as YT will recompress your uploaded video to won standards - use sufficiently high bitrate with fast encoding to get reasonable sized upload file (uploading uncompressed or losslessy compressed video is waste of time, energy and bandwidth).

Not all aspects you mentioned are worse. Let me expalin with an example:

case 1: the previous avs script, loaded into VirtualDub to generate lossless output -> processing time: good; size: bad, uploading time: bad, quality: best

case 2: the previous avs script, loaded into ffmpeg to generate h264 output -> processing time: bad; size: good, uploading time: good, quality: lower

On digitalfaq I made a comparison of the two flows in term of final quality once encoded bt YT, and the dìfference was minimal (but present)

edit: found the digitalfaq post https://www.digitalfaq.com/forum/video-capture/11510-parts-build-pc-2.html#post79818

[dellsam34]

I usually agree with pandy but I had uploaded samples to YT with different resolutions, and YT compression algorithm compressed them differently with the highest resolution has the less compression, check my channel, I used static samples to dramatically show the compression artifacts. But yes lossless files do take bandwith and data during uploading, this is not a problem for me but for folks on a data cap or low data rate it is not recommended.
The samples are titleld "BrightEye 75 VS Static, TBC Stress Test" and are in 480p, 720p and 1080p.

[Alwyn]

In VDub:
To change the colour format, use the Convert Format filter (I did ask a question on VH some time ago and never got an answer so here goes):


[Attachment 71227 - Click to enlarge]
or

[Attachment 71229 - Click to enlarge]

I don't know which is the correct one.

To resize (to my preferred YT resolution):


[Attachment 71228 - Click to enlarge]

In the x264 encoder, leave both SARs as 1.

Re YT uploads, if your files are at least 1440P ie for SD, 1920x1440, YT will use the better VP9 codec for your videos. The frame rate and bitrate is irrelevant.

I prefer lossless uploads, but as Dellsham has noted, you may be limited. In that case, upload at as higher bitrate as you can stand, in H264.

[Bwaak]

I wonder whether full-blown NLEs automatically convert to 709 when the video is resized to HD.

[Alwyn]

I wonder whether full-blown NLEs automatically convert to 709 when the video is resized to HD.

Magix Movie Edit Pro, Magix Movie Studio 18 and on, and Magix Video Pro X all do.

[Sunk]

I seem to have encountered another problem: some parts appear to be interlaced, while others aren't. You can see in my sample that there's an interlaced bit when the cat guy rides on a skateboard and falls, but the rest of the footage is progressive (I think?). I don't know if I was capturing incorrectly, but quite a few commercials have that issue. What would be an ideal solution in this case? Use QTGMC with "fpsdivisor=2" parameter to keep the video at 25 FPS, or something else altogether?

Also, I'm curious, but how typical is this when it comes to commercials? Could anyone who lives in the PAL region share their experience?

[lollo]

Your video seems always "progressive" to me, i.e. the 2 fields are from the same moment in time.
Checked with:

video_org=AviSource("raptor.avi")

# separate fields tff
video_org_sep_tff=video_org.AssumeTFF().separateFields()

# separate fields bff
video_org_sep_bff=video_org.AssumeBFF().separateFields()

# separate fields tff even
video_org_sep_tff_even=video_org_sep_tff.SelectEven()

# separate fields tff odd
video_org_sep_tff_odd=video_org_sep_tff.SelectOdd()

	# to check if progressive or interlaced
return(video_org_sep_tff)

If you have a mix of interlaced/"progressive" fields you can apply something like

TDeint(mode=0, full=false, cthresh=9, chroma=false, MI=16, edeint=QTGMC(FPSDivisor=2), emask=TMM(mode=0), slow=2)

or QTMGC() to both, depending on the type of video

[Sunk]

Your video seems always "progressive" to me, i.e. the 2 fields are from the same moment in time.
Checked with:

video_org=AviSource("raptor.avi")

# separate fields tff
video_org_sep_tff=video_org.AssumeTFF().separateFields()

# separate fields bff
video_org_sep_bff=video_org.AssumeBFF().separateFields()

# separate fields tff even
video_org_sep_tff_even=video_org_sep_tff.SelectEven()

# separate fields tff odd
video_org_sep_tff_odd=video_org_sep_tff.SelectOdd()

	# to check if progressive or interlaced
return(video_org_sep_tff)

If you have a mix of interlaced/"progressive" fields you can apply something like

TDeint(mode=0, full=false, cthresh=9, chroma=false, MI=16, edeint=QTGMC(FPSDivisor=2), emask=TMM(mode=0), slow=2)

or QTMGC() to both, depending on the type of video

Curious, but why do you put "progressive" in quotation marks? Do you imply it's not a true progressive video?

[lollo]

In short:

Because the 720x576 frames of your video is built using two 720x288 fields that are at the same moment in time. Each of the fields arrive at 1/50th of a second.

In general we call progressive a video consisting of 720x576 frames, each arriving at 1/50th of a second


More detailed (and more confusing maybe):

The analog captured video is always "interlaced", i.e. made up of 50 fields (semi-frames) per second which make up 25 frames (frames) per second.

Referring to progressive, we mean a video of telecine origin, i.e. generated by the transfer from film to TV, in which two "similar" semi-frames are present every 1/50 sec, which contain the even or odd lines of the same original frame of the film. I call it PAL50Ip.

Interlaced refers to a video of television origin, i.e. generated by a video camera, in which two different semi-frames are present every 1/50 sec, because they refer to different instants of time, and which contain the even or odd lines of the frame that will go to to form. I call it PAL50Ii.

For example your video has progressive architecture (PAL50Ip)

The PAL50Ip and PAL50Ii are formed by 625 lines transmitted 25 times per second or 312.5 lines of a semi-frame transmitted 50 times per second.

For comparison, the PAL50P consists of 625 lines transmitted 50 times for each second, double the amount of information compared to the PAL50Ip and PAL50Ii cases.
Another possible format is PAL25P which is the perfect companion to telecine source video; it is similar to PAL50Ip with the difference that the lines are not divided into even and odd semi-frames, but are sent 25 times per second in progressive mode.

[Sunk]

I see. A bit of a head-scratcher, but interesting nonetheless. But this doesn't mean that the capture itself is bad, does it? I have some concerns that it could be my setup that's causing video to be "progressive". For the record, this is what I use:
JVC HR-S7600AM -> DMR-ES10 (line TBC, since the VCR one doesn't work with MESECAM) -> Kramer FC-400 -> PCIe SAA chip TV tuner card. The tuner is probably not an ideal solution, compared to something like a Radeon AIW card (which I actually happen to have, but I've no computers with the AGP slot). You can read more about its specs here. My model is H85: http://www.beholdtv.com/products/8series.htm

[lollo]

Your capture is fine from the field architecture point of view.

The original source shows phase shift for some sequence, fixable with a simple field matching operation (applicable to the whole video, the filter will shift the fields only when needed):

AssumeTFF().TFM(order=1,mode=0,PP=0,slow=2,field=1)

edit: the histogram suffers of crushed blacks and clipped whites, probably because the card. Try to use its procamp if possible for a better distibution of the levels:

[Sunk]

Yeah, I'm aware of blacks and whites. The card does have a procamp and I tried using to fix the levels, but I wasn't aware that the problem still persists. Truth be told, I've only used VDub's builtin histogram, which I've later found out was far from reliable. I don't think capturing with Matrox MXO2 LE would fix the problem since Matrox's drivers don't feature any sort of procamp at all (something I've touched upon in conversation with jagabo here), not to mention that when I've initially tried capturing with MXO2, it had the same problem with levels, albeit much more severe since I couldn't correct them at all.

[pandy]

btw there is no sense in hq compression for YT as YT will recompress your uploaded video to won standards - use sufficiently high bitrate with fast encoding to get reasonable sized upload file (uploading uncompressed or losslessy compressed video is waste of time, energy and bandwidth).

Not all aspects you mentioned are worse. Let me expalin with an example:

case 1: the previous avs script, loaded into VirtualDub to generate lossless output -> processing time: good; size: bad, uploading time: bad, quality: best

case 2: the previous avs script, loaded into ffmpeg to generate h264 output -> processing time: bad; size: good, uploading time: good, quality: lower

On digitalfaq I made a comparison of the two flows in term of final quality once encoded bt YT, and the dìfference was minimal (but present)

edit: found the digitalfaq post https://www.digitalfaq.com/forum/video-capture/11510-parts-build-pc-2.html#post79818

I usually agree with pandy but I had uploaded samples to YT with different resolutions, and YT compression algorithm compressed them differently with the highest resolution has the less compression, check my channel, I used static samples to dramatically show the compression artifacts. But yes lossless files do take bandwith and data during uploading, this is not a problem for me but for folks on a data cap or low data rate it is not recommended.
The samples are titleld "BrightEye 75 VS Static, TBC Stress Test" and are in 480p, 720p and 1080p.

My point was:
Use modern video codec in something similar to INTRA mode, it can be 'ultrafast' preset, tune like 'fastdecode' or(and) 'zerolatency' with for example CQP with qp=4..8..16, then results should be processing time: good (comparable to lossless or better if for example HW encoder used); size: good, uploading time: good, quality: good (comparable to lossless)

This can be tested:

@setlocal
@set FFMPG=C:\FF
@set PATH=%FFMPG%;%PATH%

@SET file=%1
@SET fname=%~n1

@ECHO file=%file%
@ECHO filename=%fname%

@set qpval=8
@echo QP value is: %qpval%

@SET x264opts="qp=%qpval%:qpmin=4:keyint=1:ref=0:bframes=0:b-adapt=0:no-weightb=1:weightp=0:no-mbtree=1:no-mixed-refs=1:chroma-me=0:subme=0:open_gop=0:sliced_threads=1:slices=-1:rc-lookahead=0:force_cfr=1:cabac=0:no-8x8dct=0:constrained_intra=0:no-psy=1:scenecut=0:trellis=0:no-deblock=1:aq-mode=0:threads=auto:no_psnr=1:no_ssim=1:pic_struct=1:aud=1:stitchable=1:ipratio=1.0"

@ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v libx264 -profile:v high -x264opts %x264opts% -x264-params %x264opts% -f null -

@pause

@ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v ffvhuff -f null -

@pause

@ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v huffyuv -f null -

@endlocal
@pause

I don't understand people encoding for YT with CRF and presets like placebo...

[lollo]

I don't understand people encoding for YT with CRF

I do not even use encoding, but lossless when I can!

This can be tested

Alto the use case in post #7

[pandy]

I don't understand people encoding for YT with CRF

I do not even use encoding, but lossless when I can!

of course, this is freedom of choice but i'm not sure how long you will be able to choose - soon they will introduce carbon passport in EU so at some point this can be very important factor in your choice possibilities...

This can be tested

Alto the use case in post #7

Yes but not with proper methodology (not CRF that focus on quality with low bitrate budget - my point in first time was do very fast encode with very high bitrate as very high bitrate will be anyway lower than lossless and eventual quality loss will be marginal when compared to expected quality loss in YT) - my testing show that you get approx 3..4 times smaller files with around 1.5..2 times longer encoding (but i use very old system with only 2 cores, 1 thread per core) at some point using very fast, multicore CPU with huffyuv you will saturate your I/O with R/W operations so limit will be in your I/O not in your CPU.
Also if you use highly asymmetric internet link (my case difference between U/D is like 1:9) then ability to provide 3..4 times lower size file to upload is very neat and desired.

[lollo]

Yes but not with proper methodology

I only care about final quality. No size, no encoding time, nothing else matter to me. Different point of views.

[Bwaak]

I only care about final quality. No size, no encoding time, nothing else matter to me.

And you call yourself an engineer? I would think that engineering is all about finding the optimal solution for multiple variables.

I guess your method is solving for the edge case when quality has weight 100%, while everything else is 0%

[lollo]

I would think that engineering is all about finding the optimal solution for multiple variables.

I have to do all that for my daily activity. For my collection (Analog capture of old TV shows), I just want the best, whatever it takes...

[dellsam34]

This has nothing to do with engineering, Hobby and art don't have rules, Engineering is ruled by saving money for the company or the customer. One can spend an entire day crafting a piece of art by hand that can by made by a machine in seconds, But the end result are not the same.

[lollo]

Thanks dellsam34, nice words, as always from you!

[pandy]

My point was only to provide some alternative to time consuming encoding vs high filesize of lossless that can be problematic for people with relatively low upload speed into YT. CRF encoding may in fact be suboptimal if another encoding step is performed by YT (due cumulation of encoding errors).
And eventual quality loss for case lossless vs INTRA can be unimportant (but to be honest didn't calculated PSNR for case INTRA vs lossless - but i expect that it will be marginal if any).

And as always - everyone choose what he like.

If someone is interested in files (so PSNR can be calulated) then this is script (modification of previous one). Seem huffyuv support only 422 so any 420 is converted to huffyuv (and this is not lossless operation as such added rem to disable huffyuv)

@setlocal

@REM Where to Find ffmpeg
@set FFMPG=C:\FF
@set PATH=%FFMPG%;%PATH%

@set SAVESTAMP=%DATE:/=-%%TIME::=-%
@set SAVESTAMP=%SAVESTAMP:.=-%


@SET file=%1
@SET fname=%~n1

@ECHO file=%file%
@ECHO filename=%fname%

@set qpval=4
@SET /a qpminval=((%qpval%)/2)+1
@echo QP value is: %qpval%
@echo QPmin value is: %qpminval%

@SET x264opts="qp=%qpval%:qpmin=%qpminval%:keyint=2:ref=1:bframes=0:b-adapt=0:no-weightb=1:weightp=0:no-mbtree=1:no-mixed-refs=1:me=dia:chroma-me=0:subme=0:open_gop=0:sliced_threads=1:slices=-1:rc-lookahead=0:force_cfr=1:cabac=0:no-8x8dct=0:constrained_intra=0:no-psy=1:scenecut=0:trellis=0:no-deblock=1:aq-mode=0:threads=auto:no_psnr=1:no_ssim=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:pic_struct=1:aud=1:stitchable=1:ipratio=1.0"

@rem ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v libx264 -profile:v high -x264opts %x264opts% -x264-params %x264opts% -f null -
@ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v libx264 -profile:v high -x264opts %x264opts% -x264-params %x264opts% -f matroska "%fname%_264intra_%SAVESTAMP:.=-%.mkv"

@rem ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v ffvhuff -f null -
@ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v ffvhuff -f matroska "%fname%_ffvhuff_%SAVESTAMP:.=-%.mkv"

@rem ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v huffyuv -f null -
@rem ffmpeg.exe -y -hide_banner -v 32 -benchmark -stats -an -dn -sn -i %file% -c:v huffyuv -f matroska "%fname%_huffyuv_%SAVESTAMP:.=-%.mkv"

@endlocal
@pause

script to perform some metrics (PSNR+SSIM) on files

@setlocal
@set FFMPG=C:\FF
@set PATH=%FFMPG%;%PATH%

@SET file1=%1
@SET fname1=%~n1

@SET file2=%2
@SET fname2=%~n2


@ECHO file=%file1%
@ECHO filename=%fname1%

@ECHO file=%file2%
@ECHO filename=%fname2%


@ffmpeg.exe -y -hide_banner -v 32 -an -dn -sn -i %file1% -i %file2% -lavfi "[0:v][1:v]psnr=stats_file="%fname1%_%fname2%_psnr.log";[0:v][1:v]ssim=stats_file="%fname1%_%fname2%_ssim.log"" -f null - >NUL  2> "%fname1%_%fname2%_psnr_ssim.txt"


@endlocal
@pause

[dellsam34]

I would like to learn your method pandy if you elaborate further, Will this be involved from the de-interlacing or just during the encoding and how do I integrate this into AvsPmod? Because I do it all in one step: de-interlacing, cropping, resizing and saving into HuffUYV and let it render overnight.

[pandy]

I would like to learn your method pandy if you elaborate further, Will this be involved from the de-interlacing or just during the encoding and how do I integrate this into AvsPmod? Because I do it all in one step: de-interlacing, cropping, resizing and saving into HuffUYV and let it render overnight.

there is no my method - it is simple observation - you have video workflow and it is not different than for your current method - difference is in rendering - you can use lossless video codec (not sure what is supported by YT) so your target upload will be very high quality and large filesize, you can go for time costly encoding with CRF and some medium to placebo preset so your file will have relatively low filesize and decent quality (but it can be suboptimal as YT reencode anyway your file so some encoding errors may accumulate - for example YT may insert I frame on your B frame and quality loss will be propagated trough rest of GOP) at a cost of computation time. "My method" is somewhere in between - render (encode) video with lossy codec and with disabled all features dedicated to preserve quality at low available bitrate budget - use some "intra" settings (so instead medium or even higher preset use fastest possible preset so you not wasting CPU time, set small GOP, perhaps even GOP=1 so every frame is independent, use very fast tune to speedup encoding further, go for very high bitrate - something corresponding/ being equivalent to CQP=4..8) - at some point you will get very large, very fast compressed lossy video but still quality will be higher than commonly used CRF with fancy presets, filesize will be lower (2..5 times) than losslessly compressed one - what you get is: reduced your computation time (many times), reduced upload time (sane trade off between encoding and uploading time), probably higher quality (YT encoding algorithm should be not affected by already lossy encoding).

It would be interesting to do comparison with all 3 approaches and final results (so also YT encoding involved in overall quality).

[theseeker2]

lollo, do you have an NTSC version of our AviSynth script for preparing your AVI files for upload to YouTube?

and dellsam34, care to share your script?

thank you!