Best Way to De-interlace HDV

[Sharc]

A visual zoomed in part of the frame would be ideal for me to see the actual differences. Maybe I'll do some testing on an SD capture sample from an analog tape and see for myself ....

Yes. Nothing substitutes personal experience and practical testing.

[rgr]

I still have a problem with AVSPmod, While LWLibavVideoSource option worked for importing mpeg-2, it does not carry over audio, just video, Back to the drawing board.

1. + LWLibavAUDIOSource.



2. Top - BSSource channels, bottom - LW.LibavudioSource.
They interpret sound differently, so there is an error in one. To be reported on the LWLibavAudioSource and BSSource github.
Probably BSSource interprets correctly.

The problem probably lies here:

[mp2 @ 00000265573b3d80] Header missing
[in#0/mpeg @ 000002655731cf00] Invalid timestamps stream=1, pts=14129097, dts=14129098, size=2015
Input #0, mpeg, from 'Tilden Park.mpg':
Duration: 00:02:02.91, start: 156.464011, bitrate: 25650 kb/s
Stream #0:0[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 384 kb/s, start 156.464011
Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, top first), 1440x1080 [SAR 4:3 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, start 156.489478
Side data:
CPB properties: bitrate max/min/avg: 25000000/0/0 buffer size: 1835008 vbv_delay: N/A

Edit: more errors

Stream mapping:
Stream #0:0 -> #0:0 (mp2 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[mp2 @ 000002bd997ba100] Header missing
[aist#0:0/mp2 @ 000002bd9b32ed40] [dec:mp2 @ 000002bd997b9b40] Error submitting packet to decoder: Invalid data found when processing input
Output #0, wav, to 'Tilden Park.wav':
Metadata:
ISFT : Lavf62.15.100
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
Metadata:
encoder : Lavc62.32.100 pcm_s16le
[mp2 @ 000002bd997ba100] Header missing
[aist#0:0/mp2 @ 000002bd9b32ed40] [dec:mp2 @ 000002bd997b9b40] Error submitting packet to decoder: Invalid data found when processing input
[mp2 @ 000002bd997ba100] Header missing
[aist#0:0/mp2 @ 000002bd9b32ed40] [dec:mp2 @ 000002bd997b9b40] Error submitting packet to decoder: Invalid data found when processing input
[out#0/wav @ 000002bd997a47c0] video:0KiB audio:23008KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.000331%
size= 23009KiB time=00:02:02.86 bitrate=1534.1kbits/s speed= 741x elapsed=0:00:00.16


Such errors most often result in desync a/v - AviSynth plugins cannot load something like this correctly.

[rgr]

Channels in order from top:
1. BSSource
2. LWLibav
3. ffmpeg >wav + BSSource
4. ffmpeg >wav + LWLibav

So I suspect that:
a) use BSSource to open mpg
b) use ffmpeg to convert to wav and load using BSSource or LWLibav
c) definitely not open mpg via LWLibav

But this would still need to be checked more thoroughly because I don't know what ffmpeg does with such errors.

[pchan]

Here is a sample of the HDV file:

I uploaded the mediainfo of your file to Google AI and requested for a ffmpeg preset.
Convert to H.264

C:\Users\pc_name\Desktop\Video_apps\ffmpeg-win64\bin\ffmpeg -i C:\Users\pc_name\Desktop\Test_video\Tilden_Park.mpg -vf "yadif=mode=0:parity=-1:deint=0,scale=1920:1080,setsar=1" -c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p -c:a aac -b:a 384k -movflags +faststart C:\Users\pc_name\Desktop\Test_video\Tilden_Park.mp4

[Attachment 93336 - Click to enlarge]

Convert to H.265

C:\Users\pc_name\Desktop\Video_apps\ffmpeg-win64\bin\ffmpeg -i C:\Users\pc_name\Desktop\Test_video\Tilden_Park.mpg -vf "yadif=mode=0:parity=-1:deint=0,scale=1920:1080,setsar=1" -c:v libx265 -preset slow -crf 21 -pix_fmt yuv420p -c:a aac -b:a 384k -movflags +faststart C:\Users\pc_name\Desktop\Test_video\Tilden_Park.mp4

This is the mp4 output. It encounters missing header and invalid timestamp during the conversion.

[lollo]

As requested, a quick processing in a rush.

Basic processing code, with AviSynth 2.6 and VirtualDub 1.10.4, all 32-bit.

I know dellsam34 does not like much denoising and sharpening, so just deinterlace with minimized QTGMC denoising, and upscale.
No levels correction either (btw, there are clipped whites in the source). No cropping. No DAR/PAR care.

The filters are loaded manually to show the used version.

# plugins directory
plugins_dir="C:\Users\giuse\Documents\VideoSoft\MPEG\AviSynth\extFilters\"

	# LSMASH
loadPlugin(plugins_dir + "L-SMASH-Works-r1194.0.0.0\x86\LSMASHSource.dll")

video_org=LWLibavVideoSource("Tilden Park.mpg")
audio_org=LWLibavAudioSource("Tilden Park.mpg")

video_org_dub=AudioDub(video_org,audio_org).trim(200,600)

	# QTGMC
Import(plugins_dir + "QTGMC.avsi")
	# Zs_RF_Shared
Import(plugins_dir + "Zs_RF_Shared.avsi")
	# MaskTools2
loadPlugin(plugins_dir + "masktools2-v2.2.23\x86\masktools2.dll")
	# RgTools
loadPlugin(plugins_dir + "RgTools-v1.0\x86\RgTools.dll")
	# MVTools
loadPlugin(plugins_dir + "mvtools-2.7.41-with-depans20200430\x86\mvtools2.dll")

	# Nnedi3
loadPlugin(plugins_dir + "NNEDI3_v0_9_4_55\x86\Release_W7\nnedi3.dll")

### de-interlacing
deinterlaced=video_org_dub.AssumeTFF().QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

### upscale
upscaled=deinterlaced.nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize", fwidth=1920, fheight=1080)

return(upscaled)

Alternative processing code could be the following. I did not run it so did not check potential errors.

# plugins directory
plugins_dir="C:\Users\giuse\Documents\VideoSoft\MPEG\AviSynth\extFilters\"

	# LSMASH
loadPlugin(plugins_dir + "L-SMASH-Works-r1194.0.0.0\x86\LSMASHSource.dll")

video_org=LWLibavVideoSource("Tilden Park.mpg")
audio_org=LWLibavAudioSource("Tilden Park.mpg")

video_org_dub=AudioDub(video_org,audio_org).trim(200,600)

	# QTGMC
Import(plugins_dir + "QTGMC.avsi")
	# Zs_RF_Shared
Import(plugins_dir + "Zs_RF_Shared.avsi")
	# MaskTools2
loadPlugin(plugins_dir + "masktools2-v2.2.23\x86\masktools2.dll")
	# RgTools
loadPlugin(plugins_dir + "RgTools-v1.0\x86\RgTools.dll")
	# MVTools
loadPlugin(plugins_dir + "mvtools-2.7.41-with-depans20200430\x86\mvtools2.dll")

	# Nnedi3
loadPlugin(plugins_dir + "NNEDI3_v0_9_4_55\x86\Release_W7\nnedi3.dll")

### converting to YUV 4:2:2
video_org_dub_yuv422=video_org_dub.ConvertToYUV422(chromaresample="lanczos").ConvertBits(10)

### de-interlacing
deinterlaced=video_org_dub_yuv422.AssumeTFF().QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

### upscale
upscaled_yuv422=deinterlaced.nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize", fwidth=1920, fheight=1080)

### converting to YUV 4:2:0
upscaled=upscaled_yuv422.ConvertToYUV420(chromaresample="lanczos").ConvertBits(8)

return(upscaled)

The outcomes of the basic script have been uploaded in my YouTube channel (https://www.youtube.com/watch?v=mypJfozY3Co), processed in who-knows-which-way by YouTube algorithms and can be downloaded back using "yt-dlp" or "JDownloader2" (be sure to download the best available format).

If anybody wants to upload on youtube the outcomes of the alternative scripts (rgr, are you available?) then we can download its YouTube version and compare versus the basic processing.

All necessary materials are provided, enabling anyone to replicate the experiment and draw their own conclusions.

edit: did not check for errors in the original mpeg stream.

[rgr]

@pchan: 25fps progressive? Its bad.

@lollo: Congratulations! You’ve created a video with progressive a/v desync, because a damaged audio track needs to be fixed with ffmpeg first. But at two minutes long, maybe no one will notice

The whites aren't clipped; you simply didn't pull them down into the proper range.

[lollo]

@lollo: Congratulations! You’ve created a video with progressive a/v desync, because a damaged audio track needs to be fixed with ffmpeg first. But at two minutes long, maybe no one will notice

The goal was something else. “When someone points at the moon, the fool looks at the finger.”

I was just answering to a request about comparison of different methodologies. Never mind, my mistake, I will put you back in the ignore status

[rgr]

But I admire you for entering a video with blown-out whites and desynchronized audio into the competition. I wouldn't have the guts

At least now you know that such video needs to be filtered with higher precision...

[lollo]

The whites aren't clipped; you simply didn't pull them down into the proper range.

Sure:

entering a video with blown-out whites

I specified earlier "No levels correction either (btw, there are clipped whites in the source). No cropping. No DAR/PAR care."

[rgr]

And it should be like this:

[lollo]

The accumulation at Y=255 means that there are details that may have been lost, and that you cannot recover with the shrink. The capture should be done with a better setting (but if the loss is in the tape no way to recover back).

The basics!

[rgr]

The accumulation at Y=255 means that there are details that may have been lost, and that you cannot recover with the shrink. The capture should be done with a better setting (but if the loss is in the tape no way to recover back).

The basics!

If your "basics" allow you to recover burned points from an HDV camera, then I congratulate you

[lollo]

Do not change the cards on the table. You said that there are not clipped whites, while there are, and tried to recover them with a level shrink, which is not possible, even if the level adjustement is required.

A summary of your performances in the last days:

• VirtualDub2 is better than AmarecTV for capturing (idiocy number 1)
• Capturing in 4:2:0 or 4:1:1 is equivalent to capturing in 4:2:2 (idiocy number 2)
• JVC S-VHS VCRs are crap (idiocy number 3)
• For YouTube upload you need an overkill processing (idiocy number 4)
• Losses in bad captures can be recovered with a level shrink (idiocy number 5)

Enough for me and for the rest of the readers to understand the kind of incompetent you are.

[dellsam34]

The accumulation at Y=255 means that there are details that may have been lost, and that you cannot recover with the shrink. The capture should be done with a better setting (but if the loss is in the tape no way to recover back).

The basics!

Correct, It is expected with consumer camcorders in auto mode, Nothing I would worry about, I look at the whole picture not histograms.

[lollo]

Correct, It is expected with consumer camcorders in auto mode, Nothing I would worry about, I look at the whole picture not histograms.

Yes dellsam34, nothing serious

[rgr]

Do not change the cards on the table. You said that there are not clipped whites, while there are, and tried to recover them with a level shrink, which is not possible, even if the level adjustement is required.

Clipping is in your weak and messy script with pointless upscaling 2x and downscaling.

Difference between bad (left) and good (right) script: https://www.diffchecker.com/image-compare/btI28N3O/
Blue sky, green trees... well, that's not your world!





[Attachment 93349 - Click to enlarge]

However, I would prefer this version:



You might also like this one:

[*] VirtualDub2 is better than AmarecTV for capturing (idiocy number 1)

Do you keep making captures with dropped frames?

[*] Capturing in 4:2:0 or 4:1:1 is equivalent to capturing in 4:2:2 (idiocy number 2)

I didn't claim that (at 4:2:0), so you're clearly having trouble reading

[*] JVC S-VHS VCRs are crap (idiocy number 3)

JVC (higher models) have terrible temporal image processing, but I understand that you still ruin the video with them.

[*] For YouTube upload you need an overkill processing (idiocy number 4)

Convince me that uploading your video will give better results than mine. But first you would have to convince yourself

[*] Losses in bad captures can be recovered with a level shrink (idiocy number 5)

??? Your reading problems have gotten worse, that's probably why you don't learn anything.

Enough for me and for the rest of the readers to understand the kind of incompetent you are.

Of course! What's your plan for uploading the image on the left to YouTube so it produces a better result than the one on the right?
https://www.diffchecker.com/image-compare/btI28N3O/

[lollo]

Idiot, the same processing as the basic one must be used and just introduce the (useless) conversions and bit depth increase without changing anything else and compare the download of the YouTube version, to (not) prove your idiocies.

It is not a question of better looking video but about the need of overkill processing.

[rgr]

Idiot, the same processing as the basic one must be used

I mean, should I make a script as bad as yours? :P
All in all, he was, he just wasn't that bad.

It is not a question of better looking video

Delete the forum!

but about the need of overkill processing.

You know mine was even faster?

[lollo]

What a charlatan!

[pchan]

Can consider using this app to compare videos.
https://github.com/pixop/video-compare/releases
Or download from VideoHelp.
https://www.videohelp.com/software?d=video-compare-20260708-win10-x86_64.zip

Video guide.
https://www.youtube.com/watch?v=UxOrGlvsFac

[Alwyn]

Can consider using this app to compare videos.
https://github.com/pixop/video-compare/releases

Nice!

[rgr]

Can consider using this app to compare videos.
https://github.com/pixop/video-compare/releases
Or download from VideoHelp.
https://www.videohelp.com/software?d=video-compare-20260708-win10-x86_64.zip

Video guide.
https://www.youtube.com/watch?v=UxOrGlvsFac

Cool tool, but if it doesn't support AviSynth scripts*, it's more convenient to use the Split function in the script.

* - not supporting

[hello_hello]

One way to compare filtering with Avisynth is to interleave the frames and then step through the video.

Mpeg2Source("D:\Tilden Park.d2v")
C1 = last
Fiter1()
Fiter1()
Fiter1()
C2 = last
Interleave(C1, C2)

Personally, I'd create an index file with DGDecode/DGIndex and use it to extract the audio, especially if the audio has issues that makes it hard or impossible to re-mux or re-encode without sync errors. The only way I managed to fix the glitches and hopefully keep the audio in sync was to import the extracted audio into Audacity and silence them manually before exporting it again. I'm not certain I got them all as I only checked it quickly.

QTGMC settings, denoising and level adjustments etc can naturally be changed according to personal taste.

Mpeg2Source("D:\Tilden Park.d2v")
QTGMC(EzDenoise=1.0, DenoiseMC=true, NoisePreset="slow", Preset="very slow")
Levels(0,0.95,255,0,235, coring=false)
Spline36Resize(1920,1080)
ConvertBits(16).GradFun3Plus().ConvertBits(10, Dither=0, Dither_Bits=8)
Prefetch(12)

[rgr]

Levels(0,0.95,255,0,235, coring=false)

This pulls a Y value of 16 below 16 - out of range.
But I'm more curious why gamma below 1 if you're "darkening" the image anyway.

As for the audio - you can check it out and compare. This should fix such audio glitches correctly.

ffmpeg -fflags +discardcorrupt+genpts -err_detect ignore_err -i "Tilden Park.mpg" -map 0:a:0 -af "aresample=async=1:min_comp=0.01" -c:a pcm_s16le "Tilden Park-corr.wav" (probably better)

or

ffmpeg -fflags +discardcorrupt+genpts -err_detect ignore_err -i "Tilden Park.mpg" -map 0:a:0 -af "aresample=async=1:min_hard_comp=0.01" -c:a pcm_s16le "Tilden Park-corr.wav"

[hello_hello]

Levels(0,0.95,255,0,235, coring=false)

This pulls a Y value of 16 below 16 - out of range.

I did check there weren't any blacks close to 16 before changing the levels, but if I was paranoid I'd probably run Levels twice, rather than enable coring, as coring will crush the out of range blacks and whites even if after adjusting the levels they're no longer out of range.

Like the whites here, if you look at the HistoGram:

Coring enabled while adjusting the levels.
Levels(0, 0.95, 255, 0, 235)


[Attachment 93369 - Click to enlarge]

Coring disabled plus a second Levels with coring enabled to do the limiting after adjusting the levels.
Levels(0, 0.95, 255, 0, 235, coring=false).Levels(0, 1.0, 255, 0, 255)


[Attachment 93370 - Click to enlarge]

Actually, I have a little function that limits the levels and automatically scales according to bit depth.

Levels(0, 0.95, 255, 0, 235, coring=false).LimitLevels()

function LimitLevels(clip Source) {

    Try{ Source_Bits = BitsPerComponent(Source) }Catch(err){ Source_Bits = 8 }
    Scale = (Source_Bits == 8) ? 1: (Source_Bits != 32) ? pow(2, Source_Bits - 8) : 1.0 / 256.0

    Source.Levels(0, 1.0, 255*Scale, 0, 255*Scale, Dither=true) }

But I'm more curious why gamma below 1 if you're "darkening" the image anyway.

I just fiddled until I thought it looked okay on my monitor, but it's not exactly calibrated.

As for the audio - you can check it out and compare. This should fix such audio glitches correctly.

ffmpeg -fflags +discardcorrupt+genpts -err_detect ignore_err -i "Tilden Park.mpg" -map 0:a:0 -af "aresample=async=1:min_comp=0.01" -c:a pcm_s16le "Tilden Park-corr.wav" (probably better)

Cheers. I'll try it out later.

[rgr]

Levels(0,0.95,255,0,235, coring=false)

This pulls a Y value of 16 below 16 - out of range.

Coring disabled plus a second Levels with coring enabled to do the limiting after adjusting the levels.
Levels(0, 0.9, 255, 0, 235, coring=false).Levels(0, 1.0, 255, 0, 255)

Why not Levels(16,1,255,16,235,coring=false) ?
( Personally, I prefer ApplyGradationCurves(lumapoint="16,16,255,235") )

Actually, I have a little function that limits the levels and automatically scales according to bit depth.
Levels(0, 0.9, 255, 0, 235, coring=false).LimitLevels()

This already exists in Avisynth - Limiter

[hello_hello]

Why not Levels(16,1,255,16,235,coring=false) ?

You could in this case, but I tend to check for out of range values manually after adjusting the levels anyway, and maybe I do it that way out of habit mostly, when applying a "random" adjustment.

Actually, I have a little function that limits the levels and automatically scales according to bit depth.
Levels(0, 0.9, 255, 0, 235, coring=false).LimitLevels()

This already exists in Avisynth - Limiter

I'd forgotten about Limiter, but does it support bit depths greater than 8?

[rgr]

Yes.