Current status of ld-decode / vhs-decode (true "backup" of RF signals)

[Titan_91]

Is there some sort of straight forward tutorial for this? I'd love to try it, but I have no idea where to start. Also, would the RF hookup allow me to get the tape path alignment perfect? I messed with it a few years ago trying to fix a tracking issue, and have never been able to get it to 100% perfect.

A while ago someone posted the series of terminal commands needed from start to finish, but I can't find that post now. The best thing I can recommend is start about 10 pages back and read each post, it's a slow and tedious read but it explains the full history of what has already been tried and what works and doesn't. Here's a post that may help:

https://forum.videohelp.com/threads/394168-Current-status-of-ld-decode-vhs-decode-(tru...16#post2612088

Zcooger, I found this video by random chance and noticed your comments about vhs-decode. Considering the apparent "anything on tape" scope of this project is anyone interested in implementing PXL-2000 support? It's a "toy" camera that records in a LBTV format onto audio cassettes at about 16.875 inches per second, with a theoretical video bandwidth of about 200KHz (9x the playback rate and bandwidth of 12KHz). His video demonstrates a played back section of a single frame (slowed down to what I assume is 44100Hz) and he calculated and showed various signal timing for sync, etc. That would be a just for kicks thing, as it has literally no application and source material available in 2021.

https://youtube.com/watch?v=G4hM5X7m5_k

[peppi001]

I found another bug or error.

The vhs-decode stuck at frame 71 and the log file repeat the same message until I pressed CTRL+C "lddecode - DEBUG - VBI serration levels 3 - Sync tip: 3703.07 kHz, Blanking (ire0): 4000.73 kHz"

The command: "vhs-decode -p -t4 --10cxadc3 -cafc error.vhs error_out"
The test file: https://files.videohelp.com/u/264498/error.vhs

[peppi001]

Update: The latest version from oyvindln git is working.

[Titan_91]

Is anyone aware of a software based digital audio to video conversion tool? Like something that would take a WAV or FLAC file and convert it to a video signal with digital "pixels" that would allow me to store digital audio on VHS for archival purposes.

https://m.youtube.com/watch?v=WVDCxTtn4OQ

[peppi001]

Is anyone aware of a software based digital audio to video conversion tool? Like something that would take a WAV or FLAC file and convert it to a video signal with digital "pixels" that would allow me to store digital audio on VHS for archival purposes.

https://m.youtube.com/watch?v=WVDCxTtn4OQ

Maybe this is what you looking for:
https://forum.videohelp.com/threads/396928-Recovering-VHS-Beta-PCM-Video-DATA

[Zcooger]

Here's package that allows to convert between audio and video in PCM-F1/STC-007 format:
https://odysee.com/PCM-Video-Conversion-Kit:b
More information you can find on "Fagear Tech Corner" Discord server, #pcm_diy channel (mainly russian but they talk in english too).

[Titan_91]

Maybe this is what you looking for:
https://forum.videohelp.com/threads/396928-Recovering-VHS-Beta-PCM-Video-DATA

Yes, fantastic! What an awesome tool. I'll have to play with PCM Coder. Thanks for the link.

[Titan_91]

Following this video guide, I finally got QTGMC working in Wine:

https://www.youtube.com/watch?v=C4PyyQoz6eo

Now that I've tried it, the de-interlacing is fantastic but I don't like the noise reduction. It completely decimates any detail gains achieved using vhs-decode over a normal VCR/capture device workflow. Is there a preset that does not use noise reduction? If not, do I need to try SourceMatch to disable noise reduction and sharpness filters? Literally the only thing I want to do with QTGMC is de-interlacing.

[Skiller]

I'm surprised that decoding of these files is working so well, though. Unless Audacity is interpreting the file incorrectly, your waveform is clipping often. This may be the cause of the black streaks visible in the screenshots above.

If you are talking about these black streaks:


[Attachment 61538 - Click to enlarge]


They are called "fish" or "comets" and are an artifact of analog satellite TV reception. They tend to occur in areas of high color saturation and on sharp transitions. Not so much in flat, pale areas. Especially when the weather was bad, this was a common sight in the 90's and early 2000's in Europe.

Since they are "baked in" into the VHS recording, they cannot be fixed by ld-decode. However I used to have reasonably good success masking them with an old AviSynth plugin called "DePulse".

[lollo]

... analog satellite TV reception. They tend to occur in areas of high color saturation and on sharp transitions. Not so much in flat, pale areas. Especially when the weather was bad, this was a common sight in the 90's and early 2000's in Europe.

I remember the old time of Analog Satellite reception. Even on "strong" transponders on Astra and Hotbirds, without a laaaarge dish and ideal wheater the white/blacks spots were there on some channel. Lot of time trying to move the satellite dish just few millimiters for best reception. A nightmare

[Titan_91]

Here's the AviSynth+ script I settled on to avoid noise reduction:

FFMPEGSource2("vhs-decode.mkv")
AssumeTFF()
QTGMC(SourceMatch=1, Lossless=2, EdiThreads=4)
Prefetch(14)

I took this output from VirtualDub2, bumped brightness and contrast by 2 ticks (mouse wheel scrolls) and saturation by 3 ticks using Avidemux, and have attached the result. This, to me, is the most balanced end product from this particular tape where exposure is constantly changing from the camcorder. Indoor shots where exposure doesn't swing so much will allow me to get more contrast. My stock Ryzen 7 3700x can chew through the QTGMC and FFV1 compression workload at around 25FPS, at 100% overall utilization, not bad at all for a CPU intensive process.

[Titan_91]

I did a successful decode of another tape where the signal ended and some white noise was captured at the end of the sample. Here is the terminal output. The white noise is present in the decode at the end of the tape, which is normal. The entire tape decoded fine with no issues or artifacts. Please note I adjusted formats.py to use a 1.5MHz lower cutoff frequency for the NTSC band pass filter.

/usr/lib/python3/dist-packages/numpy/core/fromnumeric.py:3256: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/usr/lib/python3/dist-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)

no valid lines found! Guessing or using values for previous field so result will probably be garbled!

ERROR - please paste the following into a bug report:
current sample: 57572109582.63985
arguments: Namespace(AGC=False, cafc=False, chroma_trap=False, cxadc=False, cxadc3=False, cxadc3_tenbit=False, cxadc_tenbit=False, debug=False, disable_comb=False, disable_dc_offset=False, disable_diff_demod=False, dod_hysteresis=1.25, dod_threshold_a=None, dod_threshold_p=None, dodod=False, high_boost=None, infile='newhope-windowdedication95.flac', inputfreq=None, length=99999999, level_adjust=0.1, nldeemp=False, noAGC=None, nodod=False, notch=None, notch_q=10.0, ntsc=True, ntscj=False, outfile='newhope-windowdedication95', pal=False, palm=False, recheck_phase=False, sharpness=0, start=0, start_fileloc=-1, sync_clip=False, tape_format='VHS', threads=15, track_phase=None, umatic=False)
Exception: cannot convert float NaN to integer  Traceback:
  File "./vhs-decode", line 258, in <module>
    f = vhsd.readfield()
  File "/media/user/vhs-decode/vhs-decode new/vhsdecode/process.py", line 750, in readfield
    f, offset = self.decodefield(initphase=initphase)
  File "/media/user/vhs-decode/vhs-decode new/lddecode/core.py", line 3466, in decodefield
    raise e
  File "/media/user/vhs-decode/vhs-decode new/lddecode/core.py", line 3462, in decodefield
    f.process()
  File "/media/user/vhs-decode/vhs-decode new/lddecode/core.py", line 3177, in process
    super(FieldNTSC, self).process()
  File "/media/user/vhs-decode/vhs-decode new/lddecode/core.py", line 1472, in process
    self.linelocs1, self.linebad, self.nextfieldoffset = self.compute_linelocs()
  File "/media/user/vhs-decode/vhs-decode new/vhsdecode/process.py", line 307, in compute_linelocs
    int(line0loc) : int(line0loc + (self.meanlinelen) * 4)

[oln]

Will have a look at it, suspect it's just some function not accounting for no line starts being found in the noise part or something.

They are called "fish" or "comets" and are an artifact of analog satellite TV reception. They tend to occur in areas of high color saturation and on sharp transitions. Not so much in flat, pale areas. Especially when the weather was bad, this was a common sight in the 90's and early 2000's in Europe.

Yeah it's something can happen when the signal is demodulated or modulated from/to FM if there is much noise, signal is weak etc. Similar effects can be caused from tape playback/recording too as the luminance is FM encoded on tape as well, though in this case you may be right right that it could be from the reception rather than a playback issue.

[Titan_91]

Here's that last sample I did for a church. Turned out great only with a few frames of a darker image shown only at certain times (at 17:50 and 18:36 for example). The source is SP and recorded in 1995. Signal level on the tape is still good, dropout correction wasn't even needed. I no longer have the tape but kept the RF capture. Audio was captured traditionally using the VCR's standard RCA audio jacks.

https://youtube.com/watch?v=3UtiIKo47-Y

[Titan_91]

I uploaded this to Archive.org for preservation's sake:

https://archive.org/details/discoverychannel_korubuindians_202111

[Titan_91]

There's a bit of an issue when trying to mux audio on a bad tape that causes dropouts in the decoding. I have a degrading tape recorded at EP speed with what appears to be a few seconds of video loss. I think this tape was dubbed with multiple camera sources, which could explain it. Since vhs-decode skips these frames, it causes an audio/video desync in my muxes at the point the frames were skipped. As fields are dropped here and there it causes gradual desync I can't correct for. The logs shows this at certain times:

Unable to determine start of field - dropping field

In a previous version it would actually decode the end of the tape as static in the image like a VCR would, with log entries saying something like "using TBC data from previous field." Can this be re-enabled where vhs-decode displays static and does not drop/skip fields? The video just needs to be the same length as the audio I recorded separately.

[oln]

It's not an intentional change, would want it to still output something regardless. I suspect my last commit that fixed another issue causing an infinite loop may have broken it but not sure, does reverting the change on line 181 here (or going back to the previous commit if you feel comfortable with git) change the behavior back?

Your last sample looks nice, I'm noticing a slight bit of this bug on it, need to go over the line start detection code to see if it can be improved (I have spotted a similar effect a fair bit on e.g conventional captures done with no line-tbc, especially on dubbed ones.). I think this happens due to the "ghosting"/overshoot/undershoot from the bright spots at the right edge being long enough to affect the levels on the downward part of the horizontal synchronization pulse.

[Titan_91]

In process.py, would I just need to delete...

if self.prevfield is not None:
                ldd.logger.info("lastline < proclines , skipping a tiny bit")
            return None, None, max(line0loc - (meanlinelen * 20), self.inlinelen)

and add...

return None, None, line0loc - (meanlinelen * 20)

...back as it was before then try decoding again? Also I'm glad you mentioned that bug. I believe this same poor quality tape from 1993 is triggering it. There are times where SNR takes a dip and causes diagonal shearing of the image:



I do believe the root cause is over/under shoot during dark to bright transitions as I have black fishes in this scene:



Also, wasn't sure if any automatic gain control code was causing the image to go dark during corrupted fields (is AGC still turned off by default)?



As with all my decodes I'm using a lower cutoff of 1.5MHz for the NTSC band pass filter to get a bit more bandwidth. Should I revert this adjustment back to the default setting for this bad tape? I wonder if it's contributing to the black fishes and ghosting. I would assume not, as the bright transitions are on the upper side of the signal, not the lower side. If I was going too low wouldn't I see a checkerboard pattern from the chroma signal in the luma channel?

[oln]

In process.py, would I just need to delete...
...
and add...
...

...back as it was before then try decoding again?

Yup

As with all my decodes I'm using a lower cutoff of 1.5MHz for the NTSC band pass filter to get a bit more bandwidth. Should I revert this adjustment back to the default setting for this bad tape? I wonder if it's contributing to the black fishes and ghosting. I would assume not, as the bright transitions are on the upper side of the signal, not the lower side. If I was going too low wouldn't I see a checkerboard pattern from the chroma signal in the luma channel?

It could be contributing to black/white fishes/streaks, though it's possible it's baked in from happening during playback or recording too since it looks like it's been dubbed at least once judging by the head switch and excessive sharpening.

May have noted this earlier but the fishes/streaks that can occur during decoding (whether in vhs-decode or in a VCR) happen when the demodulator "misdetects" or "misses" the frequency at that point in the signal due to noise, weak signal amplitude or other things, and the filtering of the rf signal prior to that will have some impact on it.

This from illustration from a JVC techical guide may help illustrate it a bit, though the demodulation in vhs-decode works a bit different at the moment:

[Attachment 62204 - Click to enlarge]

That's why the default cutoff is a bit high at the moment as a trade-off (even though it being lower would work better on most tapes) between causing streaks and detail, though ideally we would find some way of improving the demodulation a bit and/or design more optimal filters.

[Titan_91]

The scene with the black fishes is about halfway into the capture so I can't trim it out, but I adjusted the lower cutoff back down to the default setting and the ringing/ghosting at the start of the tape was reduced a bit. However, I do believe that artifact is baked into the signal on the tape in this case. The ringing can still be seen without my adjustment, and using the adjustment significantly increases the amount of video bandwidth and sharpness in the image. So I will stick with the 1.5MHz lower cutoff also since it works so well with most of my tapes.

I will re-run this decode now that I have turned off field dropping. I'm not too concerned about the TBC problems as the image breaks up only rarely due to heavy dropouts on this poorly mastered tape. Still a much better result than using a typical capture device.

[peppi001]

Is it possible to run gen_chroma_vid_pal on windows?

[Titan_91]

No. As of right now, vhs-decode only works on Linux distributions. I recall hearing something about the Domesday Duplicator software having an experimental Windows branch, but that's of course only if you're using the DdD hardware, and even then you'd still need to decode it.

ㅤㅤ

[oln]

The other parts run/can be built, though those are shell script written in bash so not sure. It's possible they would run with a bas shell on windows like msysgit or similar. It should be doable to port them to python or something that's more portable though, those script just calls ld-chroma-decoder and ld-dropout-correct with some parameters.

[peppi001]

oln Can you port this script to windows batch?

[oln]

Unless someone else gets to it first, I might try to port it to python instead, so we don't have to maintain a variant for each system.

[peppi001]

How about the speed of vhs-decode?
I decoded a 30 minutes video and took more than 11 hours. Is it normal? (CPU: i9-10850K)

Start log:
2021-12-11 17:46:14,842 - lddecode - DEBUG - VBI EQ serration pulses search failed (using fallback logic)
End log:
2021-12-12 04:57:41,885 - lddecode - DEBUG - File Frame 45000: CAV Frame #None Unknown

[Titan_91]

I ran the decode again. The result is still about 20 seconds shorter than the audio. However, vhs-decode crashed after the last valid field at the end of the tape so it might be that. Will need to actually check the A/V sync and I'll just try and correct for it manually if it's still a problem.

[Titan_91]

As far as a possible Python port that could run on Windows, it could be the first real step to mainstream accessibility for the project. Someone could also write an interface or wizard script to allow novice users to run this who are trying to get familiar with vhs-decode. Then maybe from that a simple cross platform GUI much later down the line.

I would be happy to test a Python port on Windows but don't have a Windows 10 machine or VM, only Windows 7. There is a Python fork for Windows 7 compatibility.

[Titan_91]

Well I ran the decode again and the video is in sync with the audio at the beginning, but still drifts to where the video is ahead of the audio about 2 seconds at the end. I have attached my log file for this run on December 12 (after I made the recommend change to process.py) as well as the original run on December 4. To me it still looks like we are dropping fields here and there. If there's no fix it's not a huge deal, it's only a small project for my church. Let me know if you want part of the capture as well.

[peppi001]

Update: Updated the repo, and this error is gone

I want to try cvbs-decode, but not works for me.

Traceback (most recent call last):
File "/usr/local/bin/cvbs-decode", line 65, in <module>
vhsd = CVBSDecode(
File "/usr/local/lib/python3.9/dist-packages/cvbsdecode/process.py", line 502, in __init__
self.rf = VHSDecodeInner(
File "/usr/local/lib/python3.9/dist-packages/cvbsdecode/process.py", line 604, in __init__
SF["FVideo05"] = SF["Fvideo_lpf"] * SF["F05"]
KeyError: 'F05'