Current status of ld-decode / vhs-decode (true "backup" of RF signals)

[Titan_91]

You might be seeing some DC bias on the test point. I would recommend a ceramic 0.1uf DC blocking capacitor in series with your connection and try again with CXADC.

[uff]

The multimeter shouldn't care about DC Bias and show the actual AC though, right? Which is 0V on the test points I've tried sadly.

[Titan_91]

You can't measure the signals like that though, because multimeters are far from sensitive enough. Plus their impedence alone probably mostly decimates the incoming signal from the test point. That signal may be only 200mv or less peak to peak. If you're familiar with the leveladj tool, I would connect your input to your card, with a DC blocking capacitor in series, and run the tool. It will tell you the highest gain level you can use without clipping the input. If the source signal from your test point is too low, it will not show clipping at any of the gain levels.

[oln]

The signal won't be super strong with cxadc, even if you find a good testpoint. Last I used it the amplitude of the captured signal was quite small even at max gain, but it was enough to work for decoding.

If you have the SM that would help, I could only find one with extremely crappy scan quality for that VCR.

Some VCRs will already have a DC blocking capacitor on the test point, in which case adding another one may series won't help much, so it depends on the VCR whether it's useful or not.

[uff]

I PMed you the SM, scans should be in good quality. I will try the DC blocking cap when I get to testing this again. But about the multimeter: It should be able to measure in fractions of a millivolt, so there really shouldn't be a reason for it to show 0.00mV (with slight fluctuations) AC. Maybe the frequency is too high to the multimeter to handle it?

[oln]

Hm, seems they used video ics from panasonic. There should be output on TP1 (PV Enve) from what I can see There's no cap on the TP output from what I can see using one, and adjusting the cxadc gain may be helpful

Also, did you test with the HR-J668EU? They've changed things a bit from the 658 from the previous year I had issues with capturing from (maybe it would have worked with a DC blocking cap from that one, never tested). The TPs in both sit after the head amp when I look at the schematics, not sure why I thought it did not.

[uff]

Another thing I figured is there is an output from the IC itself, running to a transistor TR4. I don't really know what it's supposed to do, but according to the block diagram it outputs the head signal amplified? Sadly tapping into that transistor didn't really yield anything. But again, that could just be a matter of doing the DC blocking and connecting it to my cxadc instead.

I will put my VCR to the bench again in the next few days and report any results.

I never tried the JVC, since the SM shows the TP being unamplified (and it's actually sitting with my hifi which is a pain to remove since the power cable is builtin and I can't easily access the wiring behind it). Are you sure there is an amplified TP available? From what it seems the IC is doing the decoding internally so no real way to go without an amp. The output from the IC seems to be just the signal after the head switching logic without any amplification.

[oln]

Just a small update. Video track detection now seems to work for NTSC as well, though decoding NTSC color from the final chroma signal doesn't work properly yet (the current NTSC decoder seems to make some assumptions that don't hold for the vhs output as of now.) That's my next goal to improve.

We also added some decoding for U-Matic (regular NTSC), the filtering and deemphasis isn't quite there yet but it doesn't look too bad already. No color for the same reason as above. Otherwise U-Matic chroma upconversion from the tape signal is simpler than VHS though as it doesn't do any phase rotation like vhs or inversion:

[Attachment 57048 - Click to enlarge]

[VideoMem]

The multimeter shouldn't care about DC Bias and show the actual AC though, right? Which is 0V on the test points I've tried sadly.

Most of Digital Multimeters haven't the required bandwidth to measure over 1MHz. Also the input impedance of them is too low and them will attenuate the signal you want to measure, and even AC short it.

An analog one with fet input or a VTVM (vacuum tube volt meter) is required.
Or an oscilloscope, if you have one.

[uff]

Is my assumption to use the base of TR4 as a hook up point correct?


[Attachment 57058 - Click to enlarge]

[uff]

Okay, I just connected that to my cxadc and... nothing. Just minimal (same as no wire connected) noise at center level, even at the highest gain. It also doesn't change when doing anything on the VCR. Since there was no clipping, I don't think the issue was DC offset.

Then I tried the TP1 again, this time with the DC bias filter cap. It actually works! While I still don't get clipping at the highest gain, it's enough to produce a visible signal. Next thing to try now is getting vhs-decode up and running

[uff]

Probably a dumb question, but where do I get the latest version?

Also, trying out the script I could find on GitHub, it just gives me multiple

ERROR:root:Unable to find any sync pulses, jumping one second

and then hangs on saving the JSON.

[uff]

Here is a test capture I did of a retail MS Word tutorial tape at tenxfsc=1: https://anonfiles.com/v8g4L2Dfpe/test.bin.gz

[uff]

I renamed the .bin to .r8 and now it's doing something! I have also found out that the current branch is ntsc_color_broken.

[uff]

Is it possible to eliminate ghosting on the VHS? Or is that unavoidable? I also have a tape recorded in LP, which doesn't seem to decode. The tape plays fine on the VCR itself. Is LP not implemented yet?

Edit: Nevermind, the LP does decode actually. I'm trying another SP tape right now and it's doing the same. It does decode, but it seems slower (could be just me) and spams the console with

File Frame ...: CAV Frame #None Unknown
Chroma signal clipping.

guess I have to tweak the chroma gain?

Also, can someone help me getting this to run at 10bit samples? I can't get the decoding working with it.

[oln]

Depends what you mean by ghosting, the filters and de-emphasis are not quite correct yet, so there is some ringing and artifacts still. We've decoded PAL LP samples successfully at least, but there are still some bugs and the sync detection can sometimes fail a bit so maybe it just failed to detect it. LP isn't inherently very different to SP signal-wise other than lower bandwidth.

[uff]

So, I have done a quick comparison of a random snippet recorded from MTV between my TV/DVD Recorder and vhs-decode. The DVD recorder is used since it can convert SCART to HDMI. The colors on the TV look oversaturated, but on the other hand, they look washed out on the vhs-decode result. The TBC/timing artifacts are seemingly identical, I did expect vhs-decode to perform a bit better, but my DVD recorder might have a TBC integrated as well. Sadly I couldn't disable the deinterlacing of the DVD recorder for a fairer comparison.

The ghosting I talked about earlier was about the same on the TV, so it's probably just from the tape. It's that effect where patterns repeat when they switch intensity real quick.

The noise on the vhs-decode was extremely noticable (probably from a bad analog tv source), while on the TV it was there but very subtle. I have disabled all NR features on both the TV and DVD Recorder, even the picture optimization of the VCR. Is there still some NR maybe going on in one of my devices or is the cxadc simply not a good capture device? Would 10 bit recordings make much of a difference?

I will provide screenshots and sample recordings tomorrow.

[oln]

If you have checked out the latest ntsc_color_broken commit, there will be --10cxadc and --10cxadc3 options for 10-bit cxadc captures.

Yeah the vhs-decode output is a bit noisy at the moment. There is no noise reduction performed at all on it, Additionally the filtering still needs some tweaking, especially the de-emphasis. Emphasis/Deemphasis meaning that high frequencies are boosted a lot, a bit like massively turning up the treble on an audio signal, when recording and then reduced back again on playback. (This is a common thing for both audio (e.g analog radio) and video to retain the detail better).

Ah yeah, that sort of ghosting isn't uncommon on off-the-air recordings, it's not normally something caused by the VCR on it's own but rather then antenna setup not being optimal (though a bad VCR could probably make it worse).

Some DVD-Recorders can correct horizontal wiggle well, especially panasonic ones, while others don't. Nearly all VCRs (other than maybe very old models from before VHS HQ) run the luminance through analog Y noise reduction by default (noise reduction was a part of the VHS HQ specs). On some models (like the JVC you have) there is an EDIT mode or picture setting that will turn off most of it (more common on higher end VCRs designed for editing), but otherwise it's not something you can turn off. I can't find a user manual for the AKAI, but I presume the picture optimization is some auto-adjustment of sharpness/noise reduction/etc based on tape condition that many VCRs feature, I would't expect turning it off would turn off noise reduction. That said noise when recording the rf signal can also have some impact. Signal strength on the tape also has an impact on how noisy the image is before noise reduction whether in vhs-decode or a VCR.

[Titan_91]

Thanks for the development updates. I'll have to check out that sample tomorrow and do an SNR analysis when I attach my RF amp. This is an OPA657 amp, similar to the one used on the Domesday Duplicator. My LaserDisc project was a bust for now, but I might be able to work out getting a more compatible player. There's definitely something wrong with the RF frontend on my current machine, even though it plays fine.

For now, I'm going to move my RF tap using RG-174 wifi antenna cable to my VCR. I'm going to compare results with and without the VGA amp. I have a feeling I can get some solid results this time.

[uff]

If you have checked out the latest ntsc_color_broken commit, there will be --10cxadc and --10cxadc3 options for 10-bit cxadc captures.

I have actually tried that, with no success.

Some DVD-Recorders can correct horizontal wiggle well, especially panasonic ones, while others don't. Nearly all VCRs (other than maybe very old models from before VHS HQ) run the luminance through analog Y noise reduction by default (noise reduction was a part of the VHS HQ specs). On some models (like the JVC you have) there is an EDIT mode or picture setting that will turn off most of it (more common on higher end VCRs designed for editing), but otherwise it's not something you can turn off. I can't find a user manual for the AKAI, but I presume the picture optimization is some auto-adjustment of sharpness/noise reduction/etc based on tape condition that many VCRs feature, I would't expect turning it off would turn off noise reduction. That said noise when recording the rf signal can also have some impact. Signal strength on the tape also has an impact on how noisy the image is before noise reduction whether in vhs-decode or a VCR.

Yeah, this is a Panasonic one. I am actually using the JVC deck for the test and did set it to EDIT mode.

[uff]

I have created an imgur album with some side by side comparison frames: http://imgur.com/a/x7q2qP7

Left side is from the JVC HR-J668 (with B.E.S.T. disabled and picture mode set to EDIT), connected to the Panasonic DMR-EX84C, which then goes into an HDMI splitter and finally into my Game Capture HD60S. There are some artifacts which seem to be caused by the way the capture card upscales and deinterlaces video.

On the right side, there is the material from the AKAI fed into vhs-decode. I enabled deinterlacing and forced the aspect ratio to 4:3 in VLC.

In my opinion, the classic HDMI capture of the JVC looks much better, but that might just be due to some post processing, who knows.
Especially the TBC/Stability and chroma accuracy look a bit better on the left side, which is weird. Is this all something that can be fixed in software or is the cxadc capture method simply not good enough compared to a good regular capture setup?

[Titan_91]

My understanding is the signal you're getting is probably fine, but as was mentioned the filtering and de-emphasis need a little work in vhs-decode. A few months ago I tweaked the main luma filter to increase the amount of bandwidth captured. This improved the sharpness of the image quite noticeably, but at the expense of luma clipping which created white streaks. So there's still quite a bit that can be done on the software side.

That being said, to maximize the signal to noise ratio (SNR) an amplifier of some sort is recommended. From my experience, bringing the gain level of CXADC up to 15 or higher increases the noise floor which will reduce the clarity of the image. The idea of using a separate amplifier would be to amplify as close as possible to the VCR, then send the amplified 3.9MHz center frequency signal to your PC's card input and use level 1 for gain to minimize the amount of noise generated by the inexpensive card. This is what I'm testing today.

[uff]

I also have a LimeSDR Mini, which is an SDR with 12Bit Samples and 30.72MHz Bandwidth. Sadly there is no way to bypass the tuner and the minimum frequency is 10MHz. But, if I can use an upconverter (something that shifts the frequencys up), it might be possible to use it as a high quality solution, especially since it has some nice low noise amplifier and stuff.

[oln]

My understanding is the signal you're getting is probably fine, but as was mentioned the filtering and de-emphasis need a little work in vhs-decode.

Yeah you can see especially on sharp dark->bright transisions that there is some overshoot-like artifact. On the screens you're showing, the colors look less smeary and more correctly positioned on the vhs-decode ones though. It might be in part due to the vhs-decode one being less saturated (though it's possible to adjust the chroma gain if needed), though at least on the standard capture ones the chroma is visibly offset a bit to right compared to the luma. The contrast is also stretched a bit more on the standard capture.

[uff]

What's interesting is all of these chroma clipping warnings happen and in the end it's actually less saturated than usual.

[Titan_91]

I and others have seen constant issues where cxadc doesn't want to work when loading the kernel module. The errors in the terminal state the device doesn't exist, yet the /dev/cxadc0 special device file is there. I think I found out why this is constantly happening with Ubuntu and Mint. It seems to be driver signature enforcement in the kernel. When building, I get this:

make && sudo make modules_install
make -C /lib/modules/`uname -r`/build M=$PWD
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-62-generic'
  AR      /home/user/Downloads/cxadc-linux3-master/built-in.a
  CC [M]  /home/user/Downloads/cxadc-linux3-master/cxadc.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M]  /home/user/Downloads/cxadc-linux3-master/cxadc.mod.o
  LD [M]  /home/user/Downloads/cxadc-linux3-master/cxadc.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-62-generic'
[sudo] password for user:           
make -C /lib/modules/`uname -r`/build M=$PWD modules_install
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-62-generic'
  INSTALL /home/user/Downloads/cxadc-linux3-master/cxadc.ko
At main.c:160:
- SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
sign-file: certs/signing_key.pem: No such file or directory
  DEPMOD  5.4.0-62-generic
Warning: modules_install: missing 'System.map' file. Skipping depmod.
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-62-generic'

And when checking the dmesg logs, I get this:

13:48:10 kernel: cxadc driver loaded
13:48:10 kernel: cxadc: module verification failed: signature and/or required key missing - tainting kernel
13:48:10 kernel: cxadc: module verification failed: signature and/or required key missing - tainting kernel
13:48:10 kernel: cxadc: loading out-of-tree module taints kernel.
13:48:10 sudo:   user : TTY=pts/0 ; PWD=/home/user/Downloads/cxadc-linux3-master ; USER=root ; COMMAND=/usr/sbin/insmod cxadc.ko vmux=2 level=1

What's weird is I never had this issue for months at a time, then started having the problem. Maybe there was an update that changed the driver signature enforcement or something. It's not the first problem where the module wouldn't insert due to kernel version being incompatible. I get no errors in the terminal when inserting it but when I run rmmod, I get a message saying the module isn't even loaded. The device file creates fine, but the module doesn't load for that reason. I removed that folder and re-extracted/re-built the module. Tried inserting again, it says the file exists. Yet, I can't access cxadc0 and using rmmod says it's still not loaded.

I read somewhere there are two solutions, one is signing the kernel module so the kernel will load it. Another is re-building the kernel with an enforcement flag disabled (lol). I still don't understand why so many people seem to not have this issue, but one of the Discord guys and I do. I also read secure boot in the UEFI settings could cause this, and the CMOS battery did die and clear my settings about a month ago. However, I checked today and secure boot is turned off.

[uff]

Maybe try installing the bootloader on the MBR and then legacy booting it, in case nothing else helps.

[Titan_91]

Thanks, but I was told those are simply warning messages and the module isn't being blocked by the kernel. The issue was a conflict with existing kernel modules loaded on boot. My setup is working again. I opened the below issue on GitHub for happycube to take a look at.

https://github.com/happycube/cxadc-linux3/issues/8

Did a first test with this amplifier, results aren't bad to start off with. SNR is 32 and the image is decent:



Adjusted the lower cutoff for the low pass filter as mentioned earlier in this thread and tested again:

I adjusted the low pass NTSC luma filter. This, surprisingly, made quite an improvement! Not only that, but chroma isn't fully broken in this branch. I was still able to decode several fields with correct color. I really don't know what I'm doing though. I basically just changed this in vhsdecode/formats.py:

# Band-pass filter for Video rf.
# TODO: Needs tweaking
RFParams_NTSC_VHS['video_bpf_low'] = 3200000

To this:

# Band-pass filter for Video rf.
# TODO: Needs tweaking
RFParams_NTSC_VHS['video_bpf_low'] = 2400000

My amplifier is acting weird though. I'm not getting as much gain as expected compared to my LaserDisc player, and often the signal will disappear and come back as I adjust the gain. Will have to fiddle with it quite a bit more.

[oln]

Another PAL comparison after some filter tweaking, trying to tweak the bandwidth while avoiding streaks. First is JVC HR-S8500 or HR-S8600 (don't remember which) with TBC on and I think EDIT picture setting (stretched a little to have similar width). It was an old capture from it otherwise I may have used a different VCR and setting to compare. Second is vhs-decode from a Sony SLV-SE60 captured with a domsday duplicator, with contrast bumped a little to be similar to the other one (forgot to adjust chroma gain before decoding). Still some slight ringing/ghosting right of sharp transitions that needs to be figured out. On the flip side with the JVC image there is some dark overshoot after bright areas instead.

[Attachment 57071 - Click to enlarge]

[Attachment 57073 - Click to enlarge]

Next goal is trying to make NTSC color decode the correct colors.

[Titan_91]

Nice, and thanks for working on NTSC color. I'm going to try adjusting the upper luma cutoff tomorrow to try and get rid of those squiggles in my captures. I think I can do that without sacrificing the 3MHz horizontal resolution as the format is limited in that aspect anyway. If I had that in a LaserDisc capture though I wouldn't have that luxury with LD being about 4.5MHz of bandwidth.