Deinterlacing Deep Space Nine NTSC

[litmus]

Can anyone help me figure out how to deinterlace Deep Space Nine properly in Handbrake while keeping the frame rate at 29.97?

If I use Detelecine I get a near perfect result, however scenes with high motion (the space battles mostly) look bad at 23.976 fps.

I've tried I think every possible combination of Decomb, Yadif, with various interlace detections but the result always has horrible artificing or shimmer if I don't use Detelecine (which I can't do at 29.97 unless I want duplicate frames, which I don't). At least I tried every combo supported in the GUI

I've attached a VOB of the intro. Any time I use decomb or deinterlace without detelecine I get massive shimmer as soon as the close up of the space station starts

I've tried using Staxrip instead of Handbrake and while it deinterlaces perfectly using QTGMC, I cannot get the same quality of output that I get out of Handbrake. Like a Staxrip encode of H264 at CQ 10 looks visually worse than one I did in Handbrake at CQ 16. Probably user error on my part but I've spent hours trying to figure it out and would rather just get Handbrake working if possible, as it's much faster

[jagabo]

Beware the intro sequence like that are often different than the body of the show. But that sequence was shot on film, telecined with 3:2 pulldown, then different shots were crossfaded and titles overlaid. You'll probably want to inverse telecine, not deinterlace. Though sometimes special effects are overlaid as 30p or 30i. Such sequences will be a little jerky at 23.976 fps.

Mpeg2Source("ds9 test intro.d2v") 
TFM()
TDecimate()
vInverse()

[litmus]

Beware the intro sequence like that are often different than the body of the show. But that sequence was shot on film, telecined with 3:2 pulldown, then different shots were crossfaded and titles overlaid. You'll probably want to inverse telecine, not deinterlace. Though sometimes special effects are overlaid as 30p or 30i. Such sequences will be a little jerky at 23.976 fps.

Mpeg2Source("ds9 test intro.d2v") 
TFM()
TDecimate()
vInverse()

Perhaps I should attach something showing the issue. As you said, inverse telecine (or Detelecine as it's called in the Handbrake GUI) works quite well most the bulk of the show. However, some FX shots end up looking quite bad in 23.976 fps, probably because they were done on videotape? Just a guess. Anyway, I've attached the sequence where I first noticed the issue, both the VOB and a file that has been run through an inverse telecine. Note the massive judder, especially during the last tracking shot as the Defiant passes over the other spaceship. This is what I'm trying to fix

The series is full of these kinds of shots especially in the last couple of seasons

[poisondeathray]

The only way is VFR for complete sections, 23.976p sections play at 23.976, 29.97p sections play at 29.97p

If there are mixed composited sections (e.g. VFX on top of film, in the same frame), you need to decide what to do differently, take the base rate or VFX rate . Or rotoscope (moving masks) to isolate parts of frames and interpolate (perhaps using optical flow) to match the framerate of the other (VFX at 23.976 or film at 29.97)

[manono]

Handbrake is awful for this kind of work (and everything else, as far as I'm concerned). If it has to be 23.976fps (and not VFR'd or bobbed to 59.94fps), I'd decimate the fields and not the frames. Like so:

QTGMC( )
TDecimate(Mode=2,Rate=23.976)
Spline36Resize(640,480)

Still jerky during the video parts, just not as jerky.

[jagabo]

Yes, the special effects are at 30p or 30i, not 24p like the underlying film. If you really want to use Handbrake set the deinterlace option to Yadif and the Preset to Bob. Then set the frame rate to 59.94. Peak. The frame rate will vary from 24p, to 30p, to 60p depending on which is appropriate. Or set the frame rate to 59.94 constant and the whole video will be at 60p, like when you used QTGMC.

[litmus]

Thanks, I should do a test at VFR. One piece of info I held back from you guys is that I am upscaling these in Topaz Video Enhance AI, which as I understand it works better with CFR, though I have not actually tested VFR. This is also why I want to limit it to 29.97 fps, a 23.976 fps episode encode takes about 4 and a half hours to process in the tool, a 29.97 fps encode takes about 5 and a half hours, and a 59.94 encode would most likely take twice as long as the 23.976. When you have 170+ episodes to convert, this adds up. Here's a clip of what one of these upscales looks like if you are curious: https://youtu.be/oI1_T4xPmOU. Closest we'll probably ever get to having the show in HD.

[poisondeathray]

Thanks, I should do a test at VFR. One piece of info I held back from you guys is that I am upscaling these in Topaz Video Enhance AI, which as I understand it works better with CFR, though I have not actually tested VFR. This is also why I want to limit it to 29.97 fps, a 23.976 fps episode encode takes about 4 and a half hours to process in the tool, a 29.97 fps encode takes about 5 and a half hours, and a 59.94 encode would most likely take twice as long as the 23.976. When you have 170+ episodes to convert, this adds up. Here's a clip of what one of these upscales looks like if you are curious: https://youtu.be/oI1_T4xPmOU. Closest we'll probably ever get to having the show in HD.

I saw the thread at doom9, I assumed it was the same people involved

You gave no indication if there were any interlaced content sections. But those would become 59.94p

The VFR I'm talking about is timecode VFR . There are no duplicates, and you have the perfect number of frames for Enhance.

If you did it all 23.976 you'd be missing frames in some sections such as that space battle and it would play jerky in some sections because of the drops. If you did it all 29.97, you'd have duplicate frames in some sections and they would stutter in some sections. If you did it all 59.94 (assuming there were interlaced content sections), you'd have many duplicate sections, and it would play with 3:2 judder in most sections

Enhance does not perserve timecodes, because you are exporting an image sequence like PNG. (It has a mpeg4 option but it's lossy, you probably don't want to use it) . After you encode your final format, you mux in the timecodes to keep everything in sync

[litmus]

I think I may be on to something, I got the following result in Staxrip by deinterlacing with QTGMC and then outputting only the even frames with Filter-> Select Even. Seems pretty smooth to me, certainly an improvement over what I posted before. I'll have to do more testing to see if this is acceptable for processing entire episodes.

[jagabo]

As expected, in the film section every 5th frame is a duplicate.

[lordsmurf]

QTGMC fastest, then done.
The entire mixed source aspect makes it insane to try to IVTC.

[manono]

QTGMC fastest, then done.

It makes the most sense, yes. But he's worried about the length of time it'll take to upscale the result using this Topaz Video Enhance AI, if 59.94fps as opposed to 23.976fps. To which I'd respond, use nnedi3_rpow2 for the upscaling.

[lordsmurf]

I just do not understand the hard-on that some people have for upscaling.

Whatever you do will suck compared to the HDTV hardware upscaler. It makes intelligent decisions that software cannot. Software is sometimes really stupid, and this is one of those instances. Video hardware is still king for certain tasks. This is one of those tasks.

That Topaz software is laughably craptastic, marketed to low-video-knowledge suckers. Most of their software is idiotic. The one I find dumbest of all (at least for the moment, I'm sure they'll concoct something dumber in time) is the "JPEG to RAW" software. WTF? No. Hell no. GTFO of here with your BS, Topaz! We live in a disinformation age now, where too many people/companies prey on novices (newbies) with nonsense products.

[manono]

Me, I've never heard of this Topaz upscaler but after your ringing endorsement maybe litmus will rethink his workflow.

[litmus]

I just do not understand the hard-on that some people have for upscaling.

Whatever you do will suck compared to the HDTV hardware upscaler. It makes intelligent decisions that software cannot. Software is sometimes really stupid, and this is one of those instances. Video hardware is still king for certain tasks. This is one of those tasks.

That Topaz software is laughably craptastic, marketed to low-video-knowledge suckers. Most of their software is idiotic. The one I find dumbest of all (at least for the moment, I'm sure they'll concoct something dumber in time) is the "JPEG to RAW" software. WTF? No. Hell no. GTFO of here with your BS, Topaz! We live in a disinformation age now, where too many people/companies prey on novices (newbies) with nonsense products.

I'm not sure how familiar you are with Deep Space Nine, or how terrible the encode on streaming looks, or how problematic the DVD transfer is, but if you want to say that there is no difference between the DVD encode versus what you get as output from TVEAI, well you can say whatever you want, but the difference should be apparent to anyone willing to look with an open mind. I have a Sony 950G which has great upscaling but the difference between an episode I've ran through TVEAI and one I am playing off of a DVD rip and/or off of the DVD is quite noticeable. I went with the 950G in part because it has good upscaling but it can't compete with what this software does.

Just look at this.

Here's the intro without any processing:
https://youtu.be/DsOE73pxpys

Here is the intro ran through the tool (make sure you set the resolution to 4k, or the max capable of your device):
https://youtu.be/jojemqBip5U

Here's a live action scene that I posted further up the thread.

Unprocessed:
https://youtu.be/6VhSm6G7cVk

Processed:
https://youtu.be/oI1_T4xPmOU

Now I suppose you might just go into this with your confirmation bias against Topaz and not be able to see any difference, and you know what, that's fine. If it's important to you that everything Topaz does is crap, by all means run with that. I guess all of us excited about this new software and what it can do are just wasting our times.

[smrpix]

You want folks to judge quality based on files reprocessed by YouTube? Seriously?

[litmus]

You want folks to judge quality based on files reprocessed by YouTube? Seriously?

The Youtube feeds of the upscales look better than the full bitrate DVDs so ... yes?

There's also a VOB of the SD intro further up the thread if you prefer

[jagabo]

Filtered and upscaled in AviSynth, and encoded, all in less than a minute. I tried to get about the same sharpness as the youtube video.

[manono]

Looks good. litmus should forget Handbrake and Topaz if he wants good results in a reasonable amount of time.

Both examples of that scene on YouTube are flawed. The 'unprocessed' one was 29.97fps and the upscaled one was 25fps. Anyone serious about this project wouldn't want to wind up with speeded up video and audio, even if beginning with PAL DVDs.

[lordsmurf]

anyone willing to look with an open mind.

Don't try that "open mind" bull$hit on me.
This has nothing to do with philosophy, and everything to do with technicals/science of video.

Here is the intro ran through the tool (make sure you set the resolution to 4k, or the max capable of your device):
https://youtu.be/jojemqBip5U

I quit reading when I saw this:

Source video was ripped from the NTSC DVD using DVD Shrink then encoded using Handbrake with Detelecine filter to remove 3:2 pulldown before being processed in the AI tool.

You're really screwing up quality by doing this.

Upscaling is dumb. (Exception = mixed source documentaries.)
But if you insist on doing it from the DVD:

step 1 = extract the DVD (DVD Decrypter in IFO mode, or comparable) ... not DVD Shrink
step 2 = conversion from MPEG to lossless usually required
step 3 = edit, restore, upscale, whatever
step 4 = encode out

Processed:
https://youtu.be/oI1_T4xPmOU

This is equally unimpressive. The jerking is obvious, and distracting. It wasn't on the "unprocessed" clip. I really have no idea what's gone on here, be it your work, or Youtube, or a mix. I also see weird ghosting segments within some of the images.

confirmation bias against Topaz and not be able to see any difference,

Oh, I see a difference.
- fractal artifacting
- posterization
- oversharpening

That test is almost amusing to me. CG on black background is ridiculously forgiving, and is hiding a lot of errors. When the clips jumps back to actual live-action footage, everything falls apart, and the Youtube compression is most obvious of all.

jagabo's quickie Avisynth test is actually a bit worse, in terms of "jiggle" artifacts. This is a problem you run into, and it's present on all of these. I think Topaz is hiding it behind the ghosting, maybe avoiding it with by just letting the fractal noise leak through. The upscale algorithm gets confuses on the motion vectors, and it has a hard time pixel-locking the upscale. So it can be off some, so parts of an image can seem to move/jiggle because of it. Newbies get confused by this, not realize that an upscale is rendering new data, not just "blowing it up" (enlarging), or whatever.

Linear diagonals always look terrible, as bad as many non-QTGMC deinterlacing.

The "proper" way to upscale it to 4K would be to upscale it to like 16K, and then resolve back down. So if you want to upscale to 1080p, do 4K, then scale down. That may look better. May.

Just BTW, I've been upscaling images since before most people even knew it was possible. It's come a long way over the decades, and the "proof of concept" nature of video (multiple still images) is impressive. But it's very disappointing in practice, to the degree that the non-upscaled almost always looks better (and that's been confirmed many times by double-blind studies).

Topaz isn't a game changer, it's just a payware version of Avisynth (worse, actually).

I'm not sure how familiar you are with Deep Space Nine,

Big fan, BTW. Easily my 2nd favorite Trek after TNG.

I still remember recording the pilot on VHS the 1st night it aired (and still have that tape, WOC!). I'd been looking forward to it for weeks.

[litmus]

Y'all are some smug people.

1. DVD Shrink is NOT lossy if you save to a DVD Folder. It saves full bitrate VOBs. It's just a very lightweight and snappy tool and that's why I prefer to use it. Maybe if you actually used a tool instead of making assumptions you might know more about it.
2. Jagabo's example is worse than the Youtube example I posted, by quite a bit. One thing I immediately noticed is weird outlines around the credit text.
3. Do you all understand that machine learning upscaling is not the same as regular upscaling? I kind of think you don't.

One useful thing I got out of this thread is someone pointing out that the TVEAI video codec is crappy, this is true. I'm getting much better results now by outputting the frames as PNG files and then encoding to H264 using ffmpeg, so thanks to poisondeathray for pointing that out.

If you all want to read more about people upscaling DS9 with TVEAI here is some more reading:
https://www.extremetech.com/extreme/306701-upscaling-star-trek-deep-space-nine-using-t...ance-ai-review
https://www.extremetech.com/extreme/308505-star-trek-deep-space-9-upscale-project-update
https://www.reddit.com/r/startrek/comments/fnjdcq/star_trek_deep_space_nine_4k_using_ai_machine/

But again, if you are convinced this is not different than your TV upscaling and/or an AVI Synth script upscaling, what is the point right?

[jagabo]

Note that AviSynth's nnedi was one of the first generally available neural network upscalers. It was originally meant as a deinterlacer but was later adapted to more general upscaling. There are many settings you can configure to get better results. I just used the defaults. Also worth noting, it works best for cartoons (that's what I use it for). Using it with live action material can look very artificial (sharp edges with no detail between them).

I also used a simple Sharpen() that may have been responsible for the oversharpening halos around the titles.

[litmus]

Note that AviSynth's nnedi was one of the first generally available neural network upscalers. It was originally meant as a deinterlacer but was later adapted to more general upscaling. There are many settings you can configure to get better results. I just used the defaults. Also worth noting, it works best for cartoons (that's what I use it for). Using it with live action material can look very artificial (sharp edges with no detail between them).

I also used a simple Sharpen() that may have been responsible for the oversharpening halos around the titles.

OK I admit I didn't know that NNEDI was based on neural network. Nevertheless, comparing a couple of year old freely available script to a commercial software that came out two months ago (not that you did this, lordsmurf did) is a bit ridiculous. TVEAI is basically an automated version of their gigapixel software, which is a photo processing/enhancement software that attempts to extrapolate missing details based on portions of the image.

I don't know why I care that you guys are implying that a $200 commercial software is essentially a repackaged freely available Avisynth script, but I took some screenshots comparing the Avisynth upscale and the intro from one of my TVEAI upscaled episodes off of my HD. Start by comparing the starfields in these screenshots. That's not the only difference, but is the most obvious one.

[dannyboy48888]

Honestly this, voyager, and STNG I just ended up doing tdeint(mode=1, tryweave=true) in avisynth and left it at 60p. With x264 crf the bitrate didnt jump much and motion was preserved. Yes there's duplicate frames but on freezes they look fine to me. At 60fps it's not very noticable. That they play on my roku stick, bluray player, and raspberry pi. You could add srestore(frate=29.97) but I always had a jerky frame whenever the framerates changed in the source. Hqdn3d(2,2,3,3) cleaned up most residual noise I had.

[World Domination Studios]

how to deinterlace Deep Space Nine properly in Handbrake

The correct answer is "Don't use handbrake"

Get your VOB files from your DVD to your hard drive using a tool like Smartripper or DVD Decrypter. Then feed your VOB file into DGindex and save your project as a .d2v file. Then import that d2v file into avisynth and, if you're absolutely sure that you want to sacrifice smoothness of motion and effects to get 30 fps output, run the following:

Tfm(mode=0,slow=2,pp=5,mchroma=false,cthresh=2,MI=20,blockx=4,mthresh=2,micmatching=0)

:

However, if you'd rather preserve the high-framerate parts in all their glory, convert to 60 with the following:

Y=nnedi3(field=-2,nsize=3,nns=4,qual=2,etype=1)

A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=2,MI=20,blockx=4,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=2,MI=20,blockx=4,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

This will perfectly field-match the original progressive frames and then leave them the goddamn hell alone, while treating orphaned fields with a high-quality temporally-aware bob-deinterlace that leaves the original scanlines untouched. That makes it the most accurate double-rate deinterlacing algorithm you'll ever see.

If you see regular stuttering, switch field=0 to field=0 and vice versa in lines C and D.

[sophisticles]

Thanks, I should do a test at VFR. One piece of info I held back from you guys is that I am upscaling these in Topaz Video Enhance AI, which as I understand it works better with CFR, though I have not actually tested VFR. This is also why I want to limit it to 29.97 fps, a 23.976 fps episode encode takes about 4 and a half hours to process in the tool, a 29.97 fps encode takes about 5 and a half hours, and a 59.94 encode would most likely take twice as long as the 23.976. When you have 170+ episodes to convert, this adds up. Here's a clip of what one of these upscales looks like if you are curious: https://youtu.be/oI1_T4xPmOU. Closest we'll probably ever get to having the show in HD.

OK, I just have to ask, what is this obsession that seems to have crept up with numerous people trying to upscale either TNG or Deep Space Nine? Is it because of the Extremetech articles?

To me it's a ridiculous hobby. As a huge fan of THG, DSP and even Voyager, what made them great, especially TNG, were the stories and the characterizations, not the visuals or special effects. If they were to re-master TNG or DSP from the original source files to 4K, would it really make the shows anymore enjoyable? How about the Batman series from the 60's? What made those so much fun was all the camp, the corny over the top acting, the silly fight scenes, the absurd contraptions that the bad guys always used to try and kill Batman and Robin and silly ways that The Dynamic Duo found to get out of them.

To me, trying to upscale THG or DSP is totally missing/ignoring what made those series a must watch.

[Chauceratemyhamster]

One useful thing I got out of this thread is someone pointing out that the TVEAI video codec is crappy, this is true. I'm getting much better results now by outputting the frames as PNG files and then encoding to H264 using ffmpeg, so thanks to poisondeathray for pointing that out.

Can you explain a bit about how the quality has improved using this method? I'm using TVEAI to enhance DVDs of tv shows and want to achieve the best results. God I wish there was a simple deintelacer that could just deal with that aspect as it really is a massive head ache for non-technical people like myself. Apparently Topaz are looking to integrate a deintelacer into the software but it will take some time. I'm holding off using it to enhance my library till they implement it.

[Chauceratemyhamster]

Me, I've never heard of this Topaz upscaler but after your ringing endorsement maybe litmus will rethink his workflow.

Why not try it and see for yourself, then? It has a 30 day free trial

[Chauceratemyhamster]

I just do not understand the hard-on that some people have for upscaling.

Whatever you do will suck compared to the HDTV hardware upscaler. It makes intelligent decisions that software cannot. Software is sometimes really stupid, and this is one of those instances. Video hardware is still king for certain tasks. This is one of those tasks.

That Topaz software is laughably craptastic, marketed to low-video-knowledge suckers. Most of their software is idiotic. The one I find dumbest of all (at least for the moment, I'm sure they'll concoct something dumber in time) is the "JPEG to RAW" software. WTF? No. Hell no. GTFO of here with your BS, Topaz! We live in a disinformation age now, where too many people/companies prey on novices (newbies) with nonsense products.

You sound like someone who criticizes people for ordering pizza when they have flour, cheese and tomatoes at home. You are missing the point entirely.

[manono]

Why not try it and see for yourself, then? It has a 30 day free trial

No thanks. Why don't you try AviSynth's free upscaler nnedi3_rpow2