How do I make this anime video look so good upscaled from DVD to 720p

[Kyousuke]

This is the upscaled 720p:
http://limelinx.com/d9k52

I tried looking at the encoding parameters using MPC-HC but looks like that's a compressed re-encoded file, probably the original uncompressed 720p looks even better.

This is a 576p version of that:
http://limelinx.com/filje

They're both pretty good and I think that both were (probably) encoded from a 480p VOB source (because it's a DVD rip). You can see the difference using these snapshots in full-screen mode:

Snapshot 720p PNG
Snapshot 576p PNG

How do I obtain the same good results of that 720p if I want to upscale a 480p VOB? I'm really a beginner and so far I've used only MeGUI, do I need more programs or I can do everything just with this?

If you want to see the file I want to upscale, I'm uploading it now and I'll post the link later (my upload is kinda slow and it's 200 MB file). I'd like to convert it from MPEG2 480p to AVC 720p, using x264 at 10-bits.

Thanks in advance!!

[sanlyn]

Welcome.

Few members if any wouild download a 200MB video clip. Better to link to a smaller sample or two of a few seconds. You will get many more comments. If you don;t know how to cut a smaller clip from a VOB without damaging the original source, ask here.

Offhand, the image show very high black levels and severely clipped highlights. The images therefore look under saturated and washed out. Bright highlight detail in some scenes is destroyed and cannot be retrieved.


[Attachment 17708 - Click to enlarge]

[Attachment 17709 - Click to enlarge]

[jagabo]

Yes, fix the levels first. Then use an upscaler like nnedi3_rpow2. Sharpen a bit. Apply a line darkener like FastLineDarkenMod9().

[Kyousuke]

Thanks for the fast replies, so technically speaking that 576p is better than the 720p, I didn't really notice at first sight but looking at it now I see what you mean, I guess those settings used to upscale the video to 720p aren't so good as I thought. Those filters you're talking about, I found nnedi.dll but not FastLineDarken, there's no link on avisynth page: http://avisynth.org/warpenterprises is there any other up to date list of plugins?

This is a sample of the video that I wanted to upscale from 480p to 720p:
http://limelinx.com/dsnlm

It should be in raw quality (69 seconds), it was divided with MKV Merge.

[sanlyn]

Actually you might not need to do any line darkening or sharpening. nnedi3 should resize satisfactorily. Everything looks sharper and lines are darker with gamma and levels lowering for lowering the black point, and then notched contrast up just a small amount. These are samples with levels and gamma adjustment only -- you might want to play with individual shots (most people don't) with tweaks of some kind. All I used to tweak was some very slight adjustment with ColorMill. That's a matter of taste. I thought the vid was a tad too blue in the midrange. From your 720p sample, frames 191, 523, 1400, 1527 (click to enlarge):


[Attachment 17710 - Click to enlarge]

[Attachment 17711 - Click to enlarge]

[Attachment 17712 - Click to enlarge]

[Attachment 17713 - Click to enlarge]

[sanlyn]

Thanks for the demo.mkv, but that's not the unprocessed DVD original (? ?). You have some bad horizontal line interference, broken lines, and oversaturation (clipped colors). A piece of the original DVD (VOB) would be more appropriate.

Or is the mkv from a different project?

[poisondeathray]

Thanks for the demo.mkv, but that's not the unprocessed DVD original (? ?). You have some bad horizontal line interference, broken lines, and oversaturation (clipped colors). A piece of the original DVD (VOB) would be more appropriate.

Or is the mkv from a different project?

I think he just split the vob with mkvmerge. Use mkvextract-gui2 , or mkvextract to get the .m2v or .mpg

[Kyousuke]

To split the file I did all with MKVMerge importing the vob and the flac inside an existing MKV, since it wasn't re-encoded I thought there would be no loss in quality. I'm going to try again with Mpeg2Cut2 (10 mins).

[sanlyn]

MPG2Cut2 is one that I know of, but I use TMPGEnc MPEG Editor or Smart Renderer to make VOB/MPG cuts. Someone familiar with the free MPG cutters that don't do any re-encoding or processing could advise better. Playing the MKV again, the chroma problems aren't that bad, and maybe those twisting red ribbons are supposed to have coloring in them (I think). But you have some floating RF bars and horizontal lines all over the place.

If you just want to link to a cut that's m2v (no audio), use DGIndex.

[Kyousuke]

Probably the lines in the background are ok, the video is supposed to be something like a flow of projected memories or something like that, in the vob the colors look the same +/- this is a new file, created with mpg2cut2:
http://limelinx.com/8p6q

This video is really a lot darker than the first one, I wonder if it will still look good upscaled at 720p, or are there some good filters for this kind of colors too?

[sanlyn]

Ok, the thin horizontal line at the start look similar to lines on sheet music paper. The floating gray horizontal lines arenb't floating, but they do move occasionally -- looks like banding, which also shows up around the spotlight rings on the floor. The colors don't look as crazy as earlier.

Understand, I know how popular it is to upscale SD DVD. That might improve display on a big TV (if higher bitrates are maintained), if done properly and with a clean video source. I'd work on the banding problem; not sure why it should be there with a base bitrate near 8000, but it's there. Upscaled SD doesn't look like HD; it looks like upscaled SD. You can make slight but visible improvements, but keep in mind that HD looks the way it does because of higher resolution of the master, not because the frame was enlarged. Also, a source that's been previously encoded has already lost data from the original master; re-encoding doesn't replace it, even if the re-encode is HD.

Will look this over a little later today.

[Kyousuke]

Ok, the thin horizontal line at the start look similar to lines on sheet music paper. The floating gray horizontal lines arenb't floating, but they do move occasionally -- looks like banding, which also shows up around the spotlight rings on the floor. The colors don't look as crazy as earlier.

Understand, I know how popular it is to upscale SD DVD. That might improve display on a big TV (if higher bitrates are maintained), if done properly and with a clean video source. I'd work on the banding problem; not sure why it should be there with a base bitrate near 8000, but it's there. Upscaled SD doesn't look like HD; it looks like upscaled SD. You can make slight but visible improvements, but keep in mind that HD looks the way it does because of higher resolution of the master, not because the frame was enlarged. Also, a source that's been previously encoded has already lost data from the original master; re-encoding doesn't replace it, even if the re-encode is HD.

Will look this over a little later today.

Thanks for the support! I was planning to re-encode all the dvds I've (mostly anime and special amv like this), to watch them this summer on a 52 inch hd tv, but I'm not in a hurry so it's ok, even tomorrow or in the next week there's no problem.

Re-encoding many files I hope to understand more about filters and encoding settings, until now I've just copied presets from other forums or blogs but the result was never what I was trying to get.

[sanlyn]

[QUOTE=Kyousuke;2239679]

I was planning to re-encode all the dvds I've (mostly anime and special amv like this), to watch them this summer on a 52 inch hd tv, but I'm not in a hurry so it's ok, even tomorrow or in the next week there's no problem.

Ahem...I understand the zeal, but be prepared for the part about "I'm not in a hurry". This one video alone will be quite a project.

The VOB from the source MPG appears to be a re-encode of a previous PAL encode. It's encoded as 29.97 FPS interlaced (or 59.94 fields per second), but it's really progressive 25 FPS with 2:2 pulldown to make it play as 29.97 FPS NTSC. I think 2:2 is the right term, so correct me if I'm wrong. I understand this technique to be adding 1 duplicate frame for every 4 original frames. So as progressive video is plays with this frame sequence:

1 2 3 4 4
5 6 7 8 8
9 10 11 12 12

For anyone who wants to check this, I've attached a 3.5 second .m2v cut from the posted MPG.

How it plays in a media player depends on the player. VLC shows heavy banding, especially the dark floor and spotlights near the bottom. MPC and MPC-BE play smoother -- far less banding, but still visible. During the opening dissolves of the VOB sample, the lines I see look like PAL hanover bars to me. Again, I could be wrong but those horizontal lines shouldn't be there, IMO. I guess that's what they are, but they don't appear everywhere.

Two images below show the horizontal bars. They are 3X enlargements of portions of the original. The top image is from about frame 680, bottom is one of the last frames in the VOB sample.

[Attachment 17718 - Click to enlarge]

[Attachment 17719 - Click to enlarge]

Below, a 3X enlargement of banding and other compression artifacts, from about frame 680. Denoising/degraining/resize/re-encode will make these more distinct. It gets worse on fade-outs.

[Attachment 17720 - Click to enlarge]

Below, a 3X enlargement of the effects of bright clipping. The figures are supposed to be bright under the spotlights, but note how facialm detail and brightest colors are nearly gone. Most of the clipping occurs with red. From frame 1539:

[Attachment 17721 - Click to enlarge]

I came up with some code that will help reduce low-end crushing and detail loss, and some of the bright clipping.

SmoothLevels(1, 0.78, 255, 14, 230,chroma=200,limiter=2,tvrange=true,smooth=100,\
   dither=100,lmode=2,darkSTR=10) 
SmoothTweak(saturation=1.15)
ColorYUV(cont_y=-10,off_y=2)

SmoothLevels() and SmoothTweak() are in the SmoothAdjust plugin. The doom9 thread is here: http://forum.doom9.org/showthread.php?t=154971

[jagabo]

The VOB from the source MPG appears to be a re-encode of a previous PAL encode. It's encoded as 29.97 FPS interlaced (or 59.94 fields per second), but it's really progressive 25 FPS with 2:2 pulldown to make it play as 29.97 FPS NTSC.

A normal hard 3:2 pulldown interlaced video was deinterlaced (30i to 30p) resulting in every 4th frame being repeated. A simple TDecimate() will get rid of the repeated frames.

I think 2:2 is the right term, so correct me if I'm wrong.

That's the wrong term. In that terminology it would be 2:2:2:4 -- the first frame repeated for 2 fields, the second frame repeated for 2 fields, the third frame repeated for 2 fields, the fourth frame repeated for 4 fields.

[sanlyn]

Good, thanks for clarifying. Using SeparateFields, each "field" is an exact duplicate from each frame, including the 5th repeat. I should think TDecimate would drop too many frames, unless its parametres can be tweaked to get 25FPS out of it. I was afraid we'd end up with 15FPS or something like that.

[jagabo]

Good, thanks for clarifying. Using SeparateFields, each "field" is an exact duplicate from each frame, including the 5th repeat. I should think TDecimate would drop too many frames, unless its parametres can be tweaked to get 25FPS out of it. I was afraid we'd end up with 15FPS or something like that.

The source wasn't 25 fps. It was 24 fps. But in the case of 25 fps you could use TDecimate(mode=2, rate=25).

8 film frames:

1 2 3 4 5 6 7 8

3:2 pulldown by first repeating frames in a 3:2 pattern:

1 1 1 2 2 3 3 3 4 4 5 5 5 6 6 7 7 7 8 8

Then pulling down top and bottom fields:

1 1 1 2 2 3 3 3 4 4 5 5 5 6 6 7 7 7 8 8
t b t b t b t b t b t b t b t b t b t b

deinterlaced by throwing away all the bottom fields:

1   1   2   3   4   5   5   6   7   8  
t   t   t   t   t   t   t   t   t   t

Convert fields back to frames:

 1 1 2 3 4 5 5 6 7 8

[sanlyn]

Yes, I was just playing with TDecimate. It appears that one would want to delete frames repeated 3 times. But I'll try your suggestion first. Thanks again. Unusual frame rates really throw me off, so I've avoided them. May as well dig in and learn this.

[sanlyn]

Using TDecimate for 25FPS I get some slight but visible judder. 24FPS is smoother, and is the exact playback time as the original within .002 secs. 25FPS ran 1/2 sec. longer. The original clip is 0:01:10.737 according to VDub. Seeing as how 24p is BluRay compliant at 1280x720, I'll stick with it for now.

[jagabo]

All that happened was someone took normal 3:2 pulldown 29.97i fps recording and poorly deinterlaced it to 29.97p. This is not from a PAL/NTSC conversion. Take any hard telecined NTSC source and apply Yadif(). You'll get exactly the same thing, every fourth frame duplicated.

[sanlyn]

I figured the original had been around the block a few times. 3:2 pulldown in the original was my first notion (some day I'll learn to spot this processing as well as you). At any rate, I ended up with a 1280x720 24p vid that looks OK (?) but for a very small amount of judder. At least the banding looks a little cleaner than the original. I'll keep it around and see if I can learn to restore the original 23.97 with soft telecine. Will fiddle some more with this today. Thanks for the input.

[Kyousuke]

I tried using that script to encode the video from MPEG to AVC in MeGUI but it says "Script error: Invalid arguments to functions SmoothLevels line 3", so I tried creating a new avs file using the MeGUI avs script creator and writing the new lines manually, and this time it said this:



Always using VTS_01_1.MPG.
This is the full avs script (generated with MeGUI and modified after):

LoadPlugin("C:\MeGUI\tools\avs\directshowsource.dll")
LoadPlugin("C:\AviSynth 2.5\plugins\SmoothAdjust.dll")
DirectShowSource("C:\Users\Win7\Desktop\VTS_01_1.MPG", fps=23.976, audio=false, convertfps=true).AssumeFPS(24000,1001)
SmoothLevels(1, 0.78, 255, 14, 230,chroma=200,limiter=2,tvrange=true,smooth=100,\
   dither=100,lmode=2,darkSTR=10) 
SmoothTweak(saturation=1.15)
ColorYUV(off_y=2)
#deinterlace
#crop
#resize
#denoise

I'm probably doing it the wrong way, is it missing some basic plugin?
About the banding, I compared mpc-hc and vlc:

VLC snapshot
MPC-HC snapshot

I didn't notice before because I don't use vlc very often but in mpc-hc there's almost no banding, could it be a problem with some filter used by vlc?

[sanlyn]

No, it's just the way VLC plays some videos. I have occasional problems with it. Using MPC-BE most of the time now. When I laod the clips in VirtualDub I don't see banding either, I have to look for it.

The error shown is a colorspace message. SmoothAdjust wants YV12 only. Your MPG should be YV12 already, so I don;t know what's going on there. You might have the wrong masktools loaded. I'm using mt_masktools-26.dll. As it is, you'll need that version anyway as you'lll see in the script I'm posting soon. I have an earlier masktools in my plugins folder, but the '26' version is in a separate folder and I load that separately with LoadPlugin because I'm using both versions of masktools for different plugins.

Back in a short while.

[sanlyn]

Phooey, I did one encode, then made a stupid scripting error, started a new one, and another still running (slow encoder. I'll have to stop using it).

Link is to VTS_01_01_v1HD.mkv (58MB): http://dc340.4shared.com/download/GQwnl_x9/VTS_01_01_v1HD.mkv , 1280x740.

Blacks look a tad light at first, but the opening frames are plenty black. Remember, this clip darkens as it goes, and the end is pretty dark. Problem: the original encode juiced contrast too far, losing detail in darks and brights. As the mkv darkens you'll see dark disappear because of that. You can play with Levels and other params for a while, I did until I got tired of it. I ended up with a comprimse to avoid the worse-case level scenarios. Many TV's will make too-dark colors look posterized.

For a 50" TV I kept a fairly high bitrate (6500). If it were my vid, I'd go higher for my big plasma TV.

The script uses the Dither plugin and GradFun3. The "path" statements show where I keep different plugins and scripts, but you can set paths for your system. More denoising means more banding, and sharpening makes it worse (I know. I tried several sharpeners). You can always experiment. Once you get the "magic" script, it'll probably work for the whole vid.

vidpath="E:\forum\kyousuke\"
ppath="D:\Avisynth 2.5\plugins\"
dppath="D:\Avisynth 2.5\plugins\dither\"
Import(ppath+"QTGMC-3.32.avs")
LoadPlugin("D:\AviSynth 2.5\plugins\\mvtools2.dll")
Import(dppath+"Dither.avs")
Import(dppath+"mt_xxpand_multi.avs")
LoadPlugin(dppath+"avstp.dll")
LoadPlugin(dppath+"dither.dll")
LoadPlugin(dppath+"mt_masktools-26.dll")

MPEG2Source(vidpath+"VTS_01_1.d2v")
SmoothLevels(1, 0.78, 255, 12, 235,chroma=200,limiter=2,tvrange=true,smooth=100,dither=100,lmode=2,darkSTR=10) 

SmoothTweak(saturation=1.15)
ColorYUV(cont_y=-10)
deint=AssumeTFF().QTGMC(preset="very fast",sharpness=0.8,InputType=1)
TFM(order=0,clip2=deint).TDecimate(mode=2,rate=23.97)
Santiag(1,1)
GradFun3(thr=1.0,mask=0)
## -----------------------------------------------
## --- Resize - performed in a separate step, ----
## --- but can be done here (slows system !! )----
Spline36ReSize(1280,720)

Oops, I just finished another encode with TX264. Looks about the same (2MB bigger), but I'm still learning that one.

[sanlyn]

Well, for what it's worth, I think the TX264 encode looks a little cleaner. Same script, bitrate, etc., but took forever to encode. Audio seems less scratchy. Maybe I've heard it too many times today.
VTS_01_1_v2_TX264.mkv (59MB) http://dc112.4shared.com/download/k9ubn709/VTS_01_1_v2_TX264.mkv

Another learning experience. Criticism of the script welcome (I was in a hurry).

[jagabo]

I don't understand what the purpose of this is:

deint=AssumeTFF().QTGMC(preset="very fast",sharpness=0.8,InputType=1)
TFM(order=0,clip2=deint).TDecimate(mode=2,rate=23.97)

It doesn't seem to do anything appreciably different than a simple:

TDecimate()

[sanlyn]

QTGMC InputType=1 repairs bad, crummy, dirty, funky previous deinterlace (and reduces noise without destroying everything) using motion compensation. You could probably do without TFM, but it's a helper for TDecimate (tracks duplicate frames). You can try it without QTGMC as shown, but it won't be as smooth. Whoever deinterlaced the source to make the DVD did a rather sloppy job.

You can try more denosing/degraining, etc., but you'll fight banding, rough edges, and occasional block noise until the cows come home.

[jagabo]

I know what you're saying about QTGMC reducing noise, smoothing edges, etc. But in usage here the QTGMC deinterlaced video doesn't appear to be used at all. The only thing the script did differently for me was return different frames here and there because you used mode=2 rather than mode=0 in TDecimate(). Otherwise the images were identical. In this script (using the default TDecimate mode) there were no differences between v1 and v2:

Mpeg2Source("VTS_01_1.d2v")
v1=TDecimate() 
deint=AssumeTFF().QTGMC(preset="very fast",sharpness=0.8,InputType=1)
v2=TFM(order=0,clip2=deint).TDecimate()

sub = v1.subtract(v2) 
substrong = sub.levels(124,1,132,0,255) 
StackVertical(StackHorizontal(v1,v2),StackHorizontal(sub.subtitle("Difference"),substrong.subtitle("Difference amplified 32x")))

Even tiny differences are quite obvious with 32x amplification. Oh there was a small difference at the left of the topmost scanline of each frame. That was it. Do I have a different version of TFM or QTGMC? Am I missing something?

I don't think Mode=2 is the right mode to use here because the source is the result of a simple deinterlacing of a telecined film.

[sanlyn]

As mode=2 was suggested earlier, I took it up but decided that in the long run 23.97 FPS looked a little smoother. My first thought was to use a mode that would specify every 5th frame as being a 3rd duplicate, or specifying that any 3rd duplicate was a bad guy. But the mode 2 used was also suggested in a doom9 post in case the frame sequence should change. And I couldn't tell if the very first 3rd duplicate was frame #5 or not. There just might be a better mode setup that could be used.

I'm looking at a later copy of the script I actually used, and TFM and "deint" were commented-out. I apparently copied text from an early version. Cut and paste can be dangerous!

I'll Keep that source clip around. I'm certain it could be tweaked more than I had time to get to today. Anyway, thanks for spotting that glitch. I also tried a slower QTGMC preset, but it cleaned up too much and revealed more banding. But slower settings and some MCTD did get rid of almost all of those thin horizontal lines. But you can't have everything, I guess. I could live with those skinny lines, but not with thick banding.

[sanlyn]

3:2 pulldown by first repeating frames in a 3:2 pattern:

1 1 1 2 2 3 3 3 4 4 5 5 5 6 6 7 7 7 8 8

Then pulling down top and bottom fields:

1 1 1 2 2 3 3 3 4 4 5 5 5 6 6 7 7 7 8 8
t b t b t b t b t b t b t b t b t b t b

deinterlaced by throwing away all the bottom fields:

1   1   2   3   4   5   5   6   7   8  
t   t   t   t   t   t   t   t   t   t

Convert fields back to frames:

 1 1 2 3 4 5 5 6 7 8

Yes, I get that. That's what I'm seeing

Starting with frame 28 in the sample vob, a red ribbon floats across the screen from left to right. Without separating fields, but looking only at frames, the image sequence looks like this (bold numbers are repeated images):
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

So that pattern fits your earlier example. You can't just say "remove all duplicates", because in the slow-walk sequence some of those frames look as if they really were genuine dupes originally. I might not be able to get away with removing every 5th frame as I considered at first, because that sequence could change somewhere.

Meanwhile I just nodded off at the keyboard and typed about 200 lines of m's in this post. Will have to wrap my head around this in the morning.

[Kyousuke]

Well, for what it's worth, I think the TX264 encode looks a little cleaner. Same script, bitrate, etc., but took forever to encode. Audio seems less scratchy. Maybe I've heard it too many times today.
VTS_01_1_v2_TX264.mkv (59MB) http://dc112.4shared.com/download/k9ubn709/VTS_01_1_v2_TX264.mkv

Another learning experience. Criticism of the script welcome (I was in a hurry).

It looks great, if it were 10-bits could it help keeping more details in dark areas?