Need help with encdoing (ripping frames and then re-combine them)

[zanzar]

i have a bunch of dvds that i have to rip (480p,anime) BUT in an attemmt to improve their quality (please dont start argue about it and just help)

i make it in few stept.
1.make a "cleaner" rip from the dvs
2.rip all the frames from it (into jpg.)
3.run them trough waifu2x filter for anime
4.recombine them back into a video
5.run some line darkening scripts and other stuff like color correction.

ill explain a little bit more about the steps so you could understand what i am asking
1.by "cleaer" i mean that i make a rip with the simple scipt

TFM()
TDecimate(Mode=1)
McTemporalDenoise(settings="Low")
sharpen(0.15)

this allows waifu2x to work better and help it catch details and lines that otherwise would be blurred out.

2.in order to run it trough waifu2x that uses the gpu i need a program to rip all the prames into jpgs.

3.run all the jpgs trough "waifu2xcaffe"

4.use a program to combine all the jpgs into a video file (23.97 fps )

5.run the video file trough additional filters like "hysteria", also adding the sound and subs.

what i need is advice on how to do it with as less commpresion lossness up to the end render , for example with making the first rip, and when combining the jpgs back into a video ( i played around with it abit and found a program to combine the jpgs to video but it copressed the jpgs a lot which made the final video a little bit worse,also the program i used that combines the frames into a video only take a round number for frames ,24 fps and not 23.97)

also should i used x264 *scratchpad* or can we move on to x265?

[LMotlow]

i have a bunch of dvds that i have to rip (480p,anime)

I'm confused. If the DvD source is 480p, why are you running IVTC on it? Do you know what 480p means?

i make it in few stept.
1.make a "cleaner" rip from the dvs
2.rip all the frames from it (into jpg.)

Confused again. What you describe isn't "ripping".

1.by "cleaer" i mean that i make a rip with the simple scipt

TFM()
TDecimate(Mode=1)
McTemporalDenoise(settings="Low")
sharpen(0.15)

That isn't "ripping".

this allows waifu2x to work better and help it catch details and lines that otherwise would be blurred out.

Yes, it will be blurred and will have a lot of other problems, including lots of really bad block noise. But I won't argue about it. I'm turned off completely by people who use the word "rip" to mean anything and everything (that's why I'm ripping this on my keyboard after I went into the kitchen and ripped myself a ham sandwich). I'd caution that you won't get what you expect from the process, and you're definitely doing it the hard way.

So I'll skip the rest of this debate (Excuse me, I'll rip this debate) and maybe some other readers can dissuade you from botching your source video. But go ahead. It's your video, your time, and your money.

[jagabo]

Open your first AVS script in VirtualDub and export as a numbered image sequence (JPG or preferably PNG). Run Waifu2x. Import the resulting image sequence with an AviSynth script for further processing using ImageSource().

By the way, there is a Waifu2x filter for AviSynth but it's extremely slow. Personally, I'd just use sequential nnedi3_rpow2() calls with a little sharpening around them.

[Bernix]

As already mentioned, better use PNG (not indexed, presume even anime has more color) than JPG. I'm sure jpg can speed up work (lower size) but it seems to me you are aiming for quality.
@jagabo
Waifu2x application support cuda acceleration, not sure in AviSynth is it handled through cuda cores also. And speed difference is very very significant.
The difference GPU vs CPU is 13 x faster GPU.

Edit: to get video to images, you can also use potplayer, with Avisynth if you need and has script and know how to use it. And with or without filters.
add to Waifu2x, once tried resize 2 colors shapes, and it created weird thing.

Bernix

[Bernix]

Sorry,
just one example of Waifu2x on straight lines. Original was just Black and white color. No more.

[Attachment 44485 - Click to enlarge]
And just multiplied by 2

[Attachment 44486 - Click to enlarge]
Of course it is zoomed, but is it easiest task for scaling.

[zanzar]

Sorry,
just one example of Waifu2x on straight lines. Original was just Black and white color. No more.

[Attachment 44485 - Click to enlarge]
And just multiplied by 2

[Attachment 44486 - Click to enlarge]
Of course it is zoomed, but is it easiest task for scaling.

what are you talking about? i runned them and they came perfectly fine...

[Bernix]

I was talking about the original image hasn't that grey in between black and white. Nor the grey in the corner. And resize square, not rotated any angle should be easiest think on the earth. I believe all usual resize algorithm can do this without those grey colors.
Here is original image before waifu2x resize

[Attachment 44488 - Click to enlarge]

Bernix

[Bernix]

O.K. it isn't simple task then, only for nearest neighbor then. Apologize.

Bernix

[zanzar]

Open your first AVS script in VirtualDub and export as a numbered image sequence (JPG or preferably PNG). Run Waifu2x. Import the resulting image sequence with an AviSynth script for further processing using ImageSource().

By the way, there is a Waifu2x filter for AviSynth but it's extremely slow. Personally, I'd just use sequential nnedi3_rpow2() calls with a little sharpening around them.

thank you again for helping, i know poeple are not a fun of people trying to do the king of stuff i am doing (weh it will came worse! etc) but i am more then assured myself it works,since its different, all what really need to be done for anime to look better (at least on the type i am doing) is to eliminate noise and artifacts, make the lines firm and distinguishable which waifu2x does really good job also using "McTemporalDenoise" and a little bit sharpen helps it to get lines that i might miss.

i use nnedi3_rpow2() to downacale it to 1080p at the end.
also i am not very fammiliar with virtual dub, and the script -

LWLibavVideoSource(what ever source)
McTemporalDenoise(settings="Low")
sharpen(0.15)
Hysteria(strength=0.5, maxchg=30,lowthresh=18)
McTemporalDenoise(settings="Low")

seems not to work on it without plugins (which i dont know what they are)

if i understood what you explained, you are telling me to, to run the script trough virtual dub (including de interlacing and other stuff) for example

Mpeg2Source("I:\VIDEO_TS.d2v")
TFM()
TDecimate(Mode=1)
McTemporalDenoise(settings="Low")
sharpen(0.15)
Hysteria(strength=0.5, maxchg=30,lowthresh=18)
McTemporalDenoise(settings="Low")

and make it export it into single frames.
then run the frames trough waifu2x, then use this new frames with imagesource() and othe filters i want to make the final file (this way i avoid any compression to a video, only at the end)

how can i run the avs script trough virtual dub? (can they run the same as avspmod?) i guess its just not hoocked into my avisyth plugins , is there a way to do this?
sorry for asking retarded questions.

also how do i set source with "ImageSource" , for the folder with all the imgaes. for exmaple "D:\jpg"

on the wiki it states "ImageSource("%d.jpeg", 100, 199, 29.97)" but can figure out what "%d.jpeg" is

[jagabo]

If your script works in avspmod it should work in VirtualDub. Just use File -> Open Video File, then File -> Export -> Image Sequence.

VirtualDub will create images with a base name followed by a selectable number of digits. For example, with the name pic and 5 digits:

pic00000.jpg
pic00001.jpg
pic00002.jpg

To import those into AviSynth later:

ImageSource("pic%05d.jpg", fps=23.976)

Note the base name pic and %05d indicating 5 decimal digits padded with leading zeros. The default is to start at 0 and end at the last image in the sequence. If the script isn't in the same folder as the images you have to provide a path to the images:

ImageSource("C:\images\pic%05d.jpg", fps=23.976)

[zanzar]

If your script works in avspmod it should work in VirtualDub. Just use File -> Open Video File, then File -> Export -> Image Sequence.

VirtualDub will create images with a base name followed by a selectable number of digits. For example, with the name pic and 5 digits:

pic00000.jpg
pic00001.jpg
pic00002.jpg

To import those into AviSynth later:

ImageSource("pic%05d.jpg", fps=23.976)

Note the base name pic and %05d indicating 5 decimal digits padded with leading zeros. The default is to start at 0 and end at the last image in the sequence. If the script isn't in the same folder as the images you have to provide a path to the images:

ImageSource("C:\images\pic%05d.jpg", fps=23.976)

ok everything looks to work fine , the only thing is i still cant get virtualdub to run with avisyth.
i extract the VD files into some folder. run it (32 bit or 64 tried both) and it can only play avi files.
is there a specific way i need to set it up?

[jagabo]

i still cant get virtualdub to run with avisyth.
i extract the VD files into some folder. run it (32 bit or 64 tried both) and it can only play avi files.
is there a specific way i need to set it up?

Post your AviSynth script and the error message you get.

[zanzar]

[Attachment 44489 - Click to enlarge]

i still cant get virtualdub to run with avisyth.
i extract the VD files into some folder. run it (32 bit or 64 tried both) and it can only play avi files.
is there a specific way i need to set it up?

Post your AviSynth script and the error message you get.

LWLibavVideoSource("d:\4.mp4",format="YUV420P8")
McTemporalDenoise(settings="Low")
sharpen(0.15)
Hysteria(strength=0.5, maxchg=30,lowthresh=18)
McTemporalDenoise(settings="Low")

thats for a file that is allready a video
also this one for VIDEO_TS.d2v if i use a DVD

Mpeg2Source("I:\VIDEO_TS.d2v")
TFM()
TDecimate(Mode=1)
McTemporalDenoise(settings="Low")
sharpen(0.15)
Hysteria(strength=0.5, maxchg=30,lowthresh=18)
McTemporalDenoise(settings="Low")

[jagabo]

Use File -> Open Video File to open AviSynth scripts. Not File -> Run Script (that's for VirtualDub's own scripting language).

[zanzar]

Use File -> Open Video File to open AviSynth scripts. Not File -> Run Script (that's for VirtualDub's own scripting language).

thanks ! everything works great , now last thing.
if i have like 1000 frames credits that i have no reason to waste time on them, i can make a script that skips them and just exports them as they are , but how do i tell to recombine them without applying the last script?

for example applying this to all the frames exept the credits and just taking them as they are, and combining them into the video file

ImageSource("d:\picc\pic%06d.jpg",000,633, fps=23.976)


converttoyv12()
#what ever script
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1920, fheight=1080)

[jagabo]

Something like:

clip1 = ImageSource("d:\picc\pic%06d.jpg",000,633, fps=23.976).converttoyv12()
clip2 = WhateverSource(...)
clip1+clip2 # or clip2+clip1, whatever order is necessary
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1920, fheight=1080)

[zanzar]

Something like:

clip1 = ImageSource("d:\picc\pic%06d.jpg",000,633, fps=23.976).converttoyv12()
clip2 = WhateverSource(...)
clip1+clip2 # or clip2+clip1, whatever order is necessary
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1920, fheight=1080)

jagabo any idea what is this artiffacts coused by increasing the levels?

levels(19, 0.85, 246, 5, 255, coring=false)

i think its present even without the levels , just much less visable.
any idea how to get rid of it ? its mostly present on draker colored frames and not on multicolored ones.

[poisondeathray]

i think its present even without the levels , just much less visable.
any idea how to get rid of it ? its mostly present on draker colored frames and not on multicolored ones.

You think ?? Did you look at the original ?

If it's there - you should clean it up before everything else, before upscaling, otherwise you just upscale artifacts

jpg doesn't help either (lossy even at highest quality) , it was mentioned above you should use png

You might have to apply dithering at the end to cover up problems

[Bernix]

jpg doesn't help either (lossy even at highest quality)

There is lossless JPEG, and we have it in national library, but didn't used it. To public we used DJVU and for archival purpose TIFF.

Bernix

[poisondeathray]

jpg doesn't help either (lossy even at highest quality)

There is lossless JPEG, and we have it in national library, but didn't used it. To public we used DJVU and for archival purpose TIFF.

Bernix

He's probably using a waifu2x-caffe GUI . It doesn't accept LJPEG .

The other option is vapoursynth, which has waifu2x-caffe ; no need for intermediates like jpeg or png

[zanzar]

i think its present even without the levels , just much less visable.
any idea how to get rid of it ? its mostly present on draker colored frames and not on multicolored ones.

You think ?? Did you look at the original ?

If it's there - you should clean it up before everything else, before upscaling, otherwise you just upscale artifacts

jpg doesn't help either (lossy even at highest quality) , it was mentioned above you should use png

You might have to apply dithering at the end to cover up problems

its there in the original. its just that levels makes it worse. also it has nothing to do with the jpg.
it is only noticable at this scene

[Bernix]

@poisondeathray,
o.k. sorry. Don't really know if it accepts it and can't try, but of course PNG is much better and I think also smaller than lossless JPEG. Can't high level of denoise do bit with this artifact? Of course first option Denoise and Magnify?

Bernix

[zanzar]

i think its present even without the levels , just much less visable.
any idea how to get rid of it ? its mostly present on draker colored frames and not on multicolored ones.

You think ?? Did you look at the original ?

If it's there - you should clean it up before everything else, before upscaling, otherwise you just upscale artifacts

jpg doesn't help either (lossy even at highest quality) , it was mentioned above you should use png

You might have to apply dithering at the end to cover up problems

its there in the original. its just that levels makes it worse. also it has nothing to do with the jpg.
it is only noticable at this scene
how do i clean it?

[jagabo]

It's posterization. In 8 bit rec.601 or rec.709 YUV video there are only about 1/6 as many unique YUV values are there are in 8 bit RGB, even though they both cover the same range of colors. That means there is a bigger "distance" between adjacent YUV values. When you use levels to increase the contrast the distances become even bigger. Try using SmoothLevels() instead of Levels(). That will add some noise and make it less visible. Or maybe use GradFun3 before or after the levels adjustment.

[zanzar]

It's posterization. In 8 bit rec.601 or rec.709 YUV video there are only about 1/6 as many unique YUV values are there are in 8 bit RGB, even though they both cover the same range of colors. That means there is a bigger "distance" between adjacent YUV values. When you use levels to increase the contrast the distances become even bigger. Try using SmoothLevels() instead of Levels(). That will add some noise and make it less visible. Or maybe use GradFun3 before or after the levels adjustment.

thanks! i noticed this heppends in all the dvd i has. most times when there is a "smooth" frame which consists of one color (semiliar to the red hair in the pic i posted)
any suggestions on how to minimize this effect from the start?
what filters generaly make it worse?
for example does the one from the first "stage" is affecting it? (no,by my comparison but since you know everything around here ill ask...)

TDecimate(Mode=1)
McTemporalDenoise(settings="Low")
sharpen(0.16)
Hysteria(strength=0.5, maxchg=30,lowthresh=12)
McTemporalDenoise(settings="Low")

i found denoisers help the most, any other guide lines to follow to minimize it? aside just generally avoiding raising levels to much ?

[jagabo]

It can happen any time there are shallow gradients. Removing noise makes it worse. Though MCTC at some settings adds back a little noise to help prevent it.

[zanzar]

Something like:

clip1 = ImageSource("d:\picc\pic%06d.jpg",000,633, fps=23.976).converttoyv12()
clip2 = WhateverSource(...)
clip1+clip2 # or clip2+clip1, whatever order is necessary
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1920, fheight=1080)

how do i run if i need something like this

clip1- ImageSource("D:\pic\pic%06d.jpg",00000,35576, fps=23.976)

converttoyv12()
ColorMatrix(mode="rec.601->rec.709")
smoothlevels(18, 0.83, 246, 5, 255)
tweak(sat=1.1, coring=false)
Hysteria(strength=1.0, maxchg=30,lowthresh=8)
Sharpen(0.2)
Hysteria(strength=1.0, maxchg=35,lowthresh=6)
Blur(0.35)
Hysteria(strength=1.0, maxchg=30,lowthresh=4, highthresh=60)
Blur(0.35)
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1280, fheight=720)
McTemporalDenoise(settings="medium")
GradFun3 ()
Dehalo_alpha(rx=3, ry=3, darkstr=0.5, brightstr=0.5)

clip2 - ImageSource("D:\pic\pic%06d.jpg",35577,38359, fps=23.976)
nnedi3_rpow2(2, cshift="spline64resize", fwidth=1280, fheight=720)

clip1+clip2

mixing with the credits in the end that no need for the script and save time (but i need to resize them to 720p in order to combine them)