I intend to begin restoring dv files already transferred using WinDv/ Sceneanalyzer. However I have doubts related to which will be the best method to open the files to get the best results just before applying the restore filters. All of them are DV Pal files probably recorded using sony dv codec (from Sony camcorders).
Depending of the files I could begin with an avisynth script or open them previously on premiere or even use virtualdub or a mixed approach.
I intend to be able to force, if I want to, the use of a particular dv decoder letīs say mainconcept, cedocida, Panasonic, Microsoft or any other that could prove useful ( to confirm if any of them could provide better results ).
However I donīt see in avisynth how can I force the use of a specific dv decoder.If I do open the files in avisynth with avisource I can force the use of a decoder with a specific FourCC , however in this moment I do have several dv codecs installed in my system ( Cedocida, Mainconcept, Microsoft ) but GSpot gives for all of them the same FourCC DVSD so I cannot choose specifically one ( theoretically mainconcept DV codec should have used a different fourCC =DVCS but if I use it avisource doesnt work).
Using avisource without specifying pixel_type will give by default YUY2 colorspace the same as explicitly using YUY2 ,trying to specify YV12 doesnt work using pixel_type RGB also does work however I donīt know for sure which dv decoder is being used ( eventually cedocida as is assigned in the registry) but I donīt know for sure and if there is a way to select the codec I could eventually prefer.
If I do open the dv file with ffvideosource I do get by default YV12 ( which could eventually be better as the Pal original colorspace is 4.2.0 so apparently no colorspace conversion and it seems to be faster than avisource perhaps because is multithreading) but again I also donīt know which decoder is being used and if there is a way to change it if I want to.
Opening the file in virtualdub itīs in my case the only situation where is clear that itīs cedocida the codec being used , however I cannot use other codecs namely for instance mainconcept as it is a directshow filter.
Looking in the registry the active assigned dvsd dll codec is cedocida so I presume that if I use premiere to correct color and exposition in first place for instance is cedocida that is used but again Iīm not sure as I have read that premiere does include the mainconcept dv codec.
The only pratical way I see to force the use of a specific dv decoder is using a tool like graphedit/graphstudio but in this case I will have to use DirectShowSource with the GRF file in an avisynth script or transcode to an intermediary lossless format as utvideo in YUV or RGB depending of the next restore tool to be used (virtualdub/avisynth/Premiere.
So my problems
1- Identify using , avisynth, or premiere which is the dv codec used and after testing and deciding which one to retain ( probably mainconcept or cedocida) being able to force the use of the selected dv codec in the specific required restore application.
2- For what I have read it seems mainconcept or cedocida could be the best dv codecs, not sure if there is new info related to this subject debated is the past
3- Decide between using avisource or ffvideosource , this last seems to provide the original colorspace YV12 so eventual preventing one colorspace conversion and seems to be fast
4- Eventual benefit of using a tool as graphedit to assure the use of any specific dv codec vfw or directshow and use Directshowsource with the GRF file in the avisynth restore script or use an intermediary lossless format like utvideo to do all the subsequent restoration
I hope someone can shed some light on this subject and elaborate on the pros and cons of the several options
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 30 of 40
Thread
-
-
it's crappy old standard def DV. open in your editor of choice, preferably one like vegas that works well with native DV, edit and render to whatever your final destination is. it's not rocket science it's an old easily used format.
--
"a lot of people are better dead" - prisoner KSC2-303 -
It doesn't matter what DV codec gets used for decoding because DV will always decode to the same exact thing no matter which codec does the decoding. However, it makes a HUGE difference which one you use when encoding. The Windows DV codec used to be absolutely horrible (and probably still is, if it even gets included anymore). The Vegas DV codec is the gold standard, and the MainConcept one isn't bad.
In almost every application I have, you get to choose which one you want. Unfortunately, Vegas does not register its DV codec in such a way that it can be used outside of Vegas. Thus, if you truly want to use that from within another application, you have to use frameserving, where you serve into Vegas from that other application and then let Vegas encode using its built-in DV codec.
I found that the MainConcept DV codec is almost as good as the Vegas codec (neither one should give you noticeable degradation if you only do one generation of encoding). So that's what I use when I want to encode DV video from another application (like VirtualDub). I still do a lot of "crappy old standard def" work, so I am very familiar with DV, the various DV codecs, and DV's limitations. It sure is easy to use, and fun to edit with (no pauses or delays, etc.). It was the codec that enabled the digital video revolution. -
This isn't quite true. They may all decode the same internal 4:1:1 but some will output RGB (4:4:4), others YUY2 (4:2:2), or others YV12 (4:2:0). Sometimes without any choice, sometimes depending on configuration. For example, Panasonic DV codec always outputs RGB. Cedocida lets you configure it. It even allows for full range vs. limited range, and DV vs. Mpeg2 interlaced and MPEG2 non-interlaced chroma placement for YV12. Of the three formats I listed, YUY2 (YUV 4:2:2) is closest to NTSC DV's 4:1:1 chroma subsampling. For PAL DVD, YV12 is closest.
-
I stand corrected. I didn't realize that some performed colorspace conversion.
If what you say is true, then someone needs to answer the OP's question as to how you can force a program to use a particular DV codec for decoding, if more than one is installed. I seem to remember a way to do this, fifteen years ago when DV was king, and the DV codec built into Windows was to be avoided at all costs. I can't remember now how it was done, and I think it was some sort of Windows 98 hack, probably not available in more modern Windows versions.Last edited by johnmeyer; 13th Feb 2017 at 23:55. Reason: added last paragraph
-
A filter manager like FilMerit, DSFMgr, or Radlight Filter Manager should allow the OP to force a decoder by specifying its merit (highest merit is used by VFW or DirectShow.
-
Thanks for all contributions. A special thank you to jagabo for the info related with filter merit management which should allow me to test the use of specific codecs. I will try also a small utility video codec swap that does allow to bounce between codecs.
Theoretical the best dv codec for me would be the Sony DV codec as the miniDv have been hard encoded using this codec in a Sony camcorder. It seems also itīs the codec used by Vegas. However after googling a lot I have been unable so far to find it, seems to be hidden in third party applications and also probably a vfw version doesnt exist.
DV codecs even for decompression have issues as colorspaces problems signaled by jagabo. There are also cases in literature where bad Chroma up sampling and dv decoding problems does arise.
It seems also that most if not all miniDv camcorders can output luma in between 16-255 that could also pose a problem namely clipping the 235-255 portion if an application is not aware or if an adjustment is not previous made. In the example I include it seems that this is true and doing a levels adjustment with levels or smooth levels to get 16-235 could improve the results and recover some detail otherwise lost in the whites/super whites ( see the clouds in the example included).I used levels perhaps smooth levels can provide a more smooth histogram without spikes.
Cedocida seems to work fine and with the configuration panel adjusted itīs able to output YV12 ( more closed related with PAL 4.2.0 ) without color shift, interpolation or conversion when using YV12 / DV options.Having high merit, being correctly installed and assigned by default in registry and being a vfw codec assure itīs use by default in my case by virtualdub and also by Avisynth/avisource as I think avisource doesnīt support directshow ( also the case of the mainconcept codec) and Microsoft itīs not assigned as default in registry.
FFmpeg DV decoder (ffvideosource) is however amazing fast and does provides also YV12 and comparing with cedocida using MSU Video Quality Measurement Tool does seems to provide a small benefit to ffmpeg ( better signal to noise ratio and less blurriness ). Tested dv files opened with avisource vs ffvideosource and saving in both cases to lossless utvideo 4.2.0 and comparing final results with MSU.
I donīt know if anyone as compared FFmpeg dv codec with the traditional dv codecs , I have searched but have not found anything however it does seems to provide a benefit.
I made also some tests using graphstudio using mainconcept and cedocida and converting to final lossless utvideo 4.2.0 the results have been always worse that opening directly in vdub or through avisynth however between the two graphstudio pipelines cedocida vs mainconcept there is a marginal advantage to cedocida (even if is a vfw and the pipeline has to have an extra lav decoder and dv decoder).
Still have to see if I can find a Sony DV codec but I donīt think it exist a vfw version even if a directshow version exist I will probably have to use directshowsource (something for mainconcept) what I think itīs less reliable.
Conclusions so far
- Cedocida is fine outputs to YV12 and does that without color shift, interpolation or conversion with the correct YV12/DV options
- FFmpeg (ffvideosource) seems marginaly better and itīs really fast
- DV luma seems to be in between 16-255 and seems to need levels adjustment ( levels or perhaps preferably smoothlevels for instance)
Edit- I tested smoothlevels and is the correct levels adjustment to use , the histogram is fine no spikes and better final result
Note- Not related with this issue
Due to several crashes using MTmode with avisynthMT 32bits and virtualdub 32 bits and as I thought most of the problems are memory conflicts or lack of addressable memory I decided to make the vdub app large address aware ( 32bit app in 64 bit OS can only address 2GB if they are made large address aware they can address 4GB) and see it could give a benefit, I used a small utility named 4GB_Patch applied to the two Virtualdub .exe and Vdub.exe, setMemoryMax to about 1600 in the scripts and now virtualdub almost never crash with MTmode2 and even 1 (not always) and itīs a lot more stable in MTmode.
Fig1- DV luma 16-255 raw left - corrected right
Fig2- avsource_cedocida_left vs FFmpeg rightLast edited by FLP437; 15th Feb 2017 at 22:15.
-
The difference in decoders is interesting.
Theoretically, cedocida (if forced or set to YV12 for PAL DV) and ffmpeg should give identical results. But in your screenshot I can already see vertical artifacts . I'm wondering if something else is going on, for example different resizer for the screenshot.
How did you decide on decoding the "original" for the MSU tests ? Because that could skew your results if there was indeed a difference between decoders -
I just tested some random PAL DV clip and when setting the same chroma siting (DV) and forcing YV12, there are tiny microscopic differences that can be detected with amplifed difference testing (not as large as in your screenshot - you can't see them with naked eye, even with zoom on frame by frame). But it's debatable (at least on this clip) which is "better" . But still interesting that there are mathematical differences
-
@johnmeyer
I have only one slight problem, I donīt have Vegas, I do have Premiere. If I have had vegas I would have already tried it, but thanks anyway
@poisondeathray
The difference seems minimal but even so!.. I have used MSU because I didn't saw any difference with naked eye.
I used a small script to load the dv file and used alternately avisource or ffvideosource selected 4.2.0 yv12 and utvideo YUV420 BT.601 VCM in virtualdub and made two lossless utvideo files I tested with MSU. the FFmpeg almost doubles the speed of avisource ( 220 fps vs 120 fps)
Code:#AviSource("D:\DV\base.avi", fourCC="DVSD", pixel_type="YV12") ffvideosource("D:\DV\base.avi") SmoothLevels(16, 1.0, 255, 16, 235)
-
No, there are significant differences in your screenshot that you can see with human eye. For example just pay attention to the tower. There are vertical bar artifacts present on the left. That makes me thing differences in processing other than decoder
I used a small script to load the dv file and used alternately avisource or ffvideosource selected 4.2.0 yv12 and utvideo YUV420 BT.601 VCM in virtualdub and made two lossless utvideo files I tested with MSU. the FFmpeg almost doubles the speed of avisource ( 220 fps vs 120 fps)
Code:#AviSource("D:\DV\base.avi", fourCC="DVSD", pixel_type="YV12") ffvideosource("D:\DV\base.avi") SmoothLevels(16, 1.0, 255, 16, 235)
But you're testing against UT Video, now defined as the "original" for testing purposes. How did you make the UT Video version ? Whatever decoder you used for that will score higher
For example ,
1) DV => cecocida => UT
2) DV => ffvideosource => UT
cecocida will score higher in (1), but ffmpeg will score higher in (2) , no matter what metric you use
Smoothlevels just dithers the result (adds noise) . You can do it with plain levels() too (dither=true) -
well eventually there is an incorrection as avisource could be using OpenDML or AVIFile interface however I tried to force it to use a specific decoder and as cedocida is the only fourCC="DVSD" registred I assumed it will be using cedocida. Anyway I tried to reflect the usual situation where I will be using an avisynth script with avisource or ffvideosource. If I open the dv file directly in virtualdub and save to utvideo there are no differences whatever.
so for me it was
dv->avisource (using cedocida)->UT
dv->ffvideosource->UT -
I agree there's a visible difference between the left and right images. But it's the scaling. The left is point resized, the other is probably bilinear.
-
If you have internal DV codecs disabled in vdub, it will use the VFW codec with the highest merit when you open a file natively (not through avs). (options => preferences => prefer internal video codecs over installed third-party DV and MJPEG) is DISABLED
Then with an AVI loaded directly, File => file information will report which VFW decoder is being used by vdub. That will also be the same one used with AVISource()
But do you understand the MSU test isn't useful ? You don't have the "true" original, before the DV compression. You only have the original DV recording from the camera (not the raw sensor data prior to DV compression) .
So you can't use MSU to test DV decoding properly, because you're 1 generation removed. Whatever decoder you choose to decode the DV recording to make the UT video version , will score higher . (The original DV has to be DEcoded with something for you to test) . So MSU testing set up like that doesn't provide any useful information
But I can confirm there are microscopic differences between the 2 decoders, just not to the magnitude depicted in the screenshot. -
Well I can confirm that virtualdub is using Cedocida as I have made the modification indicated by you when I installed cedocida and I can also see in file information that cedocida is the codec used. So that confirms that avisource is using cedocida to decode as I already presumed.
I understand iīm subverting somewhat the MSU use as Iīm not comparing against the original but I sought it could be also used to compare two second generation not against the original but between them. If I change the relative position original vs processed it gives exactly the same thing the ffmpeg marginally better itīs not the original that has the largest values.
Anyway you can confirm that the ffmpeg approach is "microscopical" better is that so ? and is also fastLast edited by FLP437; 16th Feb 2017 at 01:08.
-
@jagabo
is it possible that one image is point resized and the other probably bilinear without making any resize or scaling only using the indicated script ? -
No - you can't say anything about "better" or "worse" in terms of quality or any related metric using that test with any validity (or any MSU test, or any metric , the way you have structured it) , because you don't have an objective "original" to test it against. You're choosing the decoder to decode the "original" , so that skews the results (bias). If the decoders didn't give different outputs, then yes, it would be a valid test. But then you wouldn't need to do this test at all because everything would be identical.
You can only say that there are differences between each other, when compared against each other
I understand iīm subverting somewhat the MSU use as Iīm not comparing against the original but I sought it could be also used to compare two second generation not against the original but between them. If I change the relative position original vs processed it gives exactly the same thing the ffmpeg marginally better itīs not the original that has the largest values.
yes, ffms2 and lsmash get 3-4x faster. (I'm getting about 1000fps for just decoding speed). And lsmash and ffms2 are both libav based so they produce identical outputs (confirmed). -
It probably doesn't matter, quality wise
I don't like the clutter with indexes with ffms2 and lsmash have with AVI (If you have a lot of footage the extra time indexing and extra clutter can become large). And they are faster because they are internally threaded even with vanilla avisynth. When you set threads=1 in the source filter, they become slower too. Some filter operations are unstable with ffms2 and lsmash with threads > 1 in the source filter
If you're doing other processing , the fps is probably going to be a moot point . When you decode at 1000 fps, the CPU usage is higher too. What you want to test is actual end to end workflow . Decoding fps is only 1 part of it
Avisynth mt can be unstable as you've seen . It really depends on the operations being done. Also have a look vapoursynth . It's dramatically improved the last few years. The threading is much better. Even single instance of QTGMC is a few % faster than avisynth mt now . And it doesn't crash as often. And when you stack a few filters it can be 2-4x faster. Avisynth MT tends to thrash with threading issues resulting in slow fps when more than 1 filter is used (if it doesn't crash in the first place) . Also have a look at Avisynth+ , which can be slightly faster than avisynth MT. The problem with vapoursynth is a more difficult learning curve, more finicky syntax than avisynth, and not all filters are available yet. The vapoursynth editor isn't as good as avspmod either (no tabbed previews) -
-
also, if you going to want help, you must make it easy for all to assemble. you have jumbled (glued) the images together as one. how can anyone get a accurate measurement out of it if you mesh them together like that? you must post separate images, and at the the same size untouched. if they are screenshots of virtualdub, then you must keep them all equally the same size. if you are posting actual video image, then you must post the image as the one size, i.e., 720x480 or 720x576 or , ... the pairs have to be the same size. don't make the helpers have to do that work. it gets tiresome to do and i don't do it. so please keep that in mind next time you post images.
if you want to post the images from virtualdub v1.9.11 and greater, then you can simply press ctrl+1 (for the left pane side) and ctrl+2 (for the right pane side). just be sure that under menu/video that you check-mark 'input pain' and 'output pane' so that the ctrl+1 and ctrl+2 work or else you will get no image in the clipboard to paste and save using your graphics editor or ms built-in paint program. and save as PNG only. ctrl+1 is the untouched source video. ctrl+2 is the filtered output video (assuming you added filters in virtualdub). -
Some other things to consider:
Any images you post will have been converted to RGB. That may be a problem depending on what you are trying to show.
VirtualDub does not handle interlaced YV12 properly. It will convert it to RGB as if it was progressive YV12. That will blur the chroma channels together. -
@jagabo
The images have not been resized they have been cropped to get rid of the borders what I think is a passive operation and don't introduce any other modification on the image. -
Well to present the images on the monitor they are obviously converted to rgb and eventually this could pose a problem. I have verified with info () that it's yv12 interlaced but I can eventually force a converttoyv12 with interlaced=true but I don't know if this cam be useful.
However the fact that Vdub doesn't handle yv12 correctly doesn't affect both options equally or is there one more affected then other.
Well for now I only want to identify between the two options if there are differences if you have any idea for this I am all ears.Last edited by FLP437; 16th Feb 2017 at 09:22.
-
Well I have not been scaling images before posting I do have a 4k monitor and the images were obtained with a grab screen utility as I had two instances of vdub running two different scripts the utility can grab also in png.I will try the converto rgb only to control image on monitor not for saving ,thanks fot the idea.
I will try also plan levels as suggested by poisondeathrayLast edited by FLP437; 16th Feb 2017 at 10:37.
-
-
If you were using SmoothLevels(16, 1.0, 255, 16, 235) , the equivalent would be Levels(16, 1.0, 255, 16, 235, dither=true, coring=false) . The latter won't be identical but it will also dither the results, smoothing any banding introduced (and in the waveform/ histogram) . The main difference is it's ~3x faster
-
They have been cropped as I said before so less then 720x576 and the pixels dimensions is the result of the 4k monitor
Last edited by FLP437; 16th Feb 2017 at 10:58.
Similar Threads
-
Restoring Damaged .AVI files
By ReggieB in forum RestorationReplies: 12Last Post: 1st Oct 2016, 18:20 -
How Can I Achieve High Quality Encodes Considering These Source Files?
By trueblueCj in forum Video ConversionReplies: 9Last Post: 21st Mar 2016, 12:18 -
Down-converting, decompressing before editing, and cooperative file types.
By Jimmyfro5 in forum EditingReplies: 3Last Post: 11th Dec 2015, 09:34 -
Help with restoring files folders software!
By KaddirB in forum ComputerReplies: 11Last Post: 3rd Jun 2014, 02:55 -
BD Rebuilder to ipad results in extremely large files
By gijoker in forum Blu-ray RippingReplies: 5Last Post: 14th Feb 2013, 18:58