VideoHelp Forum
+ Reply to Thread
Page 1 of 2
1 2 LastLast
Results 1 to 30 of 40
Thread
  1. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    I intend to begin restoring dv files already transferred using WinDv/ Sceneanalyzer. However I have doubts related to which will be the best method to open the files to get the best results just before applying the restore filters. All of them are DV Pal files probably recorded using sony dv codec (from Sony camcorders).

    Depending of the files I could begin with an avisynth script or open them previously on premiere or even use virtualdub or a mixed approach.
    I intend to be able to force, if I want to, the use of a particular dv decoder letīs say mainconcept, cedocida, Panasonic, Microsoft or any other that could prove useful ( to confirm if any of them could provide better results ).

    However I donīt see in avisynth how can I force the use of a specific dv decoder.If I do open the files in avisynth with avisource I can force the use of a decoder with a specific FourCC , however in this moment I do have several dv codecs installed in my system ( Cedocida, Mainconcept, Microsoft ) but GSpot gives for all of them the same FourCC – DVSD so I cannot choose specifically one ( theoretically mainconcept DV codec should have used a different fourCC =DVCS but if I use it avisource doesn’t work).

    Using avisource without specifying pixel_type will give by default YUY2 colorspace the same as explicitly using YUY2 ,trying to specify YV12 doesn’t work using pixel_type RGB also does work however I donīt know for sure which dv decoder is being used ( eventually cedocida as is assigned in the registry) but I donīt know for sure and if there is a way to select the codec I could eventually prefer.

    If I do open the dv file with ffvideosource I do get by default YV12 ( which could eventually be better as the Pal original colorspace is 4.2.0 so apparently no colorspace conversion and it seems to be faster than avisource perhaps because is multithreading) but again I also donīt know which decoder is being used and if there is a way to change it if I want to.

    Opening the file in virtualdub itīs in my case the only situation where is clear that itīs cedocida the codec being used , however I cannot use other codecs namely for instance mainconcept as it is a directshow filter.

    Looking in the registry the active assigned dvsd dll codec is cedocida so I presume that if I use premiere to correct color and exposition in first place for instance is cedocida that is used but again Iīm not sure as I have read that premiere does include the mainconcept dv codec.

    The only pratical way I see to force the use of a specific dv decoder is using a tool like graphedit/graphstudio but in this case I will have to use DirectShowSource with the GRF file in an avisynth script or transcode to an intermediary lossless format as utvideo in YUV or RGB depending of the next restore tool to be used (virtualdub/avisynth/Premiere.

    So my problems

    1- Identify using , avisynth, or premiere which is the dv codec used and after testing and deciding which one to retain ( probably mainconcept or cedocida) being able to force the use of the selected dv codec in the specific required restore application.

    2- For what I have read it seems mainconcept or cedocida could be the best dv codecs, not sure if there is new info related to this subject debated is the past

    3- Decide between using avisource or ffvideosource , this last seems to provide the original colorspace YV12 so eventual preventing one colorspace conversion and seems to be fast

    4- Eventual benefit of using a tool as graphedit to assure the use of any specific dv codec vfw or directshow and use Directshowsource with the GRF file in the avisynth restore script or use an intermediary lossless format like utvideo to do all the subsequent restoration

    I hope someone can shed some light on this subject and elaborate on the pros and cons of the several options
    Image Attached Files
    Quote Quote  
  2. aBigMeanie aedipuss's Avatar
    Join Date
    Oct 2005
    Location
    666th portal
    Search Comp PM
    it's crappy old standard def DV. open in your editor of choice, preferably one like vegas that works well with native DV, edit and render to whatever your final destination is. it's not rocket science it's an old easily used format.
    --
    "a lot of people are better dead" - prisoner KSC2-303
    Quote Quote  
  3. It doesn't matter what DV codec gets used for decoding because DV will always decode to the same exact thing no matter which codec does the decoding. However, it makes a HUGE difference which one you use when encoding. The Windows DV codec used to be absolutely horrible (and probably still is, if it even gets included anymore). The Vegas DV codec is the gold standard, and the MainConcept one isn't bad.

    In almost every application I have, you get to choose which one you want. Unfortunately, Vegas does not register its DV codec in such a way that it can be used outside of Vegas. Thus, if you truly want to use that from within another application, you have to use frameserving, where you serve into Vegas from that other application and then let Vegas encode using its built-in DV codec.

    I found that the MainConcept DV codec is almost as good as the Vegas codec (neither one should give you noticeable degradation if you only do one generation of encoding). So that's what I use when I want to encode DV video from another application (like VirtualDub). I still do a lot of "crappy old standard def" work, so I am very familiar with DV, the various DV codecs, and DV's limitations. It sure is easy to use, and fun to edit with (no pauses or delays, etc.). It was the codec that enabled the digital video revolution.
    Quote Quote  
  4. Originally Posted by johnmeyer View Post
    It doesn't matter what DV codec gets used for decoding because DV will always decode to the same exact thing no matter which codec does the decoding.
    This isn't quite true. They may all decode the same internal 4:1:1 but some will output RGB (4:4:4), others YUY2 (4:2:2), or others YV12 (4:2:0). Sometimes without any choice, sometimes depending on configuration. For example, Panasonic DV codec always outputs RGB. Cedocida lets you configure it. It even allows for full range vs. limited range, and DV vs. Mpeg2 interlaced and MPEG2 non-interlaced chroma placement for YV12. Of the three formats I listed, YUY2 (YUV 4:2:2) is closest to NTSC DV's 4:1:1 chroma subsampling. For PAL DVD, YV12 is closest.
    Quote Quote  
  5. Originally Posted by jagabo View Post
    Originally Posted by johnmeyer View Post
    It doesn't matter what DV codec gets used for decoding because DV will always decode to the same exact thing no matter which codec does the decoding.
    This isn't quite true. They may all decode the same internal 4:1:1 but some will output RGB (4:4:4), others YUY2 (4:2:2), or others YV12 (4:2:0). Sometimes without any choice, sometimes depending on configuration. For example, Panasonic DV codec always outputs RGB. Cedocida lets you configure it. It even allows for full range vs. limited range, and DV vs. Mpeg2 interlaced and MPEG2 non-interlaced chroma placement for YV12. Of the three formats I listed, YUY2 (YUV 4:2:2) is closest to NTSC DV's 4:1:1 chroma subsampling. For PAL DVD, YV12 is closest.
    I stand corrected. I didn't realize that some performed colorspace conversion.

    If what you say is true, then someone needs to answer the OP's question as to how you can force a program to use a particular DV codec for decoding, if more than one is installed. I seem to remember a way to do this, fifteen years ago when DV was king, and the DV codec built into Windows was to be avoided at all costs. I can't remember now how it was done, and I think it was some sort of Windows 98 hack, probably not available in more modern Windows versions.
    Last edited by johnmeyer; 13th Feb 2017 at 23:55. Reason: added last paragraph
    Quote Quote  
  6. A filter manager like FilMerit, DSFMgr, or Radlight Filter Manager should allow the OP to force a decoder by specifying its merit (highest merit is used by VFW or DirectShow.
    Quote Quote  
  7. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    Thanks for all contributions. A special thank you to jagabo for the info related with filter merit management which should allow me to test the use of specific codecs. I will try also a small utility “ video codec swap ” that does allow to bounce between codecs.

    Theoretical the best dv codec for me would be the Sony DV codec as the miniDv have been hard encoded using this codec in a Sony camcorder. It seems also itīs the codec used by Vegas. However after “googling” a lot I have been unable so far to find it, seems to be hidden in third party applications and also probably a vfw version doesn’t exist.

    DV codecs even for decompression have issues as colorspaces problems signaled by jagabo. There are also cases in literature where bad Chroma up sampling and dv decoding problems does arise.

    It seems also that most if not all miniDv camcorders can output luma in between 16-255 that could also pose a problem namely clipping the 235-255 portion if an application is not aware or if an adjustment is not previous made. In the example I include it seems that this is true and doing a levels adjustment with ” levels or smooth levels” to get 16-235 could improve the results and recover some detail otherwise lost in the whites/super whites ( see the clouds in the example included).I used levels perhaps smooth levels can provide a more smooth histogram without spikes.

    Cedocida seems to work fine and with the configuration panel adjusted itīs able to output YV12 ( more closed related with PAL 4.2.0 ) without color shift, interpolation or conversion when using YV12 / DV options.Having high merit, being correctly installed and assigned by default in registry and being a vfw codec assure itīs use by default in my case by virtualdub and also by Avisynth/avisource as I think avisource doesnīt support directshow ( also the case of the mainconcept codec) and Microsoft itīs not assigned as default in registry.

    FFmpeg DV decoder (ffvideosource) is however amazing fast and does provides also YV12 and comparing with cedocida using MSU Video Quality Measurement Tool does seems to provide a small benefit to ffmpeg ( better signal to noise ratio and less blurriness ). Tested dv files opened with avisource vs ffvideosource and saving in both cases to lossless utvideo 4.2.0 and comparing final results with MSU.

    I donīt know if anyone as compared FFmpeg dv codec with the traditional dv codecs , I have searched but have not found anything however it does seems to provide a benefit.

    I made also some tests using graphstudio using mainconcept and cedocida and converting to final lossless utvideo 4.2.0 the results have been always worse that opening directly in vdub or through avisynth however between the two graphstudio pipelines cedocida vs mainconcept there is a marginal advantage to cedocida (even if is a vfw and the pipeline has to have an extra lav decoder and dv decoder).

    Still have to see if I can find a Sony DV codec but I donīt think it exist a vfw version even if a directshow version exist I will probably have to use directshowsource (something for mainconcept) what I think itīs less reliable.

    Conclusions so far

    - Cedocida is fine outputs to YV12 and does that without color shift, interpolation or conversion with the correct YV12/DV options
    - FFmpeg (ffvideosource) seems marginaly better and itīs really fast
    - DV luma seems to be in between 16-255 and seems to need levels adjustment ( levels or perhaps preferably smoothlevels for instance)

    Edit- I tested smoothlevels and is the correct levels adjustment to use , the histogram is fine no spikes and better final result

    Note- Not related with this issue
    Due to several crashes using MTmode with avisynthMT 32bits and virtualdub 32 bits and as I thought most of the problems are memory conflicts or lack of addressable memory I decided to make the vdub app “large address aware” ( 32bit app in 64 bit OS can only address 2GB if they are made large address aware they can address 4GB) and see it could give a benefit, I used a small utility named “4GB_Patch” applied to the two Virtualdub .exe and Vdub.exe, setMemoryMax to about 1600 in the scripts and now virtualdub almost never crash with MTmode2 and even 1 (not always) and itīs a lot more stable in MTmode.

    Fig1- DV luma 16-255 raw left - corrected right
    Fig2- avsource_cedocida_left vs FFmpeg right
    Image Attached Thumbnails Click image for larger version

Name:	DV luma 16-255 raw left - corrected right.jpg
Views:	513
Size:	2.21 MB
ID:	40565  

    Click image for larger version

Name:	avsource_cedocida_left vs FFmpeg right.jpg
Views:	428
Size:	2.29 MB
ID:	40566  

    Last edited by FLP437; 15th Feb 2017 at 22:15.
    Quote Quote  
  8. The difference in decoders is interesting.

    Theoretically, cedocida (if forced or set to YV12 for PAL DV) and ffmpeg should give identical results. But in your screenshot I can already see vertical artifacts . I'm wondering if something else is going on, for example different resizer for the screenshot.

    How did you decide on decoding the "original" for the MSU tests ? Because that could skew your results if there was indeed a difference between decoders
    Quote Quote  
  9. As I said in my first post above, the Sony DV codec is not made available outside of Vegas, so you cannot use it in other applications. However, as I also said in that post, you can frameserve into Vegas and then use Vegas as an encoder.
    Quote Quote  
  10. I just tested some random PAL DV clip and when setting the same chroma siting (DV) and forcing YV12, there are tiny microscopic differences that can be detected with amplifed difference testing (not as large as in your screenshot - you can't see them with naked eye, even with zoom on frame by frame). But it's debatable (at least on this clip) which is "better" . But still interesting that there are mathematical differences
    Quote Quote  
  11. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    @johnmeyer
    I have only one slight problem, I donīt have Vegas, I do have Premiere. If I have had vegas I would have already tried it, but thanks anyway

    @poisondeathray
    The difference seems minimal but even so!.. I have used MSU because I didn't saw any difference with naked eye.

    I used a small script to load the dv file and used alternately avisource or ffvideosource selected 4.2.0 yv12 and utvideo YUV420 BT.601 VCM in virtualdub and made two lossless utvideo files I tested with MSU. the FFmpeg almost doubles the speed of avisource ( 220 fps vs 120 fps)


    Code:
    #AviSource("D:\DV\base.avi", fourCC="DVSD", pixel_type="YV12") 
    ffvideosource("D:\DV\base.avi")
    SmoothLevels(16, 1.0, 255, 16, 235)
    In the previous results I think I have not used smoothlevels, perhaps levels or nothing at all, I repeated the tests with smoothlevels now which provides a very smooth histogram without spikes and the same small differences still persist.
    Image Attached Thumbnails Click image for larger version

Name:	screen_2017-02-16 03.42.33.jpg
Views:	322
Size:	749.8 KB
ID:	40568  

    Click image for larger version

Name:	screen_2017-02-16 03.43.21.jpg
Views:	309
Size:	677.3 KB
ID:	40569  

    Quote Quote  
  12. Originally Posted by FLP437 View Post
    @poisondeathray
    The difference seems minimal but even so!.. I have used MSU because I didn't saw any difference with naked eye.
    No, there are significant differences in your screenshot that you can see with human eye. For example just pay attention to the tower. There are vertical bar artifacts present on the left. That makes me thing differences in processing other than decoder



    I used a small script to load the dv file and used alternately avisource or ffvideosource selected 4.2.0 yv12 and utvideo YUV420 BT.601 VCM in virtualdub and made two lossless utvideo files I tested with MSU. the FFmpeg almost doubles the speed of avisource ( 220 fps vs 120 fps)


    Code:
    #AviSource("D:\DV\base.avi", fourCC="DVSD", pixel_type="YV12") 
    ffvideosource("D:\DV\base.avi")
    SmoothLevels(16, 1.0, 255, 16, 235)
    In the previous results I think I have not used smoothlevels, perhaps levels or nothing at all, I repeated the tests with smoothlevels now which provides a very smooth histogram without spikes and the same small differences still persist.

    But you're testing against UT Video, now defined as the "original" for testing purposes. How did you make the UT Video version ? Whatever decoder you used for that will score higher

    For example ,

    1) DV => cecocida => UT
    2) DV => ffvideosource => UT

    cecocida will score higher in (1), but ffmpeg will score higher in (2) , no matter what metric you use



    Smoothlevels just dithers the result (adds noise) . You can do it with plain levels() too (dither=true)
    Quote Quote  
  13. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    well eventually there is an incorrection as avisource could be using OpenDML or AVIFile interface however I tried to force it to use a specific decoder and as cedocida is the only fourCC="DVSD" registred I assumed it will be using cedocida. Anyway I tried to reflect the usual situation where I will be using an avisynth script with avisource or ffvideosource. If I open the dv file directly in virtualdub and save to utvideo there are no differences whatever.

    so for me it was
    dv->avisource (using cedocida)->UT
    dv->ffvideosource->UT
    Quote Quote  
  14. I agree there's a visible difference between the left and right images. But it's the scaling. The left is point resized, the other is probably bilinear.
    Quote Quote  
  15. If you have internal DV codecs disabled in vdub, it will use the VFW codec with the highest merit when you open a file natively (not through avs). (options => preferences => prefer internal video codecs over installed third-party DV and MJPEG) is DISABLED

    Then with an AVI loaded directly, File => file information will report which VFW decoder is being used by vdub. That will also be the same one used with AVISource()




    But do you understand the MSU test isn't useful ? You don't have the "true" original, before the DV compression. You only have the original DV recording from the camera (not the raw sensor data prior to DV compression) .

    So you can't use MSU to test DV decoding properly, because you're 1 generation removed. Whatever decoder you choose to decode the DV recording to make the UT video version , will score higher . (The original DV has to be DEcoded with something for you to test) . So MSU testing set up like that doesn't provide any useful information



    But I can confirm there are microscopic differences between the 2 decoders, just not to the magnitude depicted in the screenshot.
    Quote Quote  
  16. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    Well I can confirm that virtualdub is using Cedocida as I have made the modification indicated by you when I installed cedocida and I can also see in file information that cedocida is the codec used. So that confirms that avisource is using cedocida to decode as I already presumed.

    I understand iīm subverting somewhat the MSU use as Iīm not comparing against the original but I sought it could be also used to compare two second generation not against the original but between them. If I change the relative position original vs processed it gives exactly the same thing the ffmpeg marginally better itīs not the original that has the largest values.

    Anyway you can confirm that the ffmpeg approach is "microscopical" better is that so ? and is also fast
    Last edited by FLP437; 16th Feb 2017 at 01:08.
    Quote Quote  
  17. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    @jagabo
    is it possible that one image is point resized and the other probably bilinear without making any resize or scaling only using the indicated script ?
    Quote Quote  
  18. Originally Posted by FLP437 View Post

    Anyway you can confirm that the ffmpeg approach is "microscopical" better is that so ? and is also fast

    No - you can't say anything about "better" or "worse" in terms of quality or any related metric using that test with any validity (or any MSU test, or any metric , the way you have structured it) , because you don't have an objective "original" to test it against. You're choosing the decoder to decode the "original" , so that skews the results (bias). If the decoders didn't give different outputs, then yes, it would be a valid test. But then you wouldn't need to do this test at all because everything would be identical.

    You can only say that there are differences between each other, when compared against each other



    I understand iīm subverting somewhat the MSU use as Iīm not comparing against the original but I sought it could be also used to compare two second generation not against the original but between them. If I change the relative position original vs processed it gives exactly the same thing the ffmpeg marginally better itīs not the original that has the largest values.
    Yes you can compare second generation encodes, compared to the original. But that original has to be decoded in the same fashion. Otherwise encode A, and B would be getting different inputs. Not valid.



    yes, ffms2 and lsmash get 3-4x faster. (I'm getting about 1000fps for just decoding speed). And lsmash and ffms2 are both libav based so they produce identical outputs (confirmed).
    Quote Quote  
  19. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    @poisondeathray
    ok but in practical terms and according with some of the highlighted information and all your background what would you choose if you have to between these two options, avisource using cedocida or ffvideosource and why if possible ?
    Quote Quote  
  20. Originally Posted by FLP437 View Post
    @poisondeathray
    ok but in practical terms and according with some of the highlighted information and all your background what would you choose if you have to between these two options, avisource using cedocida or ffvideosource and why if possible ?
    It probably doesn't matter, quality wise

    I don't like the clutter with indexes with ffms2 and lsmash have with AVI (If you have a lot of footage the extra time indexing and extra clutter can become large). And they are faster because they are internally threaded even with vanilla avisynth. When you set threads=1 in the source filter, they become slower too. Some filter operations are unstable with ffms2 and lsmash with threads > 1 in the source filter

    If you're doing other processing , the fps is probably going to be a moot point . When you decode at 1000 fps, the CPU usage is higher too. What you want to test is actual end to end workflow . Decoding fps is only 1 part of it

    Avisynth mt can be unstable as you've seen . It really depends on the operations being done. Also have a look vapoursynth . It's dramatically improved the last few years. The threading is much better. Even single instance of QTGMC is a few % faster than avisynth mt now . And it doesn't crash as often. And when you stack a few filters it can be 2-4x faster. Avisynth MT tends to thrash with threading issues resulting in slow fps when more than 1 filter is used (if it doesn't crash in the first place) . Also have a look at Avisynth+ , which can be slightly faster than avisynth MT. The problem with vapoursynth is a more difficult learning curve, more finicky syntax than avisynth, and not all filters are available yet. The vapoursynth editor isn't as good as avspmod either (no tabbed previews)
    Quote Quote  
  21. Originally Posted by FLP437 View Post
    @jagabo
    is it possible that one image is point resized and the other probably bilinear without making any resize or scaling only using the indicated script ?
    The script isn't resizing but the images you posted don't have the videos' frame sizes as 720x576. So they have been resized.
    Quote Quote  
  22. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    also, if you going to want help, you must make it easy for all to assemble. you have jumbled (glued) the images together as one. how can anyone get a accurate measurement out of it if you mesh them together like that? you must post separate images, and at the the same size untouched. if they are screenshots of virtualdub, then you must keep them all equally the same size. if you are posting actual video image, then you must post the image as the one size, i.e., 720x480 or 720x576 or , ... the pairs have to be the same size. don't make the helpers have to do that work. it gets tiresome to do and i don't do it. so please keep that in mind next time you post images.

    if you want to post the images from virtualdub v1.9.11 and greater, then you can simply press ctrl+1 (for the left pane side) and ctrl+2 (for the right pane side). just be sure that under menu/video that you check-mark 'input pain' and 'output pane' so that the ctrl+1 and ctrl+2 work or else you will get no image in the clipboard to paste and save using your graphics editor or ms built-in paint program. and save as PNG only. ctrl+1 is the untouched source video. ctrl+2 is the filtered output video (assuming you added filters in virtualdub).
    Quote Quote  
  23. Some other things to consider:

    Any images you post will have been converted to RGB. That may be a problem depending on what you are trying to show.

    VirtualDub does not handle interlaced YV12 properly. It will convert it to RGB as if it was progressive YV12. That will blur the chroma channels together.
    Quote Quote  
  24. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    @jagabo
    The images have not been resized they have been cropped to get rid of the borders what I think is a passive operation and don't introduce any other modification on the image.
    Quote Quote  
  25. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    Well to present the images on the monitor they are obviously converted to rgb and eventually this could pose a problem. I have verified with info () that it's yv12 interlaced but I can eventually force a converttoyv12 with interlaced=true but I don't know if this cam be useful.
    However the fact that Vdub doesn't handle yv12 correctly doesn't affect both options equally or is there one more affected then other.

    Well for now I only want to identify between the two options if there are differences if you have any idea for this I am all ears.
    Last edited by FLP437; 16th Feb 2017 at 09:22.
    Quote Quote  
  26. Don't scale the images before posting. Post PNG images (lossless) not JPG (lossy). Use ConvertToRGB(interlaced=true) in AviSynth to control the YUV to RGB conversion.
    Quote Quote  
  27. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    Well I have not been scaling images before posting I do have a 4k monitor and the images were obtained with a grab screen utility as I had two instances of vdub running two different scripts the utility can grab also in png.I will try the converto rgb only to control image on monitor not for saving ,thanks fot the idea.
    I will try also plan levels as suggested by poisondeathray
    Last edited by FLP437; 16th Feb 2017 at 10:37.
    Quote Quote  
  28. Originally Posted by FLP437 View Post
    Well I have not been scaling images before posting
    Then why aren't the portions of your images that contain the video not 720x576?
    Quote Quote  
  29. Originally Posted by FLP437 View Post
    I will try also plan levels as suggested by poisondeathray
    If you were using SmoothLevels(16, 1.0, 255, 16, 235) , the equivalent would be Levels(16, 1.0, 255, 16, 235, dither=true, coring=false) . The latter won't be identical but it will also dither the results, smoothing any banding introduced (and in the waveform/ histogram) . The main difference is it's ~3x faster
    Quote Quote  
  30. Member
    Join Date
    Mar 2015
    Location
    Europe
    Search Comp PM
    They have been cropped as I said before so less then 720x576 and the pixels dimensions is the result of the 4k monitor
    Last edited by FLP437; 16th Feb 2017 at 10:58.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!