.mxf (PAL HD422) HD to SD avi

[marcorocchini]

Hi everyone,

I shoot with a Sony PDW-F800 PAL Xdcam camcorder. When the camera write file .mxf and in DVCAM o IMX mode I never had any problem to convert the .mxf file into my old Matrox Digisuite workstation for the editing (with TMPGENC or with Virtualdub)

But the problem is with the .mxf HD files:

this is the report of the mediainfo for the file C0100.MXF that I have to convert in AVI SD:

General
Complete name : V:\automazioneclip\input\C0100.MXF
Format : MXF
Commercial name : XDCAM HD422
Format profile : OP-1a
Format settings : Closed / Complete
File size : 67.4 MiB
Duration : 9s 360ms
Overall bit rate : 60.4 Mbps
Encoded date : 2013-04-21 15:39:28.000
Writing application : SONY Opt 1.22
Writing library : SONY Opt 1.22

Video
ID : 2
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format_Settings_Wrapping : Frame
Duration : 9s 360ms
Bit rate : 50.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Stream size : 55.8 MiB (83%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

It seems to me that the HD-->SD conversion I do with Virtualdub (tmpgenc is bad) resize to 720x576 with interlace, lose definition and color deep.
Unfortunatly I'm not very able to use avisynth but I have heared that avisynth is good.
I have download avisynth and the HD2SD plugin but I'm not able to use it.

I have upload my file (C0100.MXF) on my dropbox: you can download it if you go on www.dropbox.com ad logon into it with this credential:

user: mrocchini@email.it
pass: centrale

Already I have upload another file: outputOK.avi
this file is a SD-SD conversion (.mxf to avi) and represents the correct conversion... but the source is an SD footage.
The codec used for this .avi is a Matrox Digisuite LE codec (that I have put on the dropbox in the file "codec matrox.exe")

Possibly I would to convert the HD file (C0100.MXF) without loss of definition and color depth.
Can you help me with a script of avisynth or in another way?

PS: to read .mxf files on virtualdub I have previously install - in sequence - "OpenCube MXF Reader v.2.1.0" and "MainConcept MXF Decoder v.9.6.8.2473" codecs that I have find to http://codec.kiev.ua/other.html

Thanks and best regards to all

[yoda313]

without loss of definition and color depth.

If you are converting to standard definition from high definition that is by DEFINITION a loss of definition. You are reducing the size of the image.

As for color depth I'm not sure about that. Suffice it to say going from high def to standard def won't look as good as high def no matter what you do. If there are specific color issues others might be able to help.

But basically going from high def to standard def means there will be loss of quality no way around that.

[marcorocchini]

... obviusly. "Definition" =/not resolution
Sorry for my english however for "loss of definition" I mean that the output image I achieve with the standard resize (resize to 720X576 Lanczos3 + interlace option) of Virtualdub does not have well-defined edges and semms to have a color deph reduced compare with the same image shooted in IMX mode (SD 720x576 instead of 1920x1080).

Expecially I try to understand the correct method to do the HD-SD conversion.

[_Al_]

I just tested this, loading one line Avisynth script: ffmpegsource2("C0100.MXF", atrack=1)
into VirtualDub, choosing resize filter , 720x576, Lanczos3 resize and checked interlace box.

File seems to be ok, it is regular SD that needs to be flaged 16:9 during encoding, just be careful it is Upper field first, your previous SD might be Bottom field first

[marcorocchini]

I just tested this, loading one line Avisynth script: ffmpegsource2("C0100.MXF", atrack=1)
into VirtualDub, choosing resize filter , 720x576, Lanczos3 resize and checked interlace box.

File seems to be ok, it is regular SD that needs to be flaged 16:9 during encoding, just be careful it is Upper field first, your previous SD might be Bottom field first

Hi _Al_
I have installed virtualdub in c:\virtualdub

and avisynth in c:\avisynth

so I have to open the notepad and write:

ffmpegsource2("C0100.MXF", atrack=1)

and I save this as c:\script.avs

Then if I open virtualdub I have to do: "File-->open video file--> c:\script.avs" ?
because if I do this I receive the message: "Avisynth open failure: script error: there is no function named "ffmpegsource2"(c:\script.avs, line1)

What I have to do? FFmpegsource is an external plugin for avisynth?

Thank you

[_Al_]

yes, ffmpegsource is Avisynth's plugin, you need to download it from here http://code.google.com/p/ffmpegsource/downloads/detail?name=ffms-2.17.7z unzip and just copy/paste it all in Avisynth/plugin folder

Even if you have windows 64bit, make sure you have 32 bit version of VirtualDub, 32 bit version of Avisynth . That link for ffmpegsource is 32bit also.

[marcorocchini]

yes, ffmpegsource is Avisynth's plugin, you need to download it from here http://code.google.com/p/ffmpegsource/downloads/detail?name=ffms-2.17.7z unzip and just copy/paste it all in Avisynth/plugin folder

Even if you have windows 64bit, make sure you have 32 bit version of VirtualDub, 32 bit version of Avisynth . That link for ffmpegsource is 32bit also.

thank again _Al_
now I can also open .mxf file in virtualdub with FFMPEG libreries, but the problem persist: I my dropbox have put 2 frame for understanding the problem

frameOK.BMP

and

frameWrong.BMP

Frame OK show the encoding with Virtualdub by a file .mxf record with the Sony camera in SD mode (is a .mxf in IMX/D10encoding)

Frame Wrong shown the Virtualdub-encoding of the same inquadrature of the camera but the camera recording the same picture in HD 1920x1080 mode. The result is the HD-->SD conversion is worse of the SD-->SD encoding: if you confrontinf the 2 images and switch by one to one it's clear the frameWrong.bmp has a loss of detail, gloss and depth of color.

Though both are the final product of transcoding with Virtualdub.

I have read what "poisondeathray" wrote in this 3d:

https://forum.videohelp.com/threads/339518-TMPGEnc-for-HD-to-SD-conversion

maybe a good HD to SD conversion requires the "HD2SD" Avisynth plugin but I can not able to configure a script for use it.

Note the "HD2SD" script is different than the vdub lanczos interlace resize method.

You don't have to use avisynth for vdub, but the "interlaced scaling" checkbox has to be checked. You might want to import into vdub using avisynth, because it means you skip the first step of encoding to the digital intermediate or using huge filesize imports. Vdub natively doesn't handle interlaced YV12 ideally. It upsamples as progressive and you get chroma artifacts so if you don't use avisynth to upsample , or upsample using cineform or MPEG 4:2:2 before importing into vdub, you will get problems

My .mxf input file is a 4:2:2 quality and I have have well understand it requires a particular processing, but I don't know how I have to do

[aedipuss]

maybe try using different tools. i took your mxf and ran it through mediacoder to make a huffyuv SD avi. look ok to me.

your HD mxf on the left - SD huffyuv avi on the right

[_Al_]

yes, something else,
about those colors I heard here on this forum that VirtualDub uses BT.601 even for HD source,

about picture not that crisp, well, it is this interlace oddity that broadcasters seem to defend but we amateuers do not. They have some HW boxes that shape their videos as they need, not us. To resize HD interlace to SD interlace again is not easy. You can try QTGMC , do it all in avisynth, I just tried that, but I got error , it works in YV12, but that you do not want to change it to 4:2:0, maybe I cannot set it right or do not understand that:
ffmpegsource2("C0100.MXF", atrack=1)
ConvertToYUY2()#NOT WORKING
QTGMC() #double framerate
Spline64Resize(720,576)
Assumetff()
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace
ColorMatrix(mode="Rec.709->Rec.601")

[marcorocchini]

maybe try using different tools. i took your mxf and ran it through mediacoder to make a huffyuv SD avi. look ok to me.

your HD mxf on the left - SD huffyuv avi on the right


[Attachment 17477 - Click to enlarge]

Hi aedipuss,
Mediacoder is interesting, but extremely complicated aboud some setting: in Tmpgenc the setting for "video ouput codec" is easy: "matrox Mjpeg 25fps interlaced". I have to encode in "Matrox Mjpeg" because I have a Matrox Digisuite LE hardware installed. In a pc that don't have Matrox hardware is possible install the matrox codecs (I have put the codec in the dropbox) so I have to necessarily use the Matrox Mjpeg (.avi).

Also Virtualdub can easly configure the codec.

In mediacoder the are a sea of setting I don't know to set.

Please, can you set the mediacoder in such a way as to obtainthe same setting of the file outputOK.avi there are on my dropbox (this file is Matrox Mjpeg ok)?

thank you!!

[marcorocchini]

ps: the matrox codec is configurable

in panel control, system .. hardware, video codec, proprieties, matrox VFW codecs

[marcorocchini]

General
Complete name : C:\Documents and Settings\Administrator\Desktop\outputOK.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 99.7 MiB
Duration : 10s 960ms
Overall bit rate : 76.3 Mbps
Writing library : TMPGEnc 4.0 XPress Version. 4.7.7.307

Video
ID : 0
Format : JPEG
Codec ID : MJPG
Duration : 10s 960ms
Bit rate : 76.3 Mbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 5:4
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 7.359
Stream size : 99.7 MiB (100%)

[poisondeathray]

yes, something else,
about those colors I heard here on this forum that VirtualDub uses BT.601 even for HD source,

about picture not that crisp, well, it is this interlace oddity that broadcasters seem to defend but we amateuers do not. They have some HW boxes that shape their videos as they need, not us. To resize HD interlace to SD interlace again is not easy. You can try QTGMC , do it all in avisynth, I just tried that, but I got error , it works in YV12, but that you do not want to change it to 4:2:0, maybe I cannot set it right or do not understand that:
ffmpegsource2("C0100.MXF", atrack=1)
ConvertToYUY2()#NOT WORKING
QTGMC() #double framerate
Spline64Resize(720,576)
Assumetff()
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace
ColorMatrix(mode="Rec.709->Rec.601")

You can use QTGMC in YUY2 mode if you swap the masktools for the 2.5 version (mt_masktools-25.dll)

If you use ffms2 (vanilla version) it should output YUY2 ; but the ffms2 cplugin will output YV12 on xdcam-hd sources

I would also use clamp=0 for colormatrix (it's actually clipping, not clamping in colormatrix)

maybe a good HD to SD conversion requires the "HD2SD" Avisynth plugin but I can not able to configure a script for use it.

My .mxf input file is a 4:2:2 quality and I have have well understand it requires a particular processing, but I don't know how I have to do

Yes, HD2SD or some avisynth methods like _Al_'s above will give you better quality results than vdub for sure - What help did you need specifically ?

There are video tutorials that show you step by step how to use HD2SD . If you get stuck, post the error message you get and someone will help you




But there are some "gotchas" to configuring HD2SD properly . eg. Did you want it pillarboxed 720 width? or clamped levels ? or clipped levels? or full levels (passed through) ?

Already I have upload another file: outputOK.avi
this file is a SD-SD conversion (.mxf to avi) and represents the correct conversion... but the source is an SD footage.

Your HD footage has usable data above Y=235 (above 100IRE) , but this outputOK.avi is pillarboxed (black bars) , and has levels clamped . So did you want it like that or full frame ? Your other example screenshots were not pillarboxed. The difference is the method of AR interpretation (ITU vs non ITU)

You can still use vdub to encode using matrox codecs, but it's important to input the avs script and use video=>fast recompress to prevent other color space conversions

Also note, sometimes matrox codecs are not treated as YUV in other applications (sometimes they are treated as RGB in some NLE's - so that colorspace conversion can cause problems such as clipping) . I suspect it should be ok in a Matrox application , however . Because of this concern, it might be better to clamp the levels to 16-235 instead of keeping the full range if you use matrox codecs

[jagabo]

It seems to me that the HD-->SD conversion I do with Virtualdub (tmpgenc is bad) resize to 720x576 with interlace, lose definition and color deep.

Did you deinterlace before scaling then reinterlace? Or at least use the interlace option in the resize filter.

Nevermind, I see you used the interlace option in the resize filter. You could try deinterlacing with yadif2x, downscale, then reinterlace. Also, be sure to force the input colorspace to YUY2, Video -> Color Depth.

Of course, working in AviSynth will be better. You can retain adjust the color matrix there too.

[marcorocchini]

Because of my bad english I recapitulate adding some information:

[jagabo]

The reduction in resolution looks normal for a 1920x1080 to 720x576 frame size reduction. I don't think VirtualDub can help you with the rec.709 vs rec.601 color problem. How does this m2v (mpeg 2 elementary stream) look? It was made with AviSynth and HCGui.

[marcorocchini]

The reduction in resolution looks normal for a 1920x1080 to 720x576 frame size reduction. I don't think VirtualDub can help you with the rec.709 vs rec.601 color problem. How does this m2v (mpeg 2 elementary stream) look? It was made with AviSynth and HCGui.

Thank you, but I'm constrained to Matrox Mjpg encoding. I don't know if HCGui/avisynth can export in Matrox Mjpeg .avi files

[jagabo]

Thank you, but I'm constrained to Matrox Mjpg encoding. I don't know if HCGui/avisynth can export in Matrox Mjpeg .avi files

The m2v sample was provided simply for comparison. Using AviSynth doesn't preclude the use of MJPEG. HcGUI is an MPEG 2 encoder. So you'll need to use another program. VirtualDub will work fine for opening AviSynth scripts and encoding with whatever VFW codec you want. If you only use VirtualDub for encoding it will have no effect on the video passing through.

[poisondeathray]

Maybe I think only avisynth can do the correct conversion but I don't know avisynth

It's actually pretty simple for this conversion. You basically copy & paste a script.

Look at the instruction video for HD2SD (follow along step by step)
http://bellunevideo.com/tutdetail.php?tutid=12

You didn't answer the earlier questions I had, so I'm going to assume you want full frame , not pillarboxed, levels passed through

1) Install avisynth and download the HD2SD package. Follow the instructions included with HD2SD for the prerequisite plugins (place plugins and avsi in avisynth plugin directory to autoload them)

2) Download FFMS2 (vanilla version), place the .dll and .avsi in plugins folder

3) Create a text file in notepad , changing the filepaths and names to match your files and folders. Save it. Rename the .txt extension to .avs e.g. (script.txt => script.avs)

FFVideoSource("G:\PATH\HD.MXF")
hd2sd(interlaced=true, OutputColorSpace="yuy2", inputPAR=1, outputPAR=1.4222, limitwhite=false)

You can load script.avs into vdub directly, then use video=>fast recompress , video=>compression (select compression & settings), file=>save as avi

The reason why the outputPAR is 1.4222 is you're using non ITU guidelines . The PAR is 64/45 for 16:9 PAL (so this would be full frame, not pillarboxed as in your other "outputOK.avi" example

Note, by default, hd2sd will CLIP superbrights (by default limitwhite=true) . This means you lose data. Your other examples clamped the range. I didn't do anything else in the script, but you should be aware of this because you might want to make adjustments.




SD.mxf (cropped to 720x576)


HD.mxf => hd2sd

[marcorocchini]

Maybe I think only avisynth can do the correct conversion but I don't know avisynth

It's actually pretty simple for this conversion. You basically copy & paste a script.

Look at the instruction video for HD2SD (follow along step by step)
http://bellunevideo.com/tutdetail.php?tutid=12

You didn't answer the earlier questions I had, so I'm going to assume you want full frame , not pillarboxed, levels passed through

1) Install avisynth and download the HD2SD package. Follow the instructions included with HD2SD for the prerequisite plugins (place plugins and avsi in avisynth plugin directory to autoload them)

2) Download FFMS2 (vanilla version), place the .dll and .avsi in plugins folder

3) Create a text file in notepad , changing the filepaths and names to match your files and folders. Save it. Rename the .txt extension to .avs e.g. (script.txt => script.avs)

FFVideoSource("G:\PATH\HD.MXF")
hd2sd(interlaced=true, OutputColorSpace="yuy2", inputPAR=1, outputPAR=1.4222, limitwhite=false)

You can load script.avs into vdub directly, then use video=>fast recompress , video=>compression (select compression & settings), file=>save as avi

The reason why the outputPAR is 1.4222 is you're using non ITU guidelines . The PAR is 64/45 for 16:9 PAL (so this would be full frame, not pillarboxed as in your other "outputOK.avi" example

Note, by default, hd2sd will CLIP superbrights (by default limitwhite=true) . This means you lose data. Your other examples clamped the range. I didn't do anything else in the script, but you should be aware of this because you might want to make adjustments.

Poisondeathray, with your help the situation seems much improved!

Only 2 things:

1) I get an error when I try to export with compressor "Matrox Mjpeg" (it can export in Uncompressed RGB/YCbCr) (I do use fast recompress, but if I set the full processing mode it's all ok)

2) it seems necessary at least one filter e.g. Lanczos because the contour are raw and too jagged.
Is possibile to add a line in the script of the a good filter (maybe even better of Lanczos3, if exist)?

I heartily thank you

[jagabo]

Add ConvertToRGB() to the end of your script.

[poisondeathray]

1) I get an error when I try to export with compressor "Matrox Mjpeg" (it can export in Uncompressed RGB/YCbCr) (I do use fast recompress, but if I set the full processing mode it's all ok)

It's because the codec you're using is probably expecting RGB , not YUV . I would use another codec .The problems with YUV=> RGB conversion are quality loss , superwhite clipping.

The "fast recompress" mode is to prevent the colorspace conversion (you keep YUV 4:2:2 same as original)

But if you insist converting to RGB, you probably want to fix superbright levels before converting to RGB (otherwise they clip)

Then you would change OutputColorSpace=RGB in hd2sd , instead of YUY2

If you use ConvertToRGB separately, you should use interlaced=true
ConvertToRGB(interlaced=true)

2) it seems necessary at least one filter e.g. Lanczos because the contour are raw and too jagged.
Is possibile to add a line in the script of the a good filter (maybe even better of Lanczos3, if exist)?

I don't understand what you're asking ? Can you clarify ?

Lanczos is for resizing, and it's sharp (it will enhanced edges)

[jagabo]

Oh, I forgot, many MJPEG codecs use video range RGB. So you probably want to use ConvertToRGB(matrix="PC.601").

[marcorocchini]

Hi anyone:

kindly I demand: I would build a script to load in virtualdub in witch avisynth have to do:

1) Deinterlace in double frame rate using bottom flied first and interpolate using Yadif algorithm
2) Lanczosresize(720,576)
3) Interlace as the interlace virtualdub filter with this option: progressive frames and odd field first

Ho can it be traduced in avisynth?

THANKS TO ALL

[poisondeathray]

Hi anyone:

kindly I demand: I would build a script to load in virtualdub in witch avisynth have to do:

1) Deinterlace in double frame rate using bottom flied first and interpolate using Yadif algorithm
2) Lanczosresize(720,576)
3) Interlace as the interlace virtualdub filter with this option: progressive frames and odd field first

Ho can it be traduced in avisynth?

THANKS TO ALL

FFVideoSource("video.MXF")
Yadif(1,1)
LanczosResize(720,576)
Colormatrix(mode="rec.709->rec.601", clamp=0)

AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

This will reinterlace with bottom field first

If you wanted top field first, replace last line with

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[marcorocchini]

[poisondeathray]

Download yadif.dll
http://avisynth.org.ru/yadif/yadif.html

but it's a CPlugin , so you have to load it with LoadCPlugin

LoadCPlugin("PATH\yadif.dll")

FFVideoSource("video.MXF") 
Yadif(1,1) 
LanczosResize(720,576) 
Colormatrix(mode="rec.709->rec.601", clamp=0)  

AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[marcorocchini]

LoadCPlugin("c:\avisynth\plugins\yadif.dll")

FFVideoSource("v:\mxf\hdmovimento.mxf")
Yadif(1,1)
LanczosResize(720,576)
Colormatrix(mode="rec.709->rec.601", clamp=0)

AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[poisondeathray]

You probably different colormatrix plugin version, try clamp=false

I'm using 2.5
http://web.missouri.edu/~kes25c/ColorMatrixv25.zip

[marcorocchini]

Now with this seem works!


LoadCPlugin("c:\avisynth\plugins\yadif.dll")

FFVideoSource("v:\mxf\hdmovimento.mxf")
Yadif(1,1)
LanczosResize(720,576)
ColorMatrix(mode="Rec.709->Rec.601")

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

#thks for the moment!!