.mxf (PAL HD422) HD to SD avi

[marcorocchini]

poisondeathray do you know if exist a way for increase o reduce the amount of "detail" (e.g."crispening") of the lanczos? FFDSHOW have a "treshold" to do this, and instead in avisynth is possibile?

[poisondeathray]

poisondeathray do you know if exist a way for increase o reduce the amount of "detail" (e.g."crispening") of the lanczos? FFDSHOW have a "treshold" to do this, and instead in avisynth is possibile?

Taps=x

eg
LanczosResize(720,576, taps=5)

Alternatively, you can use a different resizing algorithm (either sharper or softer) e.g. BilinearResize() is very soft or BicubicResize() is more neutral , Lanczos4Resize() is very sharp , ringing edges

For interlaced DVD, often too sharp is no good => interline twitter, buzzing edges. Often you need to lowpass with a vertical blur (make blurrier) . IMO, Lanczos is "too sharp" for interlaced DVD

There are special resizing algorithms you can use for content that is prone to aliasing (e.g. checkerboard patterns, etc..) , but they produce softer results (you wouldn't apply filter to entire video, just small segments that are prone)

[jagabo]

Yes, LanczosResize(..., taps=3). Or use a less sharp resizing filter like BicubicResize(). Or some of the other resizing filters:
http://avisynth.org/mediawiki/Resize

[marcorocchini]

oh yes: in fact is 48 hours than I try and try various algorithm.

Average I found GaussResize good, but a little softer

Sorry I have another question: I need to push from the below (of the video-frame) upward of few line: just 2-3 lines.
It's possible this in avisynth?

Thanks!!

[poisondeathray]

Sorry I have another question: I need to push from the below (of the video-frame) upward of few line: just 2-3 lines.
It's possible this in avisynth?

Sorry, I don't understand this question

Do you want to crop 2 pixels of the top , then add a 2 pixel black border on the bottom ?

(It's not possible to do with 3 lines height for interlaced video 4:2:2 , and if it's interlaced 4:2:0 it must be by 4 pixel increments)

[jagabo]

Crop(0,-2,-0,-0) # remove two lines from the top of the frame
AddBorders(0,0,0,2) # add two lines to the bottom of the frame

[marcorocchini]

LoadCPlugin("c:\avisynth\plugins\yadif.dll")

FFVideoSource("v:\mxf\casaHD.mxf")
Yadif(1,1)
AddBorders(0,0,0,1)
GaussResize(720,576,-1,-1,-1.5,0,P=41)
ColorMatrix(mode="Rec.709->Rec.601")

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()



I add the AddBorder before the resize.
This way the two frames (SDtoSD and HDtoSD) are perfectly superposable.
The only difference is the color of HDtoSD is slightly worse, but this result (in avisynth is at the moment the best)

[poisondeathray]

The only difference is the color of HDtoSD is slightly worse, but this result (in avisynth is at the moment the best)

The color should be the same as the HD clip when viewed in HD (Rec709)

It might be the method of your taking screenshots, or a problem with your editing software

[poisondeathray]

Would you agree the colors are very close ?

ImageSource("HDtoSD.bmp")
ConvertToYUY2()
ColorMatrix(mode="rec.709->rec.601", clamp=0)

What you've done is made a mistake somewhere

This takes the HDtoSD RGB image, converts to YUY2 with a Rec601 matrix, then applies the wrong conversion 709->601 .

Probably the method you are using to take screenshots or viewing is incorrect




It might be your old matrox software isn't handling the colors correctly . XDCAM-HD is supposed to use Rec709 (All HD uses 709) , the metadata even says so

Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

I verified your outdoor shot in pro software and it is true.

When you convert to SD or have anything is SD, typically you use Rec601. Otherwise your DVD colors will not look correct






Do you know how to use ffmpeg (commandline) to cut samples from mxf ?

eg. This will cut 1 second sample (just for color verification purposes) . If you can cut a SD mxf sample, and a HD mxf sample from that indoor shot, we can verify

 

ffmpeg -i input.mxf -vcodec copy -an -t 00:00:01 output.mxf

You can upload samples directly to this website instead of going through dropbox

[jagabo]

SDtoSD is oversharpened too.

[poisondeathray]

SDtoSD is oversharpened too.

Yes, I agree

I think that's the native SD recording , but then put through some processing to Matrox MJPEG AVI intermediate . It's not clear whether the camera is oversharpening or some other processing he did oversharpened it

[marcorocchini]

Here I can not upload .mxf: he says "invalid format" however I have upload on my dropbox - in the folder called "3" - another 2 short clip recorded with the Sony pdw-f800 camera.

The clip SD.MXF is a IMX[D10] video format 720x608 and need to cut the timecode information with a filter "null trasform" in virtuadub (I attach it on the dropbox is called imx.vcf) that cut (crop) 32 line Y1 to achieve a pure 720x576 video windows

The clip HD.MXF is recorder immediately after, and is shoot keepeng the camera exactly in the same position.
In the hd format, the camera record .mxf hd file @ 1920x1080

[SD.mxf] In virtual dub I do file --> "open video file" "... sd.mxf" -->I apply the filter and I have save the .avi file in Uncompressed

this uncompressed file is on the dropbox with the name: tmpSD.AVI


[HD.mxf] In virtual dub I do file --> "open video file "gatto.avs"

#gatto.avs
LoadCPlugin("c:\avisynth\plugins\yadif.dll")

FFVideoSource("v:\automazioneclip\input\hd.mxf")
Yadif(1,1)
AddBorders(0,0,0,1)
GaussResize(720,576,-1,-1,-1.5,0,P=41)
ColorMatrix(mode="Rec.709->Rec.601")

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

#######################################


and I have saved the .avi file in Uncompressed

this uncompressed file is on the dropbox with the name: tmpHD.AVI


General
Complete name : V:\automazioneclip\input\SD.MXF
Format : MXF
Commercial name : IMX 50
Format profile : OP-1a
Format settings : Closed / Complete
File size : 35.8 MiB
Duration : 4s 800ms
Overall bit rate : 62.6 Mbps
Encoded date : 2013-04-24 12:19:02.000
Writing application : SONY Opt 1.41
Writing library : SONY Opt 1.41

Video
ID : 2
Format : MPEG Video
Commercial name : IMX 50
Format version : Version 2
Format profile : 4:2:2@Main
Format settings, BVOP : No
Format settings, Matrix : Default
Format settings, GOP : N=1
Format_Settings_Wrapping : Frame (D-10)
Duration : 4s 800ms
Bit rate : 47.8 Mbps
Maximum bit rate : 50.0 Mbps
Width : 720 pixels
Height : 576 pixels
Original height : 608 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 4.611
Stream size : 54.5 MiB

Audio
ID : 3
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (D-10)
Duration : 4s 800ms
Bit rate mode : Constant
Bit rate : 12.3 Mbps
Channel(s) : 8 channels
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 7.03 MiB (20%)

Other #1
ID : 1
Type : Time code
Format : MXF TC
Time code of first frame : 00:51:38:06
Time code settings : Striped

Other #2
Type : Time code
Format : SMPTE TC
Muxing mode : SDTI
Time code of first frame : 00:51:38:06


General
Complete name : V:\automazioneclip\input\HD.MXF
Format : MXF
Commercial name : XDCAM HD422
Format profile : OP-1a
Format settings : Closed / Complete
File size : 15.8 MiB
Duration : 2s 160ms
Overall bit rate : 61.3 Mbps
Encoded date : 2013-04-24 12:18:51.000
Writing application : SONY Opt 1.22
Writing library : SONY Opt 1.22

Video
ID : 2
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format_Settings_Wrapping : Frame
Duration : 2s 160ms
Bit rate : 50.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Stream size : 12.9 MiB (82%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio #1
ID : 3
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #2
ID : 4
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #3
ID : 5
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #4
ID : 6
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #5
ID : 7
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #6
ID : 8
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #7
ID : 9
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Audio #8
ID : 10
Format : PCM
Format settings, Endianness : Little
Format_Settings_Wrapping : Frame (AES)
Duration : 2s 160ms
Bit rate mode : Constant
Bit rate : 1 152 Kbps
Channel(s) : 1 channel
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 304 KiB (2%)

Other #1
ID : 1
Type : Time code
Format : MXF TC
Time code of first frame : 00:51:36:02
Time code settings : Striped

Other #2
Type : Time code
Format : SMPTE TC
Muxing mode : SDTI
Time code of first frame : 00:51:36:02





The problem of the color in clip tmpHD.avi is present also if I try to scaling HD-->SD with set of filter of Virtualdub, and also with the HD2SD plugin of Avisynth, and also with Avisynth.

In this procedure I have not used the matrox codecs, seems to me is a problem irresolvible

[jagabo]

Here I can not upload .mxf: he says "invalid format"

You can always ZIP the file and upload the ZIP archive.

seems to me is a problem irresolvible

It's easily resolvable: ColorMatrix(mode="Rec.709->Rec.601") in AviSynth.

[poisondeathray]

I get the same color, when viewing SD.mxf (viewing with 601 matrix to convert to RGB) , or with HD.mxf with the HD=>SD script you are using (viewing with 601 matrix to convert to RGB)

I did not find tmpHD.avi or tmpSD.avi on dropbox

How are you taking the screenshots?




BTW, I'm doing this in avisynth . It might be your mxf or installed codecs causing the difference (since you are opening the SD in vdub directly), or your method of taking screenshots

ie. FFVideoSource for both HD and SD mxf ( you can crop the SD in avisynth) . Both look similar in terms of color after HD is treated with the script

[marcorocchini]

I did not find tmpHD.avi or tmpSD.avi on dropbox

sorry: it's uploaded completed in 1 hour

I consider my broadcast CRT tv monitor as reference to watch the pictures and the matrox .avi

The strange thing is that from SD file I see all well on the CRT monitor: in virtualdub I can achieve .avi from Matrox Jpeg compressor or .avi as Uncompressed

In this case for booth the input-files I achieve uncompressed .avi file that I place on the timeline of my speed razor (or adobe premiere, it's the same results) then I compare the 2 clips switching alternatively.

In this case I have not properly encode via the "matrox mjpeg codec" because I have use the uncompressed option of virtualdub as .avi output.

The .bmp frame "tmpSD.BMP" and "tmpHD.BMP" I got them by esporting-frame-option of adobe premiere 6.5 (I know i older... but for the SD is suitable) however the file tmpSD.AVI I see well both on display monitor and in CRT monitor connected to the Matrox Digisuite card.

The file tmpHD.AVI (the conversion HD-->SD) have wrong colors: almost seems don't have the same color depht. This fact is also clearly visible in the display pc monitor, not only on the crt monitor.

At the best of my knowledge I think the SD-->SD conversion is ok, while the HD-->SD have many problems.

However I do a script avisynth that crop the SD in avisynth so is possible to use the same FFVideoSource for both the files.

If I load a file in Virtualdub I think Virtualdub use the "Mainconcept MXF codec" previusly installed as directshow filter.

However I believe that the problem is not them.

[poisondeathray]

I'm certain that the screenshot you took for the SD file is technically incorrect. But whether or not you "Like" the colors or not , is a different discussion

The native SD file used Rec709 for the conversion to RGB for that screenshot (wrong matrix) . It's supposed to look like the other picture

If you want to prove it to yourself:

FFVideoSource("SD.mxf")
ConvertToRGB(matrix="rec709", interlaced=true)

or

FFVideoSource("SD.mxf")
ConvertToRGB(matrix="rec601", interlaced=true)


If you view the HD file with Rec709 , it looks right, matching the SD file viewed with Rec601 .

In both cases, FFVideoSource returns YUV, not RGB until it's converted to RGB . It might be that you are incurring an RGB conversion somewhere in your workflow unaccounted for


The problem with using the wrong matrix - everything you produce will look different to everyone else. The DVD won't look like you think it does.

[marcorocchini]

now I see using the Mainconcept directshow codec, and the FFvideosouce of Avisynth produce 2 different results in color

The file-input is the same: SD.mxf

However, by eye, I see the colors better MainConcept but only on certain areas of the image

[poisondeathray]

now I see using the Mainconcept directshow codec, and the FFvideosouce of Avisynth produce 2 different results in color

The file-input is the same: SD.mxf

However, by eye, I see the colors better MainConcept but only on certain areas of the image

As I said earlier, whether or not you "like" it or not is a different topic . You can make the file "look" however you want.

If you open the native files in Sony XDCam Viewer or Sony Vegas (your camera is made by Sony), or newer version of Adobe - it will be as I described. Rec709 for HD, Rec601 for SD - these are the standards for production developed by ITU . The reason these are used so color can be reproduced correctly for broadcast and all consumer equipment.

[marcorocchini]

FFVideoSource("SD.mxf")
ConvertToRGB(matrix="rec709", interlaced=true)

or

FFVideoSource("SD.mxf")
ConvertToRGB(matrix="rec601", interlaced=true)

YES: you have reason!

Mainconcept MXF reader=rec601 colors
matrix="rec709"=...matrix="rec709" :=)

However it semms, by eye, the rec709 table have less intense colors and slightly more cold

while rec601 have brighter color and warm colors more emphasized

Poisondeathray you say the rec709 is technically right?

[poisondeathray]

Poisondeathray you say the rec709 is technically right?

709 is correct for HD , when doing YUV<=>RGB conversions

601 is correct for SD, when doing YUV<=>RGB conversions


This is why the "colormatrix" filter is used, when converting your HD footage to SD . It changes the values in YUV (no RGB conversion is incurred) , so it is "as if" the file used 601 in the first place

[marcorocchini]

now I see using the Mainconcept directshow codec, and the FFvideosouce of Avisynth produce 2 different results in color

The file-input is the same: SD.mxf

However, by eye, I see the colors better MainConcept but only on certain areas of the image

As I said earlier, whether or not you "like" it or not is a different topic . You can make the file "look" however you want.

If you open the native files in Sony XDCam Viewer or Sony Vegas (your camera is made by Sony), or newer version of Adobe - it will be as I described. Rec709 for HD, Rec601 for SD - these are the standards for production developed by ITU . The reason these are used so color can be reproduced correctly for broadcast and all consumer equipment.

So the table to treat SD files is 601 and 709 for HD although visually are slightly different?

[poisondeathray]

So the table to treat SD files is 601 and 709 for HD although visually are slightly different?

Visually, the colors will be almost identical.

ie. SD file viewed with 601 matrix will look almost identical to HD file viewed with 709 matrix.

The "matrix" specifies the calculations used when converting YUV<=>RGB. Whenever you look at monitor or TV, that's expressed in RGB , not YUV .

It's only when you use the wrong matrix, or you inadvertently incurred a YUV<=>RGB conversion somewhere in your workflow that the colors will be messed up.

Guess what? When you use HD e.g. blu-ray , it uses 709 matrix to convert to RGB for display . DVD uses 601. Both will look the same (or very similar in terms of color) if you did it correctly . But if you make a mistake, the colors will be off

[marcorocchini]

So the table to treat SD files is 601 and 709 for HD although visually are slightly different?

Visually, the colors will be almost identical.

ie. SD file viewed with 601 matrix will look almost identical to HD file viewed with 709 matrix.

The "matrix" specifies the calculations used when converting YUV<=>RGB. Whenever you look at monitor or TV, that's expressed in RGB , not YUV .

It's only when you use the wrong matrix, or you inadvertently incurred a YUV<=>RGB conversion somewhere in your workflow that the colors will be messed up.

Guess what? When you use HD e.g. blu-ray , it uses 709 matrix to convert to RGB for display . DVD uses 601. Both will look the same (or very similar in terms of color) if you did it correctly . But if you make a mistake, the colors will be off

Ah, so the correct parameter for the SD .mxf file-input is

ConvertToRGB(matrix="rec709", interlaced=true)

?

[marcorocchini]

and if, for iphotesys I would to apply the standard 601 colors to an HD file, it's possible?

[poisondeathray]

Ah, so the correct parameter for the SD .mxf file-input is

ConvertToRGB(matrix="rec709", interlaced=true)

?

It should be rec601

If the input is YUV, and if you want it to look like the screenshot that you "liked" better, then use 709, but it's technically the "wrong" matrix

You lose quality with colorspace conversions, and converting to RGB with standard matrix can clip you data (I mentioned this earlier) . The more colorspace conversions, the more loss accumulates

For best practices, technically it's better to stay in YUV 4:2:2 (same as source) all the way through at least until the very end

[poisondeathray]

and if, for iphotesys I would to apply the standard 601 colors to an HD file, it's possible?

You can do whatever you want , color correct or grade however you want

But the "correct" matrix is 709 for HD

IMO, it's better idea to follow standard production guidelines. You can grade it later however you want

[Cornucopia]

(Haven't understood whether this was already bypassed or not):
Wouldn't it be likely that applying the virtualdub crop filter is also converting the YUV to RGB (and maybe incorrectly)?
If the crop for the SD IMX were done prior to Vdub, in AVISynth, one could also apply the correct ColorMatrix conversion code at that point, and maintain truer color quality.

Scott

[marcorocchini]

(Haven't understood whether this was already bypassed or not):
Wouldn't it be likely that applying the virtualdub crop filter is also converting the YUV to RGB (and maybe incorrectly)?
If the crop for the SD IMX were done prior to Vdub, in AVISynth, one could also apply the correct ColorMatrix conversion code at that point, and maintain truer color quality.

Scott

Hi, maybe but I'not very practically with virtualdub and avisynt, however it seems to me that avisynth has more degrees of freedom in the choice of the ResizeFilter and it's taps. For example I'm still not able to set good filter, because looms constantly the risk of to see too often steps and artifacts in the image contours... otherwise trying to soften it all seems too soft.
However, I do not exclude to treat all with Virtualdub

[marcorocchini]

poisondeathray please can you tell me how is the correct method to understand what deinterlace filter is suitable for an HD .mxf video files?

I used the yadif but I'm not shure it's the best, considering the report of mediainfo in witch my video file have this proprieties:

General
Complete name : V:\automazioneclip\input\HD.MXF
Format : MXF
Commercial name : XDCAM HD422
Format profile : OP-1a
Format settings : Closed / Complete
File size : 15.8 MiB
Duration : 2s 160ms
Overall bit rate : 61.3 Mbps
Encoded date : 2013-04-24 12:18:51.000
Writing application : SONY Opt 1.22
Writing library : SONY Opt 1.22

Video
ID : 2
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format_Settings_Wrapping : Frame
Duration : 2s 160ms
Bit rate : 50.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Stream size : 12.9 MiB (82%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

is possible to determine the exact method for Deinterlace[-->resize]and Interlace?

thank you

[jagabo]

how is the correct method to understand what deinterlace filter is suitable for an HD .mxf video files?

Yadif is fairly fast but often delivers artifacts (examine low contrast detail and watch for buzzing edges when there's motion). QTGMC() usually looks better but it's much slower. A simple SeparateFields().WhateverResize().Weave() can look ok if your source doesn't have very sharp horizontal edges/lines.