Converting mpeg2 to prores - looks worse! Help! Lots of info provided

[Skeptjk]

Hi folks.

First off I am a total noob at this, I am doing the best I can with alot of reading and using chatGPT but I'm out of my depth by a lot.

I'm trying to convert files from my sony handycam from MPEG2 to something that I can bring into davinci resolve. However whatever I try to convert them into, they look worse!

The MPEG2 files from the camera (it has an HD, so there's no ingest process other than copying the files, this one is not miniDV, it's all digital)

I've tried a bunch of file formats, but they all look the same. Hopefully it will come out in the pic, but basically the converted files look too contrasty, and have artifacts that look a bit like an interlaced image - but the original file is not interlaced! It's progressive!

Here's a pic, I opened the .mov file and the original mpeg2 in vlc and media player, put them side by side and used the windows capture tool to grab a screenshot.


[Attachment 74291 - Click to enlarge]

I'm using the following script for the conversion:

@echo off
setlocal enabledelayedexpansion

rem Set the path to the FFmpeg executable
set FFmpegPath="C:\ffmpeg\bin\ffmpeg.exe"

rem Loop through all .mpg files in the current directory
for %%F in (*.mpg) do (
    set input_file=%%~nF.mpg
    set output_file=%%~nF.mov

    rem Run FFmpeg to convert to ProRes LT with the same bit depth
    %FFmpegPath% -probesize 32M -i "!input_file!" -c:v prores -profile:v 0 -c:a pcm_s16le "!output_file!"
)

rem End of script

When I examine one of the original files using MediaInfo, here's what it tells me about it:

General
Complete name : D:\VIDEOS VAULT\CurrentVideoProject\M2U00009.MPG
Format : MPEG-PS
File size : 66.3 MiB
Duration : 59 s 40 ms
Overall bit rate mode : Variable
Overall bit rate : 9 426 kb/s
Frame rate : 25.000 FPS

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format settings, picture structure : Frame
Duration : 59 s 40 ms
Bit rate mode : Variable
Bit rate : 8 983 kb/s
Maximum bit rate : 9 100 kb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.866
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Closed
Stream size : 63.2 MiB (95%)

Audio
ID : 189 (0xBD)-128 (0x80)
Format : AC-3
Format/Info : Audio Coding 3
Commercial name : Dolby Digital
Muxing mode : DVD-Video
Duration : 59 s 40 ms
Bit rate mode : Constant
Bit rate : 256 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 1.80 MiB (3%)
Service kind : Complete Main



I just want to convert these clips into something I can bring into resolve without losing quality or changing the look of them - the colour space, etc.

I have tried multiple different versions of Prores, including 10bit variants. I'm on my 7th iteration of this script/code to do the conversion, each of them has looked basically the same, all equally as bad.

I just deleted 157gb of failed attempts!

Can anyone help me? I can provide more information if required

[Skeptjk]

QUICK UPDATE:

Ok, I'm an idiot, the original files ARE interlaced - I've changed the script to take care of that using QTGMC - Here's my new script:

@echo off
setlocal enabledelayedexpansion

rem Set the path to the FFmpeg executable
set FFmpegPath="C:\ffmpeg\bin\ffmpeg.exe"
rem Loop through all .mpg files in the current directory
for %%F in (*.mpg) do (
    set input_file=%%~nF.mpg
    set output_file=%%~nF.mov

    rem Run FFmpeg to convert to a high-quality format with QTGMC deinterlacing
    %FFmpegPath% -probesize 32M -i "!input_file!" -vf "yadif=0:-1:0,format=yuv420p" -c:v prores -profile:v 3 -c:a pcm_s16le "!output_file!"
)

rem End of script

But the new files still lose a LOT of detail in the dark areas of the image.
Here's a new example:


[Attachment 74292 - Click to enlarge]

[Sharc]

709 vs 601 color matrix? Limited vs full range?
What does MediaInfo report for the .mov file?

[Skeptjk]

Man I don’t even know what that means. I will be back at my computer in an hour or so though and I’ll post you whatever media info tells me about the .mob files.

I should prolly mention, I’m not overly attached to .mov/proeres, I just want to edit in resolve with conversions as good as the originals

[Skeptjk]

Here's the MediaInfo from one of the mov files:

Format : MPEG-4
Format profile : QuickTime
Codec ID : qt 0000.02 (qt )
File size : 91.2 MiB
Duration : 22 s 80 ms
Overall bit rate mode : Variable
Overall bit rate : 34.6 Mb/s
Frame rate : 25.000 FPS
Writing application : Lavf60.13.100

Video
ID : 1
Format : ProRes
Format version : Version 0
Format profile : 422 HQ
Codec ID : apch
Duration : 22 s 80 ms
Bit rate mode : Variable
Bit rate : 33.1 Mb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:2
Scan type : Progressive
Bits/(Pixel*Frame) : 3.192
Stream size : 87.1 MiB (96%)
Writing library : fmpg

Audio
ID : 2
Format : PCM
Format settings : Little / Signed
Codec ID : sowt
Duration : 22 s 80 ms
Bit rate mode : Constant
Bit rate : 1 536 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Bit depth : 16 bits
Stream size : 4.04 MiB (4%)
Default : Yes
Alternate group : 1


Just to reiterate -the goal is to deinterlace and convert multiple files with QTGMC and have the new files look as good as the originals, then edit them in Davinci Resolve. I don't particularly care if they're prores, I've just picked that because it's what the tutorials I've followed have suggested.

Edit:
Just looking at the code, despite asking Chat GPT to use QTGMC it's using Yadif. Tried getting it to use QTGMC, but it seems FFMPEG doesnt support QTGMC?

I was previously using AVISynth and VirtualDub 64 but I can only do one file at a time using that, and besides, I just did a test file, and it comes out the same - better quality than Yadif, but with the same loss of detail in the dark areas.

[jagabo]

It's a limited range vs. full range problem. What does your AVS script look like? Try flagging the prores encoding as full range with "-color_range pc" or limited range with "-color_range tv".

QTGMC is not implemented in ffmpeg. Just AviSynth and VapourSynth. Try using bwdif in ffmpeg (better than yadif, not as good as QTGMC).

[Skeptjk]

At the moment there is no Avs script - the batch file generates one and deletes it when it’s done I think.

I haven’t much experience with code so I don’t know how to implement those changes without just feeding your answer into chat gpt

That said, are you saying I need to make the ProRes full range? Or limited range?

[jagabo]

There are too many places where things could go wrong so I can't say for sure where your problem is. Your DVD MPG source is certainly limited range rec.601. And should probably be encoded as limited range rec.601. But I suspect somewhere in your processing it is being converted to full range rec.601 without being flagged as such. That is causing the prores decoder to perform a second limited range to full range conversion, crushing blacks and blowing out brights.

You might be able to fix this by flagging the prores mov as full range. Where you encode with ffmpeg to prores mov add "-color_range pc" to the command line after specifying the video codec.

[Skeptjk]

I’ll give it a go in the morning!

The files are not from a dvd though, they are from a Sony handycam camcorder - not sure if that is relevant tho

[jagabo]

The files are not from a dvd though, they are from a Sony handycam camcorder - not sure if that is relevant tho

The MPG source is almost certainly limited range 601 even though it's not indicated by MediaInfo.

[Skeptjk]

Man this stuff is like rocket science to the layman (ie me)

I’ll post back tomorrow with how I get on with this new info!

Many thanks and watch this space I guess !

[Cornucopia]

Bittate for ProRes HQ 4:2:2 seems low also. I would expect it more in the range of 50-65 (Apple white paper docs expect 61Mbps for SD PAL).

Then there is the 4:2:0 --> 4:2:2 chroma upsampling in addition to the resizing and deinterlacing, and 601/709 conversion, and possibly the limited vs full range (though prores should almost always be limited range, IIRC).


Scott

[Bwaak]

DaVinci Resolve can import MPEG-2 in MXF Op1A, MXF Op‑Atom and MOV containers. You can re-wrap your MPG file into MOV simply by doing this:

ffmpeg.exe -i myInputFile.mov -acodec copy -vcodec copy myOutputFile.mov

If you use other extension than "MOV" for the output file, you need to specify output container format explicitly. If you have file names with spaces, enclose them in double-quotes.

EDIT: Ah, you want to deinterlace as well. DaVinci Resolve can deinterlace, and some people claim that its Neural Engine, starting from version 17, is as good or better than QTGMC. It would be interesting to see a comparison, since I don't use DaVinci Resolve.

EDIT 2: Then again, some people say, that its deinterlacer is trash

[Skeptjk]

I understand some of these words.

I will have a play and report back!

[Skeptjk]

Actually, not sure if this will help - but here's a link to one of the original clips - maybe this is one of those things where a mind with the right knowledge, experience and intelligence will discern the answer within about 30 seconds, wheras it may yet take me a day or more of fumbling in the dark to arrive at the same conclusion!

https://www.dropbox.com/scl/fi/n3jeydxy871bdeppunj5v/M2U00015.MPG?rlkey=06roxhamt2wq4s...c6ik159e7&dl=0

[jagabo]

Your video (video only), QTGMC in AviSynth+, ffmpeg to prores (your command line, unflagged range, matrix).

Mpeg2Source("M2U00015.d2v", Info=3) 
QTGMC()
prefetch(12)

ffmpeg -probesize 32M -i %1 -c:v prores -profile:v 0 -aspect 16:9 -c:a pcm_s16le %1.prores.mov

No levels/color change here.

[Skeptjk]

Your video (video only), QTGMC in AviSynth+, ffmpeg to prores (your command line, unflagged range, matrix).

Mpeg2Source("M2U00015.d2v", Info=3) 
QTGMC()
prefetch(12)

ffmpeg -probesize 32M -i %1 -c:v prores -profile:v 0 -aspect 16:9 -c:a pcm_s16le %1.prores.mov

No levels/color change here.

Ok! That looks ideal!

Problem is I have no idea what to do with the code you just gave me, do I save the top one as an AVSynth script and run the bottom one in the command line? What do I save the script as? I'm assuming it has to be in the same directory as the source file?

Oh, also, sound! How do I get the sound back in?

Sorry you're having to baby-step me through this, I do appreciate the help and patience - I didn't realise how much i'd be throwing myself in at the deep end with this.

[jagabo]

do I save the top one as an AVSynth script

Basically, yes. See below for more...

What do I save the script as?

Whatever filename you want. Just specify .AVS as the extension when you save (with Notepad, for example). I usually name my scripts with the name of the file it's opening, with .AVS appended. So with your M2U00015.mpg I would use M2U00015.mpg.avs.

and run the bottom one in the command line?

Yes. But the line I gave is set up for use in a batch file. You save the text as something.BAT. Then you can drag/drop your AviSynth script onto it. I used the name _avs2prores.bat. If you put the batch file in Windows' Send To folder you can right click on a AVS script and select Send To -> _avs2prores.bat to encode it. The command line saves the video with the name of the AVS script plus ".prores.mov". So if you encode M2U00015.mpg.avs with that batch file you will get M2U00015.mpg.avs.prores.mov as your final output.

If ffmpeg.exe isn't in your search path you can give the entire path to the program in the batch file. Using your ffmpeg path in post #1:

"C:\ffmpeg\bin\ffmpeg.exe" -probesize 32M -i %1 -c:v prores -profile:v 0 -aspect 16:9 -c:a pcm_s16le %1.prores.mov

I'm assuming it has to be in the same directory as the source file?

As it's written, yes. But it's possible to put the files anywhere and use full path names instead of just the filenames.

Oh, also, sound! How do I get the sound back in?

Note that I used Mpeg2Source() in the AviSynth script. That is a third party source filter for MPEG 2 sources (you may need to download and install it if you don't have it already). It's the most reliable of of the source filters for reading MPEG 2 video in MPG or VOB containers. But it doesn't include audio. Here's a variation that uses the LSMASHSource filter to open both the audio and video:

a = LWlibavAudioSource("M2U00015.mpg", cache=false)  # get the audio
v = LWlibavVideoSource("M2U00015.mpg", cache=false)  # and the video
AudioDub(v,a) # join them together
QTGMC()
prefetch(12)

LSMASH can be found here:
http://avisynth.nl/index.php/LSMASHSource
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/download/1144.0....r1144.0.0.0.7z

Download the 7z file, extract the LSMASHSource.dll file, and put it in AviSynth's Plugins64+ folder.

I highly recommend using a better profile for your prores encoding. 2 is probably the best compromise for what you're doing (less loss of quality, bigger files).