Color Conversion: How To Tonemap BT.2020 To BT.709

[LouieChuckyMerry]

Hello, and thanks in advance for any help.

I was gifted a UHD Blu-ray but don't have a 4K TV, so I figured I'd do the color conversion and resizing losslessly with VirtualDub2-UtVideo then shrivel that as I normally would with MeGUI-AviSynth. Unfortunately, after several days and many hours, I can't for the life of me figure out how to tonemap BT.2020 to BT.709. I've tried everything from the seemingly obvious

ColorMatrix(Mode="Rec.2020->Rec.709")

to HDRTools plugin and

ConvertYUVtoXYZ(Color=1)
ConvertXYZtoYUV(pColor=1)

with no noticeable change in the video's colors. Please, can anyone offer a solution (or a link to a tutorial)?

Ahhh, here's a test clip.

[davexnet]

Search the forum for "tonemap" lots of discussion.
It was mentioned that Handbrake\Vidcoder seemed to do it if you select bt.709 in Filters\Colorspace

Here's an Avisynth example:

lwlibavvideosource("C:\Users\Public\Videos\Mamma Mia_sample2_HEVC_HDR_re-encode.mp4")

ConvertBits(10) # Use 12 for 12-bit
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:l", dither_type="none")
# Choose one of these:
DGReinhard()
#DGHable()
z_ConvertFormat(pixel_type="YV12",colorspace_op="rgb:linear:2020:l=>709:709:709:l",dither_type="ordered")

[LouieChuckyMerry]

Thanks for your reply, davexnet. Honest, I searched for many hours, and on more than this forum. I've looked into HandBrake but am running Win 7, so the latest version is rather old (plus, the only "lossless" options are either x264 or x265 with a CRF of 0 which, after more searching, makes me wonder if it's actually lossless). I'd prefer VirtualDub2/UtVideo for the first pass, and appreciate your above script, but it seems the syntax is off because, even after trying to correct it, I receive error messages that I'm unable to overcome. Unfortunately I'm Extremely CLI Challenged, so ffmeg is too daunting (I'll search for a GUI, though). Anyway, if I find a solution that seems acceptable I'll post it here so that others searching for a solution can find one (hence, the title of my post which, to be fair, is probably technically incorrect ).

[LouieChuckyMerry]

Trying to change title of post... "Preview Changes" shows the new title... Sigh...

[davexnet]

The syntax is fine, you may have some external filters missing, eg. DGTonemap and possibly L-Smash Works (for LWLibavvideosource)
http://avisynth.nl/index.php/DGTonemap
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases
older releases of Lsmash here:
https://github.com/HolyWu/L-SMASH-Works/releases/

[LouieChuckyMerry]

Thanks again for your help, davexnet. I already index with L-Smash Works and have DGTonemap in my Plugins folder; however, when I attempt to run your script in VirtualDub2 I receive the error message "There is no function named 'z_ConvertFormat'."

Edit: Maybe I need to manually load DGTonemap? I'll try...

EditEdit: Nope, same error message. Here's my script (using AviSynth+):

LoadPlugin("FilePath\LSMASHSource.dll")
LoadPlugin("FilePath\DGTonemap.dll")
LWLibavVideoSource("FilePath.mkv", format="YUV420P8")
SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(10) # Use 12 for 12-bit
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:l", dither_type="none")
# Choose one of these:
DGReinhard()
#DGHable()
z_ConvertFormat(pixel_type="YV12",colorspace_op="rgb:linear:2020:l=>709:709:709:l",dither_type="ordered")
PreFetch(5)

[davexnet]

Thanks again for your help, davexnet. I already index with L-Smash Works and have DGTonemap in my Plugins folder; however, when I attempt to run your script in VirtualDub2 I receive the error message "There is no function named 'z_ConvertFormat'."

Edit: Maybe I need to manually load DGTonemap? I'll try...

Looks like you also need this...
http://avisynth.nl/index.php/Avsresize

[ProWo]

You can do the colorspace change and the resize at the same time.
Take clever Ffmpeg-GUI, load your video, click main, click encode videostream, click resize.
Make your settings and click encode.


[Attachment 71533 - Click to enlarge]

When that's done, click multiplex and mux your new video.

[LouieChuckyMerry]

davexnet: Thank you, I'll try to carve out some time tonight to add AVSResize to my Plugins folder (I need to wait for an encode to finish).
_____________

ProWo: Wow, excellent timing (and thanks for your reply). I went hunting for an FFMpeg GUI yesterday and settled on clever FFMpeg-GUI for my first to try (thanks for your GUI, too; I especially appreciated the set up guide upon first opening), so thank you also for the above picture. After color conversion and resizing I plan to encode the result with my usual AviSynth script using MeGUI (denoising, sharpening, debanding, etc.), so I'd like to do the first bit losslessly. I'd normally use VirtualDub2/UtVideo; do you have a recommendation for a lossless codec in clever FFMPeg-GUI?

[ProWo]

Use x264 crf 0

[LouieChuckyMerry]

Use x264 crf 0

Thank you. Do the "Preset" or "Profile" matter? I've never used x264 for lossless encoding before.

[ProWo]

Use x264 crf 0

Thank you. Do the "Preset" or "Profile" matter? I've never used x264 for lossless encoding before.

With lossless, the profile is automatically selected by the encoder.
I would choose veryfast as preset, so that the encoding is fast.
This has no influence on the quality.

[LouieChuckyMerry]

ProWo: Thanks a bunch for your help; I really appreciate it .

[LouieChuckyMerry]

ProWo: What an awesome GUI . And the choice of color is a very nice touch. Thanks again .

[LouieChuckyMerry]

ProWo: I was excited to run some tests today but clever FFMpeg-GUI "opens" minimized and can't be maximized so there's no GUI. This happens on two computers, both running Win 7, even after reboots. Any ideas?

[ProWo]

ProWo: I was excited to run some tests today but clever FFMpeg-GUI "opens" minimized and can't be maximized so there's no GUI. This happens on two computers, both running Win 7, even after reboots. Any ideas?

Delete this folder (it will be created again automatically):
C:\Users\xxx\AppData\Local\clever_FFmpeg_GUI (where xxx=your user name on windows).

[LouieChuckyMerry]

That did it, thank you very much.

[LouieChuckyMerry]

Having some time this weekend I ran some tests on a short clip to compare six different methods of tonemapping:

1) AviSynth[davexnet's Above Script + Downscaling]

2) clever FFMpeg-GUI[Reinhard + Downscaling]

3) clever FFMpeg-GUI[Hable + Downscaling]

4) HandBrake[Colorspace BT.709 + Downscaling]

5) Avidemux[Reinhard + Downscaling]

6) Avidemux[Hable + Downscaling]

I didn't note speed because not every method displays that information and I did no fine-tuning of the settings because not every method allowed for that. Here are the results (in the above order):

1,1:


[Attachment 71652 - Click to enlarge]

2,1:


[Attachment 71653 - Click to enlarge]

3,1:


[Attachment 71654 - Click to enlarge]

4,1:


[Attachment 71655 - Click to enlarge]

5,1:


[Attachment 71656 - Click to enlarge]

6,1:


[Attachment 71657 - Click to enlarge]

1,2:


[Attachment 71658 - Click to enlarge]

2,2:


[Attachment 71659 - Click to enlarge]

3,2:


[Attachment 71660 - Click to enlarge]

4,2:


[Attachment 71661 - Click to enlarge]

5,2:


[Attachment 71662 - Click to enlarge]

6,2:


[Attachment 71663 - Click to enlarge]

To my eyes the results with HandBrake are the worst, but to be fair I'm running Win 7 and the last version of HandBrake I can use, 1.4.2, is verrrrrry old. I also think the davexnet's AviSynth script is not so good but, again to be fair, I don't know enough to make the necessary adjustments (although I'm sure davexnet does ). Clever FFMpeg-GUI with Hable is a bit dark, but it's results with Reinhard look good. Avidemux's two results are quite similar but I find the Hable a bit too bright. So, down to clever FFMpeg-GUI with Reinhard and Avidemux with Reinhard I'd pick clever FFMpeg-GUI for the indoor scene, given it's light from candles and through stained glass, but Avidemux for the outdoor scene. Both clever FFMpeg and Avidemux's settings are tweakable so I'd do some more testing until happy (or I could beg ProWo for some advice ).
_____________

EDIT: I just noticed that I failed to downscale a couple of the test clips but it seems to me that that shouldn't affect the colors (but, hey, I've been wrong many times before).