ProRes 422 HQ vs x264 CRF 0?

[nikkmann]

I'm upscaling a movie with Video Enhance AI and want the final codec to be x265. Problem is, it doesn't output x265, only x264 or ProRes 422 HQ.

If I want a "lossless" output that can later be converted to x265 with VirtualDub, which would technically give better quality (even if its not visually distinguishable), x264 with a CRF of 0 or ProRes 422 HQ?

The original video is an old DVD using YUV 420 with an 8 bit depth.

Thanks!

[rrats]

ProRes uses less space and is visually losless (true lossless won't make much difference since you are upscaling video)

[Cornucopia]

Crf0 is lossless, but prores never is no matter what setting is used. However, prores color depth is probably overkill when it is 4:2:2 (depends on what processing is subsequently done with it).

Using a nonzero but low figure for crf with x264 would likely be of similar quality but much smaller filesize.

Scott

[jagabo]

Yes, x264 at crf=0 will be lossless if your source has 4:2:0 chroma. If it's 4:2:2 chroma you will lose half the chroma's vertical resolution (x264 will convert 4:2:2 to 4:2:0 before compressing).

[4kblurayguru]

the source is already compressed. Use handbrake to upscale (change resolution) then choose x265, try crf 17 to crf 23

[jagabo]

Handbrake is not a good upscaler.

[poisondeathray]

Yes, x264 at crf=0 will be lossless if your source has 4:2:0 chroma. If it's 4:2:2 chroma you will lose half the chroma's vertical resolution (x264 will convert 4:2:2 to 4:2:0 before compressing).

x264 supports 8bit, 10bit, 420, 422, 444, and RGB lossless encoding .

--input-csp i422 --output-csp i422 --qp 0 will encode 422 lossless.

(you have to use qp=0 for 10bit lossless; crf=0 is not lossless for 10bit)

[jagabo]

Yes, x264 at crf=0 will be lossless if your source has 4:2:0 chroma. If it's 4:2:2 chroma you will lose half the chroma's vertical resolution (x264 will convert 4:2:2 to 4:2:0 before compressing).

x264 supports 8bit, 10bit, 420, 422, 444, and RGB lossless encoding .

--input-csp i422 --output-csp i422 --qp 0 will encode 422 lossless.

(you have to use qp=0 for 10bit lossless; crf=0 is not lossless for 10bit)

Thanks. Does Video Enhance AI include versions of x264 that support and allow you to specify those features?

[Cornucopia]

Yes, I didn't mention those other things because the OP said it was a direct rip from DVD which is also 4:2:0. But it is good to know all the options.


Scott

[poisondeathray]

Yes, x264 at crf=0 will be lossless if your source has 4:2:0 chroma. If it's 4:2:2 chroma you will lose half the chroma's vertical resolution (x264 will convert 4:2:2 to 4:2:0 before compressing).

x264 supports 8bit, 10bit, 420, 422, 444, and RGB lossless encoding .

--input-csp i422 --output-csp i422 --qp 0 will encode 422 lossless.

(you have to use qp=0 for 10bit lossless; crf=0 is not lossless for 10bit)

Thanks. Does Video Enhance AI include versions of x264 that support and allow you to specify those features?

Looks like no... and the CLI interface doesn't support advanced x264 switches.

So VEIA is converting from 4:2:0 DVD src, internally to RGB, to 4:2:0 , then to x264 output locked to 4:2:0

If the DVD was good quality to begin with , and the model chosen works well for that source, you could theoretically approach 4:2:2 real chroma resolution - you could make an argument against using x264 crf0 in VEIA. But if OP's the "want the final codec to be x265" a 4:2:0 output, then it would not make a difference

[Cyrax9]

I realize that this thread is a few years old now, but since it popped up while I was looking for somewhat related information, I figured it might be worth a slight update since some things related to TVEAI (or TVAI as it's now known) have changed while others unfortunately remain the same. Likewise the fact that there are a good number of people still using older versions of TVEAI/TVAI, including TVEAI Version 2 kind of makes this topic still somewhat relevant.

I'm upscaling a movie with Video Enhance AI and want the final codec to be x265. Problem is, it doesn't output x265, only x264 or ProRes 422 HQ.

If I want a "lossless" output that can later be converted to x265 with VirtualDub, which would technically give better quality (even if its not visually distinguishable), x264 with a CRF of 0 or ProRes 422 HQ?

The original video is an old DVD using YUV 420 with an 8 bit depth.

Thanks!

Since "lossless" is in air quotes I'm assuming that the OP considers "visually/perceptually lossless" (AKA lossy/mathematically lossy) to be an acceptable option. The good news is that newer versions of TV(E)AI do indeed support h.265, (note that I did not say x265; more on that in a bit,) which kind of renders this issue moot. However, if you're one of the people whose still swearing by an older version of TV(E)AI, you'll probably benefit from ProRes 422HQ over h.264, at least in TVEAI 2, where I'm honestly not even sure that you could set -crf 0 as an option. In TVAI 3 and newer, the app is built around a custom build of ffmpeg, and that custom build tends to have some "interesting" decisions, including the inability to use x264 on some platforms! Does the Windows build actually include x264 support? If so, x264 might be a better option than ProRes if you also enable the "-g 1" switch and use all-intra mode since you're just creating an intermediate file to recompress with x265, but this only makes sense if you have access to x264's mathematically lossless mode and not another h.264 encoder only, (e.g. nvenc, QSV, et. al.,) especially if it's one that doesn't have mathematically lossless support enabled.

Crf0 is lossless, but prores never is no matter what setting is used. However, prores color depth is probably overkill when it is 4:2:2 (depends on what processing is subsequently done with it).

Using a nonzero but low figure for crf with x264 would likely be of similar quality but much smaller filesize.

Scott

Scott, given that the OP was running the video through Topaz V(E)AI, this information is actually (somewhat) inaccurate. CRF 0 is only lossless on 8-bit material; once he puts it through TV(E)AI it's no longer 8-bit material, so -qp 0 would be needed in order for the file to be mathematically lossless. Likewise, TV(E)AI is somewhat infamous for screwing up the color of videos because it doesn't really handle colorspaces properly. Basically, TVAI (and presumably the older TVEAI versions as well,) start by converting the input video into the rgb48le color space for internal processing. Since TV(E?)AI only understands material once it's in the rgb48le colorspace, and only makes its enhancements in the rgb48le colorspace, including the ones that lead to the issues with color-shifting, the argument could be made to use either an rgb colorspace or a yuv444 colorspace for the intermediate file to preserve the full range of colors that TV(E)AI has generated. The first issue that this raises though is where the OP (or anyone else for that matter) actually wants to do the colorspace conversion back to yuv420p, or even if they want to convert back to yuv420p and not, for example yuv420p10le, (10-bit 4:2:0 chroma subsampling,) yuv422p (8-bit 4:2:2 chroma subsampling,) or yuv422p10le (10-bit 4:2:2 chroma subsampling,) depending on how much "new" color data the user intends to preserve, and whether or not the file is going to be given an additional pass of color correction to fix what TV(E)AI effectively breaks. Assuming that -qp 0 and a colorspace larger than yuv420p can be enabled, I agree that x264 would probably be the better option for this version of TV(E)AI, at least if the file is just being transcoded to h.265 with x265, but if users intend to do any color correction, I would suggest maybe creating a couple of ten second samples and just trying to drop them into the tool that you intend to use for color correction to see if the software chokes on -qp 0 h.264 with a wide color gamut, or if it can handle it. If visually lossless mathematically lossy codecs are acceptable to a user who plans to make further refinements to the file, ProRes may be a better choice depending on the tool that's being used for such refinements. For virtualdub, -qp 0 h.264 with a wide color gamut shouldn't be an issue though.

Thanks. Does Video Enhance AI include versions of x264 that support and allow you to specify those features?

Looks like no... and the CLI interface doesn't support advanced x264 switches.

So VEAI is converting from 4:2:0 DVD src, internally to RGB, to 4:2:0 , then to x264 output locked to 4:2:0

If the DVD was good quality to begin with , and the model chosen works well for that source, you could theoretically approach 4:2:2 real chroma resolution - you could make an argument against using x264 crf0 in VEIA. But if OP's the "want the final codec to be x265" a 4:2:0 output, then it would not make a difference

Jagabo, I have the same question regarding the Windows build of TVAI as I haven't exactly had the chance to mess with it on a Windows system in awhile. The Mac build irritatingly doesn't even include x264 or x265, and instead relies on Apple's internal h.264 and h.265 "videotoolbox" encoders. The "good news" there though is that because Topaz enabled "videotoolbox" in their FFMPEG build, and because Apple FINALLY made their built–in ProRes encoder accessible to other apps through the "videotoolbox" API on M-series Macs, provided that you're running macOS 13 (I think, but it might go back as far as macOS 12,) or newer, anyone running Topaz's build of FFMPEG on the Mac can create ProRes encodes that are compliant with the official standard. This might not be a big deal for the average user, but on certain professional networked systems ProRes_KS is a real headache due to its non-compliance with Apple's official standard. Somewhat confusingly, Topaz has implemented (or at the very least at one point had implemented, I'm not sure if this is still the case,) both ProRes_videotoolbox and ProRes_KS on the Mac, but NOT libx264 and libx265 alongside h264_videotoolbox and hevc_videotoolbox on the Mac. Basically, this means that on that platform if you want to go straight to h.264/h.265 on the Mac, you have to use one of the system's internal encoders with Topaz's FFMPEG build. (I have no idea why they did this, and while Apple's encoders are quite good, the lack of a lossless mode or any information on how to enable them basically make them useless for my workflow for anything other than test encodes.)

Poisondeathray, TVAI is also scaling the chroma up to effectively full chroma while it works in the rgb48le colorspace, so the end result is no longer 4:2:0 material, meaning it kind of makes sense to consider other colorspaces for an intermediate file, particularly if any attempts to fix the way Topaz handles color are going to be made, or if further color correction is to be done.

Oh and while this won't help the OP, hopefully it'll help the next person who stumbles across this thead running into the same issue: As of TVAI 3 (I can't remember the exact build,) it's possible to use FFV1 as an output codec. If you're just trying to "get output into a lossless format and compress to h.265," my recommendation is to output to FFV1 with whatever your desired settings are, and to transcode that to h.265 with a more standard ffmpeg build that has the features you need in libx265 enabled.

Since the process in this thread is unfortunately still relevant in 2025, I was going to add how I normally handle the limitations of TVAI's FFMPEG build here, but realized that I had enough material for that to be its own thread after I wrote it up as a draft. If people here are interested, I'll try to remember to dump it in its own thread, and maybe someone can turn it into an actual guide at some point.