h264 to h265 doesn't reduce size

[Hahaha]

I'm trying to compress a video that's 4.6GB 720p 100 minutes. It's an old 1970's show with grain.
I'm using 19.5RF which shows exact quality in preview. Going higher and I can start to see a difference but only on certain parts such as a black tuxedo while everything else is perfect. I'm wondering is there any other way I can compress it to get a smaller size or is there nothing else that can be done.
920x720 same as original
anamorphic: auto
modules: 2
no filters
H265 10-bit
23.976 constant as source
RF 19.5
Encoder: Slow
Tune: Grain
Profile Auto
Level 6.2
audio passthru

After about 3 hours the 4.6GB file is 4.4GB
I'm thinking maybe my eye sight is too good and picks up the smallest difference easily. Maybe there are other things that could be changed or I might have to sacrifice some quality. Is there any way to maybe have handbrake pickup the quality on dark areas?
Can I post a link so someone can try to play around this video if someone would like to? It is Columbo TV show

[davexnet]

I'm trying to compress a video that's 4.6GB 720p 100 minutes. It's an old 1970's show with grain.
I'm using 19.5RF which shows exact quality in preview. Going higher and I can start to see a difference but only on certain parts such as a black tuxedo while everything else is perfect. I'm wondering is there any other way I can compress it to get a smaller size or is there nothing else that can be done.
920x720 same as original
anamorphic: auto
modules: 2
no filters
H265 10-bit
23.976 constant as source
RF 19.5
Encoder: Slow
Tune: Grain
Profile Auto
Level 6.2
audio passthru

After about 3 hours the 4.6GB file is 4.4GB
I'm thinking maybe my eye sight is too good and picks up the smallest difference easily. Maybe there are other things that could be changed or I might have to sacrifice some quality. Is there any way to maybe have handbrake pickup the quality on dark areas?
Can I post a link so someone can try to play around this video if someone would like to? It is Columbo TV show

Why don't you edit out a representative 30 second clip and post it here to the forum. Much grain and noise will increase the size,
sometimes quite a bit

[Hahaha]

Sure I can do that. How would I post it here?

[davexnet]

form here:

[manono]

920x720 same as original

I hope you meant to write 960x720.

You downloaded it right? So, the previous encoder has already rung out all the bitrate savings possible without screwing up the video quality too much. By reencoding yet again you degrade it further, no matter the bitrate you give it, no matter the codec you use. Increasing the RF is about your only choice unless you know how to reduce the grain/noise to make it more compressible.

Me, I think you're wasting your time.

[Hahaha]

It is actually 960x720. My mistake.
19.5 RF seems lossless but pretty much no real decrease in file size 4.6GB vs 4.4GB
I just tried 23 RF preview and file size is much smaller. But I can notice the difference between the two. I attached both on here. This would be a nice 35% reduction from the 320GB files
I was almost actually almost agreeing with myself that I am wasting my time. But before I agree with that, I thought I see if someone on here might have some good ideas. I need to look up more info on what the other tune and profile options are. I only know Grain and None for tune and Encoder profile on Auto.

[davexnet]

two questions. is that first clip you posted from your downloaded source file,
or is it your encode?
Are you trying to maintain the grain?

[Hahaha]

Both are my encoded. One is 19.5rf and other one is 23rf. Both with grain on.
I very much do prefer grain. Without it humans look like dolls. But I guess maybe there can be a fine balance where you can have grain off and still get a good look?
I wonder if there is a way to handbrake put more quality into darker pixels. Is such a thing possible?

[davexnet]

Perhaps this?
https://forum.videohelp.com/threads/369199-Anyone-try-x264-s-new-AQ-mode

[hello_hello]

I'm still not sold on x265 for resolutions under 4k, although admittedly I haven't used it in quite a while.

You'll probably save some hard drive space by re-encoding the DTS audio. It's 1509kbps and it's only stereo. Re-encoding your sample as AAC didn't require much bitrate as it's only sparse speech, and using QAAC's default variable bitrate encoding quality, the resulting audio was only 110kbps. Size-wise, it went from a 5.3 MB DTS file to a 410kB AAC file.

Personally I hate film grain and noise, and I hate badly encoded film grain and noise even more. Not having the original video to play with, I re-encoded your "19.5" encode using x264 and CRF 18 (8 bit encoding), with some filtering to clean it up (using Avisynth). I think it looks better, but that's just my opinion. Your encode was 3807kb/s. After a bit of filtering the x264 encode came in at 2015 kb/s.

With AAC instead of DTS, the final file size was 7.7MB, so less than half the original (being your encode). See what you think....
PS. I can see a slight brightness/color difference between my encode and yours. It may be a playback issue on my PC, or it may be something else, but I wasn't motivated enough to look for the cause.

[Hahaha]

Thank you for the great contribution. That's amazing work. I haven't even thought about re-encoding audio until you mentioned it and I just realized its 1GB. I noticed you said variable bit-rate, isn't constant better? I've never done audio, I always use pass-through. How does one figure out the best compression? Just listen and compare multiple points? I noticed you took the 1500kb down to 112kb, did you randomly select this? my current audio equipment is poor but one day I hope to have better equipment. This confuses me a little, I'm not sure how low I should go with the bit-rate. I'm afraid to not hear the difference much and lose too much detail.
And what kind of filter(s) did you use to clean it up. I hope I can do it with handbrake. AviSynth will be tough for me to learn.
And I guess I'll have to settle for this. It's pretty good. I think my expectations were too high for reality.
On the left is your work. I had to zoom in this much to ask others of what they think.

[Attachment 53143 - Click to enlarge]

[Attachment 53144 - Click to enlarge]

[hello_hello]

Thank you for the great contribution. That's amazing work. I haven't even thought about re-encoding audio until you mentioned it and I just realized its 1GB. I noticed you said variable bit-rate, isn't constant better? I've never done audio, I always use pass-through. How does one figure out the best compression? Just listen and compare multiple points? I noticed you took the 1500kb down to 112kb, did you randomly select this? my current audio equipment is poor but one day I hope to have better equipment. This confuses me a little, I'm not sure how low I should go with the bit-rate. I'm afraid to not hear the difference much and lose too much detail.

I'm not sure what options Handbrake has for audio as I don't use it, but most AAC encoders have a variable bitrate mode. It's similar to x264's CRF encoding in that you select the quality and the bitrate will be what it needs to be. I used QAAC's default "true variable bitrate" quality. The command line option would be -V 91. At that quality, there's definitely space to be saved when re-encoding DTS as AAC, at least when it's high bitrate (1509). You mightn't reduce the file size a great deal when re-encoding 754kbps DTS, and at that quality I never re-encode AC3, because it doesn't reduce the bitrate enough. For 5.1ch audio, I'd take a guess at something like 400kbps being a typical bitrate for -V 91, but it'll vary from source to source.
Your sample had 2 channels, but it's mono, so AAC doesn't have to use a much higher bitrate than it would for a single channel, and your sample was pretty sparse (lots of almost silent sections which VBR doesn't have to spend many bits on), while a CBR encode would keep using the same bitrate. That's why the bitrate was so low.

For CBR, the bitrate you specify is effectively the maximum possible quality, and if sections of the audio don't require a high bitrate, you're effectively wasting bits for no perceivable increase in quality. For true variable bitrate you specify a quality and the bitrate changes to maintain that quality. If you need to control the file size there's usually an average bitrate mode. You specify the bitrate and it's adjusted up or down as required while aiming for the average bitrate specified. VBR is better as it can adjust the bitrate without having to adjust the quality in order to achieve an over-all average bitrate.

And what kind of filter(s) did you use to clean it up. I hope I can do it with handbrake. AviSynth will be tough for me to learn.
And I guess I'll have to settle for this. It's pretty good. I think my expectations were too high for reality.
On the left is your work. I had to zoom in this much to ask others of what they think.

I denoised with an Avisynth deinterlacer.
QTGMC(InputType=1, Preset="medium", EzDenoise=1.5)
It has a progressive mode for repairing bad de-interlacing, and even though your source wasn't deinterlaced (I assume), there was a fair bit of "shimmering" around objects with sharpish outlines. I'm not sure if the original video looks the same, or if it was a result of encoding the noise with x265, but I didn't like it, and it's similar to the sort of thing QTGMC can repair. QTGMC naturally denoises a little anyway, but it has a denoising filter you can enable while specifying the amount of denoising. Technically, it's probably not the right tool for the job, but I denoise with QTGMC a lot because it blurs less for an equal amount of denoising compared to most denoising functions.

Noise can be odd because your brain has a tendency to perceive it as detail, so sometimes a denoised picture can look less detailed even when there's only less noise, and I know what you mean regarding the plastic look, although it has a touch of "plastic" to it before it's denoised. That's possibly whatever they did to the "remastered" version.

I followed QTGMC with LSFMod for some subtle sharpening. Did you notice my encode was a bit sharper? It's probably most noticeable for the books on the shelf. Lastly, I used a function from the DitherTools plugin to convert the 8 bit video to 16 bit, do it's smoothing thing to gradients, then dither back to 8 bit. It can reduce any existing color banding and help prevent the encoder creating it. Combined with QTGMC it also makes existing encoder blocking less noticeable.
I still haven't discovered why the color/brightness changed a little, although I haven't tried. I converted your 10 bit video to 8 bit (because DitherTools only supports 8 bit sources) so maybe I did something silly.

If you want to try some AAC VBR encoding, download my portable version of foobar2000. You can just unzip it somewhere and run it. It'll decode almost anything, and there's converter and encoder presets for days.....
All the required goodies are included except for ffmpeg and ffprobe, as they double the file size. There's a text file explaining where to put them, and how to configure some things that require configuring etc. foobar2000 will open common video file types and play/encode the audio inside, but if you do it that way, when you replace the existing audio with the new version you need to check for any audio delay relative to the video and apply the same delay when muxing. You can check with MediaInfo or gMKVExtractGUI. Have you done that sort of thing before? MKVToolNixGUI is pretty easy to use.

By the way, have you compared x264 with x265 for 720p? I'm just wondering if x264 would do a better job at the same bitrate. If you compare the two please let me know as I'm curious, although you'll probably have to encode as 8 bit for x264, given almost nothing supports h264 10 bit.