Size reduction - optimal codec and settings

[alfik0]

Hi,

I want to reduce the size of dynamic animations with various quality(1080p@15mbps, 1080p@10, 720p@3, examples below) to around 300-500MB per hour. From googling I have found that I should use ffmpeg+some gui with x265 codec on high crf(~30) and avcodec on quality 4. Now I have a few questions:

1) Are my findings correct? or would you suggest a different approach?
2) Should I resize the videos to 360p/480p/720p or leave them at source resolution?
3) Should I include some additional settings or just let crf take care of everything?
4) Anything else I should pay attention to?

Thanks in advance.

Example 1

General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/iso2/avc1/mp41)
File size                                : 8.81 GiB
Duration                                 : 1 h 23 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 15.0 Mb/s
Writing application                      : HandBrake 1.0.3 2017022500

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 h 23 min
Bit rate                                 : 14.8 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 60.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.119
Stream size                              : 8.71 GiB (99%)
Writing library                          : x264 core 148 r2708 86b7198
Encoding settings                        : cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=6 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=600 / keyint_min=60 / scenecut=40 / intra_refresh=0 / rc_lookahead=10 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=20000 / vbv_bufsize=25000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 h 23 min
Bit rate mode                            : Variable
Bit rate                                 : 158 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 94.9 MiB (1%)
Title                                    : Stereo / Stereo
Language                                 : English
Default                                  : Yes
Alternate group                          : 1

Example 2

General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp42/isom)
File size                                : 1.37 GiB
Duration                                 : 29 min 18 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 6 669 kb/s

Video
ID                                       : 2
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.2
Format settings                          : CABAC / 2 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 2 frames
Muxing mode                              : Container profile=High@4.0
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 29 min 18 s
Bit rate mode                            : Variable
Bit rate                                 : 6 341 kb/s
Maximum bit rate                         : 26.0 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.051
Stream size                              : 1.30 GiB (95%)
Language                                 : English
Codec configuration box                  : avcC

Audio
ID                                       : 1
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 29 min 18 s
Bit rate mode                            : Constant
Bit rate                                 : 320 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 67.1 MiB (5%)
Language                                 : English

Example 3

General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/iso2/mp41)
File size                                : 1.55 GiB
Duration                                 : 1 h 14 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 977 kb/s
Writing application                      : HandBrake 0.10.5 2016021100

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L4@Main
Codec ID                                 : hev1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 1 h 14 min
Bit rate                                 : 2 652 kb/s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 59.940 (60000/1001) FPS
Minimum frame rate                       : 29.980 FPS
Maximum frame rate                       : 59.960 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.048
Stream size                              : 1.38 GiB (89%)
Writing library                          : x265 1.9:[Windows][GCC 4.9.0][64 bit] 8bit
Encoding settings                        : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=3 / merange=57 / rect / amp / max-merge=3 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=600 / min-keyint=60 / scenecut=40 / rc-lookahead=30 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=4 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=6 / psy-rd=2.00 / rdoq-level=2 / psy-rdoq=1.00 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : hvcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 h 14 min
Bit rate mode                            : Variable
Bit rate                                 : 315 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 168 MiB (11%)
Title                                    : Stereo / Stereo
Default                                  : Yes
Alternate group                          : 1

[smrpix]

When you encode short samples the way you propose, are the results acceptable to you?

[ProWo]

Reencode video stream only, leave the audio streams as they are.
1) 1920x1080, x264, CRF 27, frame Rate 30fps
2) 1920x1080, x264, CRF 27, frame rate 29.97fps
3) 1280x720, x264, CRF 24, frame rate 29.97fps

[sophisticles]

Your first clip says that it's 1hr23min 1080p60 with a file size of nearly 9GB and you want to reduce that to 300-500 mb, which works out to 18-27 times smaller. Since the file you have is 15 mb/s, that means you need to reduce the bit rate to 555kb to 833kb for 1080p60!!! I don't care what codec you use or what settings, that is going to look like crap!

[alfik0]

I want to thank you for your time, but... Ehhh, I was counting for someone who had done similar before and is willing to share the experiences.
Smrpix, I appreciate advice "if it's good enough, don't waste more time tweaking it and over-optimizing it", it's very good one, but it's not what I'm looking for.
ProWo, your advice is just outright harmful. The input files which I have are from many sources and audio is unknown, it can be much crazier than what I posted. 2) and 3) have 320 kbps audio which weighs 144MB per hour. You are suggesting placebo quality audio taking 30-50% of the intended filesize. Due to your comment I included x264 in testing and now I have clear view of advantages/disadvantages of both codecs. x265 is better looking at only quality per bitrate, but results in worse playback experience. Thanks for x264 suggestion.
sophisticles, The entire point of codecs is to reduce the filesize at the cost of playback performance. No, the result won't look like "crap' because input is high quality... Yes, result will look much worse, but that's normal and reason isn't high quality source. I accept it.

In the end, I set up short tests with x264 and x265 to compare the quality and size between different resolutions and crf settings. You can find the script, results, settings and files below. My subjective opinion after looking at videos with desired bitrate(~550kbp, green color in pivot) in somehow random testing are that
1) x265 has better quality than x264 at a similar bitrate and resolution
2) higher resolution results in better details during static scenes and increase artifacts during movement
So the "optimal" setting for me will be x265 with crf selected to acceptable level of movement artifacts and resolution reduced to obtained desired size. It will be something around 480p at crf=20 to 720p at crf=25.
Now I have to decide how important fast seeking is and if enabling fastdecode/keyint/b-frames is worth it. Or if switching to x264 would be better. If someone knows about these or could point me in the right direction I would be grateful.

The script can be run in powershell with argument-executionpolicy bypass. So [Win]+[R]

powershell -executionpolicy bypass

enter, travel to the desired folder and run script.

$resolutions = "144", "360", "480", "720", "1080", "0"

foreach ($resolution in $resolutions) {
    for($crf = 15; $crf -lt 51; $crf += 5){ 
        $con = '_'+$resolution+'_'+$crf+'.mp4'
        $output = ".\x265"+$con
        ffmpeg -ss 02:50 -i .\input.mp4 -map 0 -vf scale=-8:$resolution -c:v libx265 -preset fast -x265-params crf=$crf -strict -2 -movflags faststart -c:a libopus -ac 2 -b:a 128k -vbr on -compression_level 10 -codec:s copy -t 10 $output
        $output = ".\x264"+$con
        ffmpeg -ss 02:50 -i .\input.mp4 -map 0 -vf scale=-8:$resolution -c:v libx264 -preset fast -x264-params crf=$crf -strict -2 -movflags faststart -c:a libopus -ac 2 -b:a 128k -vbr on -compression_level 10 -codec:s copy -t 10 $output
    }
}

[Attachment 53952 - Click to enlarge]

csv of results

Size,Audio Size,Video Size,Video bitrate,Resolution,CRF,Codec
3742,300,3442,2754,1080,20,x264
2853,300,2553,2043,1080,20,x265
2219,300,1919,1535,1080,25,x264
2033,300,1733,1386,720,20,x264
1704,300,1404,1123,1080,25,x265
1694,300,1394,1115,720,20,x265
1422,300,1122,898,1080,30,x264
1292,300,992,793,720,25,x264
1248,300,948,758,480,20,x264
1124,300,824,659,480,20,x265
1114,300,814,651,1080,30,x265
1104,300,804,643,720,25,x265
1012,300,712,569,1080,35,x264
948,300,648,518,360,20,x264
902,300,602,482,720,30,x264
878,300,578,462,360,20,x265
863,300,563,450,480,25,x264
806,300,506,405,1080,35,x265
795,300,495,396,480,25,x265
790,300,490,392,1080,40,x264
786,300,486,389,720,30,x265
692,300,392,314,720,35,x264
692,300,392,314,360,25,x264
656,300,356,285,360,25,x265
649,300,349,279,480,30,x264
646,300,346,277,1080,45,x264
639,300,339,271,1080,40,x265
614,300,314,251,720,35,x265
612,300,312,250,480,30,x265
571,300,271,217,720,40,x264
556,300,256,205,1080,50,x264
548,300,248,198,1080,45,x265
546,300,246,197,360,30,x264
531,300,231,184,480,35,x264
530,300,230,184,360,30,x265
513,300,213,170,1080,50,x265
512,300,212,169,720,40,x265
506,300,206,165,480,35,x265
505,300,205,164,144,20,x264
502,300,202,161,144,20,x265
491,300,191,153,720,45,x264
465,300,165,132,360,35,x264
461,300,161,129,480,40,x264
456,300,156,125,360,35,x265
455,300,155,124,720,45,x265
447,300,147,117,720,50,x264
443,300,143,115,480,40,x265
437,300,137,110,144,25,x265
433,300,133,107,144,25,x264
432,300,132,106,720,50,x265
417,300,117,94,360,40,x264
416,300,116,93,480,45,x264
413,300,113,90,360,40,x265
408,300,108,86,480,45,x265
398,300,98,78,144,30,x265
394,300,94,75,480,50,x265
393,300,93,74,480,50,x264
392,300,92,74,144,30,x264
389,300,89,71,360,45,x265
388,300,88,70,360,45,x264
379,300,79,64,360,50,x265
375,300,75,60,144,35,x265
372,300,72,58,360,50,x264
369,300,69,55,144,35,x264
363,300,63,50,144,40,x265
356,300,56,45,144,40,x264
356,300,56,45,144,45,x265
353,300,53,42,144,50,x265
349,300,49,39,144,45,x264
345,300,45,36,144,50,x264

encode testing.zip

[ProWo]

ProWo, your advice is just outright harmful. The input files which I have are from many sources and audio is unknown, it can be much crazier than what I posted. 2) and 3) have 320 kbps audio which weighs 144MB per hour.

For the audio, reencode it to 192 aac. You can use for all conversions (audio and video) and remuxing my clever FFmpeg-GUI.

[alfik0]

ProWo, your advice is just outright harmful. The input files which I have are from many sources and audio is unknown, it can be much crazier than what I posted. 2) and 3) have 320 kbps audio which weighs 144MB per hour.

For the audio, reencode it to 192 aac. You can use for all conversions (audio and video) and remuxing my clever FFmpeg-GUI.

192kbps still takes 28.8% if the intended filesize is 300MB. I think 128(20%) is already a bit of overkill, when in ABX testing it's difficult to notice higher bitrates completely focusing on music.

PS: yup, I will settle at 96kbps(15%) since I value audio quality. Although something around 70 probably would be "optimal" according to this and this topic.
PSS: reading about this, I will also probably reencode my music collection from 320 to 128...