VideoHelp Forum
+ Reply to Thread
Results 1 to 14 of 14
Thread
  1. I have a large number of lecture files directly from the camcorder. Hence, these are huge .MTS files. They are taking up almost 800 GB right now with each hour-long lecture being around/over a GB or even 2. I am very new to the encoding area and only know a few things from different quality anime I download and why their file sizes vary so much because of bitrate.

    As for my goal, since they are simply lectures, I don't care much about video quality. I only look at it for facial/hand expressions and in case any powerpoint slides (latter very rarely). The most important thing is just not toooo blurry on a computer screen (32" monitor max) and having OK audio since the primary thing is to listen.

    While digging the internet, I found this option to convert MTS to MP4 for individual files while controlling the quality through this CRF parameter
    Code:
    ffmpeg -i input.mts -crf 23 -s vga output.mp4 -preset veryfast
    This does a very good job of getting a 1.97 GB file down to just 122 MB. However, it also takes it from 1080p to 480p without my control. The audio is bearable though I'd prefer a bit above. Funny when I used it yesterday on another file (can't find it right now), it gave me around 240 MB, 480p, but almost identical audio. This is OK but if I could somehow control the audio to be a bit better and make it batch, I'd probably be good.

    While looking for a batch method, I found this code
    Code:
    for %%a in ("*.*") do ffmpeg -i "%%a" -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k "D:\Converted\%%~na.mp4"
    This is in batch form so it's good. But it gives me 700 MB, 1080p, with good audio. Too big files. I want some batch form that gives 240 MBish like the previous code (good audio, 480p or 720p) but in batch form.

    As a bonus, since these are in well-structured folders, is it possible to write them in a similar folder structure in another drive? The above code is being run on an external drive but writing new files to the "Converted" folder in D drive. This is because I don't have the space in the external right now to make new copies of 800 GB of files. It's best if FFMPEG can make the files with the same names in folders with the same names in a folder/drive of my choice. Is that possible?
    Quote Quote  
  2. Something like:

    Code:
    for %%a in ("*.mts") do (
        md "D:%%~pa"
        ffmpeg -i "%%a" -vf scale=w=1280:h=-4 -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k "D:%%~pna.mp4"
    )
    pause
    will downscale to 1280 by the nearest mod 4 height that's necessary to keep the aspect ratio. It will place the new files in the same folder tree on drive D:. You can decrease the file size by using a higher crf value (and hence lower picture quality), or by using a smaller frame size (trading off resolution to get smaller files). Note that crf encoding doesn't guarantee the final file size because some videos will compress more than others. So two source videos that are exactly the same size may result in very different sizes after compression.
    Quote Quote  
  3. Originally Posted by jagabo View Post
    will downscale to 1280 by the nearest mod 4 height that's necessary to keep the aspect ratio.
    Thank you so much for the code. Could you explain what the "nearest mod 4" means? But since my videos are actually 4:3, I will probably fix the height to 720 instead since this gives 1280x960, which is a bit odd. I guess that would just be "scale=h=960:w=-4"? Assuming the mod 4 thing remains the same.

    Originally Posted by jagabo View Post
    Note that crf encoding doesn't guarantee the final file size because some videos will compress more than others. So two source videos that are exactly the same size may result in very different sizes after compression.
    Do you mean the same file, with FFMPEG run on it twice at the same CRF value, will give 2 different file sizes?

    The bad news, unfortunately, is it's only converting the file(s), if any, in the main folder, nothing in the subfolders. Do you know what could be the problem?
    Quote Quote  
  4. Originally Posted by bloodtalon View Post
    Originally Posted by jagabo View Post
    will downscale to 1280 by the nearest mod 4 height that's necessary to keep the aspect ratio.
    Thank you so much for the code. Could you explain what the "nearest mod 4" means? But since my videos are actually 4:3, I will probably fix the height to 720 instead since this gives 1280x960, which is a bit odd. I guess that would just be "scale=h=960:w=-4"? Assuming the mod 4 thing remains the same.
    "Nearest mod 4" means the nearest integer multiple of 4 (4, 8, 12, 16...). So if your source was 1920x812 it would be reduced to 1280x540, not the more exact 1280x541. Many codecs have restrictions on frame sizes. x264 requires mod 2 with YV12 sources but there are some old decoders that require mod 4. So it's safest to stay with mod 4. When the width or height is specified as a negative value ffmpeg calculates the correct size for you using the absolute value of the number as the modulus. You can always specify the exact size yourself for example "-vf scale=w=960:h=720" for a 4:3 source. Keep in mind that any small text may become unreadable if you reduce the frame size too much.

    Originally Posted by bloodtalon View Post
    Originally Posted by jagabo View Post
    Note that crf encoding doesn't guarantee the final file size because some videos will compress more than others. So two source videos that are exactly the same size may result in very different sizes after compression.
    Do you mean the same file, with FFMPEG run on it twice at the same CRF value, will give 2 different file sizes?
    No, the same file encoded twice will be the same size. But two different videos of the same frame size, frame rate, and running time may turn out with very different sizes because of the picture content. Foe example, an action movie will turn out much larger than a lecture full of static pie charts, text, etc.

    Originally Posted by bloodtalon View Post
    The bad news, unfortunately, is it's only converting the file(s), if any, in the main folder, nothing in the subfolders. Do you know what could be the problem?
    It was only meant to process files in a particular folder. If you want to include all sub-folders just add the "recursive" switch just after the "for" command. Ie, change "for %%a..." to "for /R %%a..."
    Quote Quote  
  5. will downscale to 1280 by the nearest mod 4 height that's necessary to keep the aspect ratio. It will place the new files in the same folder tree on drive D:. You can decrease the file size by using a higher crf value (and hence lower picture quality), or by using a smaller frame size (trading off resolution to get smaller files). Note that crf encoding doesn't guarantee the final file size because some videos will compress more than others. So two source videos that are exactly the same size may result in very different sizes after compression.
    And also, although not to the same extent, by using a slower preset than "medium". I'm not sure how much space can be saved on average by going from "medium" to "veryslow" for instance. Plus it may be dependant on the type of content. You could test the effect on one video, and decide if the extra saved space is worth the increased encoding time. If you don't care that much about the video quality, using a higher "CRF" value like "-crf 25" might be more efficient. Or encoding with x265, which is supposed to yield significantly smaller files for a similar subjective quality, but is much slower...

    No, the same file encoded twice will be the same size. But two different videos of the same frame size, frame rate, and running time may turn out with very different sizes because of the picture content. Foe example, an action movie will turn out much larger than a lecture full of static pie charts, text, etc.
    In this case, since these videos are all lectures, the bitrate should be in the same ballpark, it would be surprising if two videos of the same duration with the same parameters (resolution, framerate, etc.) recorded with the same device end up being compressed as 122MB and 240MB with the same compression settings. Of course if one is 20min and the other 40min that's normal. The lighting conditions could affect the level of digital noise and hence the video bitrate, but not by a factor 2.

    This is OK but if I could somehow control the audio to be a bit better and make it batch, I'd probably be good.
    If the audio is already in a compressed format (which it most likely is – you can check this with MediaInfo) then you should just copy the streams, no need to recompress if no further processing is required (see below). Based on the script above (with /R for "recursive", a resolution of 960x720 and "-crf 25" for more video compression) :

    Code:
    for /R %%a in ("*.mts") do (
        md "D:%%~pa"
        ffmpeg -i "%%a" -vf scale=w=960:h=-4 -c:v libx264 -preset medium -crf 25 -c:a copy "D:%%~pna.mp4"
    )
    pause
    This is in batch form so it's good. But it gives me 700 MB, 1080p, with good audio.
    It's surprising that you get better audio with "-c:a aac -b:a 128k", than with no specified audio setting (as in the first command line shown), since this is the default ffmpeg setting for MP4 as far as I know, so the audio output should be the same.

    A good reason to recompress the audio would be to make it "more audible" if it's too weak in the original recordings, with a normalization or dynamic range compression processing. In this case, for optimal quality, it would be better to use QAAC rather than ffmpeg's internal AAC encoder, or at least use a higher bitrate than 128k to preserve the audio signal as much as possible. There's a --normalize option in QAAC which may be enough for that purpose, but if there are already peaks near the limit (due for instance to someone coughing or clapping hands next to the recording device) then it won't do much, in this case a more advanced dynamic range compression processing would be needed, I don't know if there's a way to automate it efficiently.
    Last edited by abolibibelot; 9th Jul 2020 at 10:51.
    Quote Quote  
  6. I was trying to post after everything was done to give feedback but it's looking good so far on the files I have tried, so I am commenting now since it's going on for >a day now.

    Originally Posted by abolibibelot View Post
    And also, although not to the same extent, by using a slower preset than "medium". I'm not sure how much space can be saved on average by going from "medium" to "veryslow" for instance. Plus it may be dependant on the type of content. You could test the effect on one video, and decide if the extra saved space is worth the increased encoding time. If you don't care that much about the video quality, using a higher "CRF" value like "-crf 25" might be more efficient. Or encoding with x265, which is supposed to yield significantly smaller files for a similar subjective quality, but is much slower...
    Since I am looking for a reasonably fast process with 800ish GB to process, I will stick to x264. I did try all the different presets and found veryslow actually gave me a bigger filesize (maybe better quality) and so did ultrafast. I found veryfast to be the best speed and also almost the lowest filesize. Happy with its speed!
    Originally Posted by abolibibelot View Post
    In this case, since these videos are all lectures, the bitrate should be in the same ballpark, it would be surprising if two videos of the same duration with the same parameters (resolution, framerate, etc.) recorded with the same device end up being compressed as 122MB and 240MB with the same compression settings. Of course if one is 20min and the other 40min that's normal. The lighting conditions could affect the level of digital noise and hence the video bitrate, but not by a factor 2.
    Yeah, I am not entirely clear on those 2 files and why that happened. I can see in the ones I get now but generally it's same lighting and stuff. Either way, I am not going with audio compression anymore as mentioned below.
    Originally Posted by abolibibelot View Post
    If the audio is already in a compressed format (which it most likely is – you can check this with MediaInfo) then you should just copy the streams, no need to recompress if no further processing is required (see below). Based on the script above (with /R for "recursive", a resolution of 960x720 and "-crf 25" for more video compression) :

    Code:
    for /R %%a in ("*.mts") do (
        md "D:%%~pa"
        ffmpeg -i "%%a" -vf scale=w=960:h=-4 -c:v libx264 -preset medium -crf 25 -c:a copy "D:%%~pna.mp4"
    )
    pause
    THANKSSSS! I am using this code now with the preset to veryfast. It's amazing. The audio is exactly same since it's just a copy. Filesize in a quick comparison is definitely bigger (113 MB vs 179 MB for aac 128k vs copy res down to 720p). So far, I have reduced a 135 GB folder to around 15 GB and I am getting a tenth of the old file sizes usually. I am really happy with this!

    Originally Posted by abolibibelot View Post
    It's surprising that you get better audio with "-c:a aac -b:a 128k", than with no specified audio setting (as in the first command line shown), since this is the default ffmpeg setting for MP4 as far as I know, so the audio output should be the same.
    Hmm maybe my ears are just not the best haha.
    Originally Posted by abolibibelot View Post
    A good reason to recompress the audio would be to make it "more audible" if it's too weak in the original recordings, with a normalization or dynamic range compression processing. In this case, for optimal quality, it would be better to use QAAC rather than ffmpeg's internal AAC encoder, or at least use a higher bitrate than 128k to preserve the audio signal as much as possible. There's a --normalize option in QAAC which may be enough for that purpose, but if there are already peaks near the limit (due for instance to someone coughing or clapping hands next to the recording device) then it won't do much, in this case a more advanced dynamic range compression processing would be needed, I don't know if there's a way to automate it efficiently.
    I found the audio audible enough to be honest and just want to keep it simple so I will skip this. But thanks for the information.

    I now have a simple code that does the job better than I ever imagined! Big thanks to both of you! Will let you know if I have anymore problems/questions.
    Quote Quote  
  7. Originally Posted by bloodtalon View Post
    I am using this code now with the preset to veryfast.
    I was going to suggest you try the veryfast preset. It's about 3x faster than medium and usually gives slightly smaller files. The quality may be adequate, depending on the material. Veryfast loses a lot of noise/grain (and crf=23 is already bad at that) which can cause posterization artifacts, especially in dark areas. Moving edges will be a little rougher too. It's likely that neither of those issues will make much of a difference with lecture type material. But now you know what to look for.
    Last edited by jagabo; 11th Jul 2020 at 20:40.
    Quote Quote  
  8. Since I am looking for a reasonably fast process
    You could use Quick Sync, -c:v h264_qsv
    Quote Quote  
  9. If bloodtalon's profile is correct, using QS on a i7-4790K probably won't be much faster than x264 at veryfast. And it will be lower quality (per bitrate). It is less CPU intensive so he may want to try it anyway.
    Quote Quote  
  10. Originally Posted by jagabo View Post
    Originally Posted by bloodtalon View Post
    I am using this code now with the preset to veryfast.
    I was going to suggest you try the veryfast preset. It's about 3x faster than medium and usually gives slightly smaller files. The quality may be adequate, depending on the material. Veryfast loses a lot of noise/grain (and crf=23 is already bad at that) which can cause posterization artifacts, especially in dark areas. Moving edges will be a little rougher too. It's likely that neither of those issues will make much of a difference with lecture type material. But now you know what to look for.
    Yeah it's definitely one of the smallest file sizes I have seen. I dunno why but the original files had a lot of noise/grain so I am actually glad it reduced it. Not really looking for dark areas or movement but yes now that I know the concept, it's a lot easier to edit and work with in the future. Thanks!

    I just finished and the 800 GB library came down to 100! I am so glad. I did find out it terminated early because around 1/3rd of the files were actually .mov so I had to rerun the batch file with .mts changed to .mov in the first line. A quick google search seems to show there is no easy "or" operator in cmd though.

    Yeah it's an i7-4790K. You mean QS helps in weaker CPUs because it is less CPU intensive? Mine is pretty old, 4th gen...
    Quote Quote  
  11. Originally Posted by bloodtalon View Post
    it terminated early because around 1/3rd of the files were actually .mov so I had to rerun the batch file with .mts changed to .mov in the first line. A quick google search seems to show there is no easy "or" operator in cmd though.
    You can include multiple wildcards in a for loop:

    Code:
    for /R %%a in (*.mts *.mov) do ...
    But in this case you already processed the *.mts files so you want to use just *.mov in a second batch.

    Originally Posted by bloodtalon View Post
    Yeah it's an i7-4790K. You mean QS helps in weaker CPUs because it is less CPU intensive? Mine is pretty old, 4th gen...
    QS AVC encoding (depending on settings) might be a faster than x264/veryfast on your CPU. But it definitely will consume less power and the computer will probably run quieter while encoding. With a low end CPU (dual core, for example) it would make more of a difference.
    Quote Quote  
  12. Originally Posted by jagabo View Post
    Code:
    for /R %%a in (*.mts *.mov) do ...
    But in this case you already processed the *.mts files so you want to use just *.mov in a second batch.
    Thanks for that extension. I have saved it for the future but Yeah I just reran it with *.mov for the second batch and was done!

    Originally Posted by jagabo View Post
    QS AVC encoding (depending on settings) might be a faster than x264/veryfast on your CPU. But it definitely will consume less power and the computer will probably run quieter while encoding. With a low end CPU (dual core, for example) it would make more of a difference.
    I found my computer wasn't too loud or hot during the process and it went on for days. So I will probably stick to this for now. If I need faster, I can instead just change it to super or ultra fast.
    Quote Quote  
  13. I dunno why but the original files had a lot of noise/grain so I am actually glad it reduced it.
    It's common if it was shot indoor, in low light conditions, with a small sensor device.
    You could significantly reduce the size for a similar subjective quality by using Avisynth and a denoising filter.
    (Or perhaps ffmpeg has an embedded denoising filter ? If it does, how does it compare with Avisynth's filters ? When I asked about good denoising filters some years ago the main suggestions were SMDegrain and MCTemporalDenoise – have better filters been developped since then ?)
    But it would be more complex, and it would significantly increase the encoding time, so if you're satisfied with the current outcome, don't bother.
    Last edited by abolibibelot; 15th Jul 2020 at 18:50.
    Quote Quote  
  14. Originally Posted by abolibibelot View Post
    I dunno why but the original files had a lot of noise/grain so I am actually glad it reduced it.
    It's common if it was shot indoor, in low light conditions, with a small sensor device.
    You could significantly reduce the size for a similar subjective quality by using Avisynth and a denoising filter.
    (Or perhaps ffmpeg has an embedded denoising filter ? If it does, how does it compare with Avisynth's filters ? When I asked about good denoising filters some years ago the main suggestions were SMDegrain and MCTemporalDenoise – have better filters been developped since then ?)
    But it would be more complex, and it would significantly increase the encoding time, so if you're satisfied with the current outcome, don't bother.
    Haha Yeah I am more than satisfied with the speed and filesize reduction of the current settings. And those sound very complicated and since I don't care too much about video quality, I am not too irked to put in another week to it. Thanks for everything!
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!