Using FFMPEG to check integrity of video files

[gmc9960]

I am currently going through the exercise of checking the integrity of all the video files I have on my Windows PC.

There are many thousands of files to examine.

I am using FFMPEG and the command below to check the file which I understand is outputing the data to a null (empty) output file and writing the result of the run to the log file. Any files that are OK have an empty log file and any with errors have details of the error.

For example:

C:\DATA\VideoIntegrity\ffmpeg -v error -i "d:\data\TV\File1.mkv" -map 0:1 -f null - 2> "C:\Data\VideoIntegrity\Result\File1.mkv.log"

The log file contains something like:
[matroska,webm @ 000001faefa5b880] Length 5 indicated by an EBML number's first byte 0x0a at pos 840867904 (0x321ea040) exceeds max length 4.
[matroska,webm @ 000001faefa5b880] Element at 0x322002e5 ending at 0x3ced794a exceeds containing master element ending at 0x322dd29f
Error while decoding stream #0:1: Error number -16976906 occurred

etc

Obviously examining each file takes some time but to check a folder containing 24 videos takes between 4 and 15 minutes depending on the size of the video files.

These times are fine and everything is working OK.

Where I am having a problem is with checking files with HEVC video stream.

To check 24 files encoded with HEVC it is taking anything from two to three hours to check 24 files which are about 2Gb each.

I am assuming that some sort of re-encoding is going on here.

I was wondering if anyone had any suggestions of what I can do to speed it up.

As I say; all I want to do is just to check the integrity of the file and write the result to a log file. I don't want to change anything from the input file.

Thanks

[Selur]

Try adding '-c:v rawvideo', this should tell ffmpeg that the output is rawvideo and thus just the decoding should be done.

[sneaker]

Log everything if you are unsure about what exactly ffmpeg is doing. My guess is your HEVC files simply have higher decoding complexity (HEVC > AVC, 4K > HD, 10 bit > 8 bit, higher bitrate, more fps). Make sure you are using 64 bit ffmpeg when doing HEVC software decoding. (Maybe it's worth checking hardware decoding if your HW is capable? Albeit I don't know how verbose hardware decoding is in terms of decoding errors.)

About your command-line:

-map 0:1

This will map only the 2nd track (ffmpeg starts counting at zero). Check if this is the video or the audio track. If you are decoding audio for one but video for the other files it may explain the speed difference. (Again: check the complete ffmpeg log to see what exactly it is doing.)
https://trac.ffmpeg.org/wiki/Map
Then again idk what kind of errors you are looking for? Video, audio, container damage?

[gmc9960]

Thanks for all the information:

The only errors I am looking for are those that will stop a video playing. The kind where you have lots of pixelated frames; where the video freezes and has difficulty jumping around the playback time or crashes out before the end. I don't care about any other types of error.

I will look into sneaker's comments tomorrow. You mention -map 0:1 will only map the second track; having looked at the FFMPEG documentation it seems to imply that if I omit -map completely it will pick up what is most appropriate which would seem the best option.

Anyway; I have done some tests and the results seem a bit odd as the h264 file is slower but HEVC slightly faster

Does it matter where parameters are placed; I specified ffmpeg -v error -i "D:\Video Conversion\VideoIntegrityTests\File1.mkv" -map 0:1 -c:v rawvideo -f null - 2> "C:\Data\VideoIntegrity\FailedFilesTemp\file1. log"

File 1 264 file

Run without -c:v rawvideo time: 03:48
Run with -c:v rawvideo time: 04:21

containerFormat: Matroska,
videoFormat: AVC,
videoCodecID: V_MPEG4/ISO/AVC,
videoProfile: High@L4.1,
videoCodecLibrary: x264 - core 148,
videoBitrate: 7000000,
videoBitDepth: 8,
videoMultiViewCount: 0,
videoColourPrimaries: BT.709,
videoTransferCharacteristics: BT.709,
width: 1920,
height: 1080,
audioFormat: AAC,
audioCodecID: A_AAC-2,
audioCodecLibrary: ,
audioAdditionalFeatures: LC,
audioBitrate: 0,
runTime: 00:48:17.3980000,
audioStreamCount: 1,
audioChannels: 6,
audioChannelPositions: 3/2/0.1,
audioChannelPositionsText: Front: L C R, Side: L R, LFE,
audioProfile: ,
videoFps: 23.976,
audioLanguages: ,
subtitles: ,
scanType: Progressive,
schemaRevision: 5


File 2 HEVC File

Run without -c:v rawvideo time: 06:24
Run with -c:v rawvideo time: 05:11

containerFormat: Matroska,
videoFormat: HEVC,
videoCodecID: V_MPEGH/ISO/HEVC,
videoProfile: Main@L4@High,
videoCodecLibrary: x265 - :[Windows][MSVC 1800][64 bit],
videoBitrate: 0,
videoBitDepth: 8,
width: 1920,
height: 1080,
audioFormat: AAC,
audioCodecID: A_AAC,
audioCodecLibrary: ,
audioBitrate: 0,
runTime: 00:45:44.7030000,
audioStreamCount: 1,
audioChannels: 2,
audioChannelPositions: 2/0/0,
audioChannelPositionsText: Front: L R,
audioProfile: LC,
videoFps: 23.976,
audioLanguages: English,
subtitles: ,
scanType: ,
schemaRevision: 4

[Selur]

wild guess: might be a virus scanner or similar interfering.
Normally ' -c:v rawvideo ' should not be needed, since '-f null' without any filters etc. should tell ffmpeg to only use the decoder and demuxer,... (https://trac.ffmpeg.org/wiki/Null)
If you run this on a machine with nvidia hardware decoding support, you can try whether adding '-hwaccel cuda' (https://trac.ffmpeg.org/wiki/HWAccelIntro) helps,...

[gmc9960]

Thanks for all the suggestions/comments.

I do have an Nvidia graphics card although it is only a GeForce 750

I disabled my virus scanner when running the tests

I noticed that there is an option to specify the hwaccel_output so I have done a test specifying that too.

I amended the map parameter to -map 0

I have done a bit more work and the following are the results:

All tests also include the following -v error and -map 0 -f null

H264 File

(1) -hwaccel cuda -hwaccel_output_format cuda time taken - 02:08
(2) -hwaccel cuda - time taken - 02:39
(3) No hardware acceleration - time taken - 03:36

Log file shows no errors

HEVC File

(1) -hwaccel cuda -hwaccel_output_format cuda - time taken - 05:18
(2) -hwaccel cuda - time taken - 05:09
(3) No hardware acceleration - time taken - 05:13

Log file when using the cuda parameters (1) and (2) has errors:

Hardware is lacking required capabilities
Failed setup for format cuda: hwaccel initialisation returned error.