Scaning a whole folder of mp4 videos for errors?

[Nico Darko]

Hi,

I have a lot of mp4 videos from a common source, and I suspect that some of them are corrupted/are missing frames.
What is the best/fastest way to check the entire folder for errors?
I think ffmpeg can do that, but I don't know what the proper command line in batch mode would be?

Thank you.

[Budman1]

There are a lot of errors in an MP4's, some may be playable and some may not be. I don't know of a test that catches all the errors.

I closest I can think of to at least suspect error filled videos is a CMD file :

@echo off
@echo.  > "error.txt"
for %%a in ("*.mp4","*.mkv") Do (
echo "%%~dpna" >> "%%~dpaerror.txt"
echo "%%~dpnxa"
ffmpeg.exe -v error -i "%%a" -f null - >>error.txt 2>&1
cd "%%~dpa"
@echo. >> error.txt
)

Of course ffmpeg and this cmd file should be in the same folder. You will get and output in error.txt like the following. The titles with no errors below are 'good'?

"C:\Users\Bud\Desktop\New folder\Aspect_0_0_xInstagram"

"C:\Users\Bud\Desktop\New folder\Virtual Insanity Upscaled 480-720-1440-CLIP"

"C:\Users\Bud\Desktop\New folder\VideoTest"

"C:\Users\Bud\Desktop\New folder\VideoTest_W_Sound"
[null @ 0797b040] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2 >= 2
[null @ 0797b040] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 6 >= 6
[null @ 0797b040] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 9 >= 9
[null @ 0797b040] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 12 >= 12
[null @ 0797b040] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 16 >= 16

[Nico Darko]

Thanks, it's half working. I tried two mp4 files that have different issues.
The first one freezes for a few seconds with audio issues as well, but strangely no error was found.
The second one has missing frames, parts of the frame freeze for a second, errors were found for that one.

So I tried scanning other files, it did find similar errors for some, but how do I know where in the timecode the error is?

example:

"F:\Divers\H515"
[h264 @ 000001a5b9d09680] Reference -1 >= 16
[h264 @ 000001a5b9d09680] error while decoding MB 0 16, bytestream 12857
[h264 @ 000001a5b9d08d40] left block unavailable for requested intra4x4 mode -1
[h264 @ 000001a5b9d08d40] error while decoding MB 0 15, bytestream 22137
[null @ 000001a5b9ce11c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 154414 >= 154414

[jagabo]

The first one freezes for a few seconds with audio issues as well, but strangely no error was found.

Are you sure that's an error, not a "feature" -- ie, just an incompatablity with your player? Also, the process won't detect errors that occurred before the encoding. For example, a VHS cap may have many corrupt frames but to the encoder that's just detail to be compressed. And to the decoder it's just detail to display.

[Nico Darko]

The first one freezes for a few seconds with audio issues as well, but strangely no error was found.

Are you sure that's an error, not a "feature" -- ie, just an incompatablity with your player? Also, the process won't detect errors that occurred before the encoding. For example, a VHS cap may have many corrupt frames but to the encoder that's just detail to be compressed. And to the decoder it's just detail to display.

Yes, those specific files were downloaded from a website with youtube-dl-gui, and some of them were badly downloaded it seems, and I'd like to identify which ones. They play the same way on my player and my computer.

The other files, that actually showed errors with that command line, are dvdrips, but not ripped perfectly it would seem.

[Budman1]

There are a lot of errors in an MP4's, some may be playable and some may not be. I don't know of a test that catches all the errors.

IMHO MP4 encoding is very widely used but complexly encoded. As Jagabo noted, badly spliced, badly encoded, and a whole list of other operations may cause errors noticed only at playback. BY noting some of the list of errors you can often determine what the playback will be like. However many playback errors will not show during the test above unless you are checking in depth with ffprobe, PTS time, etc

Freeze frame is usually caused by an incorrect start point and PTS time. E.G. If you cut a video with FFMpeg and some other editors, half way between Key 'I' Frames that are 10 seconds apart, the video will cut there but freeze frame for whatever time it takes to play from the cut point until the following I frame is reached. You can often check this using FFPROBE PTS output.

None of the following show errs except during playback.

"C:\Users\Bud\Desktop\New folder\0_0_xInstagram.mp4"
"C:\Users\Bud\Desktop\New folder\new,mp4"
"C:\Users\Bud\Desktop\New folder\ALIENTheatrical (1)-002.mkv"
"C:\Users\Bud\Desktop\New folder\merged_T0_0_xInstagram.mp4"

Correctly cut 60 GOP, no freeze 0_0_xInstagram.mp4"
No. pts_time type
0 0.000000 1 I
1 0.033333 0 B
2 0.066667 0 B
3 0.100000 0 B
4 0.133333 0 P
5 0.166667 0 B


Badly cut 60 GOP at 90 seconds, 0.9 seconds freeze: new,mp4"
No. pts_time type
0 1.800000 1 I
1 1.833333 0 B
2 1.866667 0 P
3 1.900000 0 B
4 1.933333 0 P
5 1.966667 0 B

Less than 1 second start time you will hardly ever notice but will cause a glitc mid video.: ALIENTheatrical (1)-002.mkv
No. pts_time type
0 0.084000 1 I
1 0.125000 0 B
2 0.167000 0 B
3 0.209000 0 B
4 0.250000 0 P
5 0.292000 0 B

Freeze frame mid video from bad splice: merged_T0_0_xInstagram.mp4
57 1.922982 0 B next frame 0.033333
58 1.956315 0 B next frame 0.033333
59 1.989648 0 P next frame 0.518494
60 2.414974 1 I next frame 0.033333
61 2.448307 0 B next frame 0.033333

[Nico Darko]

Ok. But for the errors it was actually able to detect, how do I get the timecode of these errors?

[h264 @ 000001a5b9d09680] error while decoding MB 0 16, bytestream 12857
[h264 @ 000001a5b9d08d40] error while decoding MB 0 15, bytestream 22137

[Nico Darko]

@Budman1: Do you know if there's a way to know the timecode of these errors? thanks.

[Budman1]

Yes. The codes I posted in an earlier post are from a batch code I use to get pts for vid ed os as well as a program I am writing to cut frame accurate segments. I will post later.
The batch creates an file which all information about a video plus an abbreviated te t files as
No. pts_time type
0 0.084000 1 I
1 0.125000 0 B
2 0.167000 0 B
3 0.209000 0 B
4 0.250000 0 P
5 0.292000 0 B

[Nico Darko]

So we can modify this code to include the timecode in the txt file? that you'll post later? I'm not sure I got what you were saying

@echo off
@echo. > "error.txt"
for %%a in ("*.mp4","*.mkv") Do (
echo "%%~dpna" >> "%%~dpaerror.txt"
echo "%%~dpnxa"
ffmpeg.exe -v error -i "%%a" -f null - >>error.txt 2>&1
cd "%%~dpa"
@echo. >> error.txt
)

[Budman1]

OOPs wrong Post

[Budman1]

OOps wrong post

[Nico Darko]

The 3 txt files are created but they're empty.
I've downloaded MediaInfo.exe and MediaInfo.dll, but maybe I got the wrong files?

[Budman1]

Ok. But for the errors it was actually able to detect, how do I get the timecode of these errors?

Use this batch:

setlocal enabledelayedexpansion
@echo off
::for %%a in (*.mp4,*.mpg,*.flv) Do (
for %%a in ("%~nx1") Do (
set /a count=0
cd %%~dpa
echo frame,media_type,stream_index,key_frame,pkt_pts,pkt_pts_time,pkt_dts,pkt_dts_time,best_effort_timestamp,^
best_effort_timestamp_time,pkt_duration,pkt_duration_time,pkt_pos,pkt_size,^
Width,Height,pix_fmt,sample_aspect_ratio,pict_type,coded_picture_number,display_picture_number,^
interlaced_frame,top_field_first,repeat_pict,color_range,color_space,color_primaries,color_transfer,^
chroma_location > "%%~na_ffprobe.csv"
echo No.  pts_time  type > "%%~na_AllFrames.txt"
ver > nul
set /a Number=0
ffprobe.exe -v quiet -select_streams v:0 -print_format csv -show_entries frame "%%~nxa"  >> "%%~na_ffprobe.csv"
for /F "tokens=4,6,18,19 delims=," %%b in ('findstr "video" "%%~na_ffprobe.csv"') do (
set "x=%%b"
set "y=%%c"
set "z=%%d"
set "w=%%e"
set "v=%%f"
set "u=%%g"
set sort=!y:~0,-7!
::if !sort! GEQ 0 if !sort! LEQ 10 echo !count!  !y:~0,13!  !x! !w:~0,1! >> "%%~na_AllFrames.txt"
echo !count! !y:~0,13!  !x! !w:~0,1! >> "%%~na_AllFrames.txt"
set /a count+=1
 set /a ekko=count%%100
 if !ekko! EQU 0 echo Frame !count! processed
)
)
rem pause

It will create a CSV file with all information and a TXT file with just frame numbers, PTS times and their type (I. P, B). I have it so it lists every 100 frames so you know where its at. It will have a pause when first starting so be patient.

[Nico Darko]

Ok, the scan worked but I don't see any error. The previous batch found at least these for the same file:

[h264 @ 000001a5b9d09680] error while decoding MB 0 16, bytestream 12857
[h264 @ 000001a5b9d08d40] left block unavailable for requested intra4x4 mode -1
[h264 @ 000001a5b9d08d40] error while decoding MB 0 15, bytestream 22137
[null @ 000001a5b9ce11c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 154414 >= 154414

But all I see with this one is those numbers, what do they mean? how do I see where any error was found?

No. pts_time type
0 0.000000 1 I
1 0.033367 0 B
2 0.066733 0 B
3 0.100100 0 B
4 0.133467 0 B
5 0.166833 0 B
6 0.200200 0 B
7 0.233567 0 B
8 0.266933 0 B
9 0.300300 0 P
10 0.333667 0 B
11 0.367033 0 B
12 0.400400 0 B
13 0.433767 0 B
14 0.467133 0 B
15 0.500500 0 B
16 0.533867 0 B
17 0.567233 0 B
18 0.600600 0 P
19 0.633967 0 B
20 0.667333 0 B
21 0.700700 0 B
22 0.734067 0 B
23 0.767433 0 B
24 0.800800 0 B
25 0.834167 0 B
26 0.867533 0 B
27 0.900900 0 P
28 0.934267 0 B
29 0.967633 0 B
30 1.001000 0 B
31 1.034367 0 B
32 1.067733 0 B
33 1.101100 0 B
34 1.134467 0 B
35 1.167833 0 P
36 1.201200 0 B
37 1.234567 0 B
38 1.267933 0 B
39 1.301300 0 B
40 1.334667 0 B
41 1.368033 0 P
42 1.401400 0 B
43 1.434767 0 B
44 1.468133 0 B
45 1.501500 0 P
46 1.534867 0 B

etc

[Budman1]

Left number under no. Is frame number. Decimal is pts time. Column of 1 or 0 means key frame/not key frame. Last column is the type of that frame.

Example from you list:
35 1.167833 0 P
Frame 35
Pts time 1.16 ppl7833
0=not key frame I
P=frame is P type

[Nico Darko]

Right, but why does it show no error when the previous code did?
And if it did, I still don't understand where the timecode would be?

Sorry I have a pretty basic knowledge of these things, this is very technical.

[Budman1]

I really have no idea why the error disappeared. If it is truly the original and unchanged, the error should show again unless you do not have the files in the same folder as the batch.

The batch essentially converts the original but sends it to null(nowhere) but records error data.
If the file was there not listed as error, that would suggest it wasnt tested.

Also, knowing the frame and stream of error doesnt make any difference since, as far as I know, you cannot correct individual frames. Encoding usually does that by replacing non sequential with sequential.