Playback Issue: Source Video with Inconsistent FPS

[ctr2yellowbird]

Hello.

I have a decent understanding of my issue, but solutions are not forthcoming.

The short version is that I recorded some PC gaming footage at 60 FPS, except for one 24 second loading screen that was recorded at 30 FPS. The source AVI video plays back correctly, but my post-encode MKV file now plays back the 30 FPS segment at 60 FPS and seemingly stretches the rest of the video to accommodate the "gained" 12 seconds, resulting in desync'd audio after the loading screen passes, which of course slowly resyncs as the video progresses.

Now for the specifics:

The source video was recorded with MSI Afterburner's video capture feature using MJPG compression and configured for 60 FPS with the AVI container.
I've been playing the videos back only through MPC-HC 1.7.10 with madVR, ReClock, and the internal LAV Splitters.
I've only been encoding the video with x264 through MeGUI, accepting updates as they popped up. The audio was encoded with Nero AAC. The source file was indexed using MeGUI's file indexer.
I've tried muxing the encoded files with MeGUI's MKV Muxer and MKVToolnix 8.6.1.

I've compared both the source and output videos using MeGUI's video preview feature, and both videos have 72625 frames with the problem loading screen starting at frame 655 and ending at frame 1368 in both (713 frames/60 FPS = 11.88s). In MPC-HC, the source video shows the loading screen from 0:11 to 0:35 (I don't know how to check exact frame position in MPC-HC), whereas the encoded video shows the loading screen from 0:11 to 0:24. The encoded video's audio is in sync for those first 11 seconds. What's more, Afterburner's on-screen display shows roughly 30 FPS during loading screens in that game, which seems hard-coded. Given those observations, I reached the conclusion stated in my "short version" above.

MediaInfo of source file from MPC-HC

General
Complete name                  : W:\MassEffect3_2015_12_18_11_47_33_009.avi
Format                         : AVI
Format/Info                    : Audio Video Interleave
Format profile                 : OpenDML
File size                      : 17.9 GiB
Duration                       : 20mn 44s
Overall bit rate               : 123 Mbps

Video
ID                             : 0
Format                         : JPEG
Codec ID                       : MJPG
Duration                       : 20mn 44s
Bit rate                       : 119 Mbps
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate                     : 60.000 fps
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.955
Stream size                    : 17.2 GiB (96%)

Audio
ID                             : 1
Format                         : PCM
Format settings, Endianness    : Little
Format settings, Sign          : Signed
Codec ID                       : 00001000-0000-0100-8000-00AA00389B71
Duration                       : 20mn 44s
Bit rate mode                  : Constant
Bit rate                       : 4 608 Kbps
Channel(s)                     : 6 channels
Channel positions              : Front: L C R, Back: L R, LFE
Sampling rate                  : 48.0 KHz
Bit depth                      : 16 bits
Stream size                    : 683 MiB (4%)
Alignment                      : Aligned on interleaves
Interleave, duration           : 17 ms (1.00 video frame)

MediaInfo of encoded and muxed file from MPC-HC

General
Unique ID                      : 234967376368943299184329705214647374993 (0xB0C512138458515DBC5D8EB63B669491)
Complete name                  : W:\MassEffect3_2015_12_18_11_47_33_009.avi-muxed.mkv
Format                         : Matroska
Format version                 : Version 2
File size                      : 4.35 GiB
Duration                       : 20mn 44s
Overall bit rate               : 30.1 Mbps
Encoded date                   : UTC 2015-12-18 21:48:35
Writing application            : mkvmerge v8.5.2 ('Crosses') 32bit
Writing library                : libebml v1.3.3 + libmatroska v1.4.4
DURATION                       : 00:20:44.172000000
NUMBER_OF_FRAMES               : 53582
NUMBER_OF_BYTES                : 121608815
_STATISTICS_WRITING_APP        : mkvmerge v8.5.2 ('Crosses') 32bit
_STATISTICS_WRITING_DATE_UTC   : 2015-12-18 21:48:35
_STATISTICS_TAGS               : BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

Video
ID                             : 1
Format                         : AVC
Format/Info                    : Advanced Video Codec
Format profile                 : High@L4.2
Format settings, CABAC         : Yes
Format settings, ReFrames      : 3 frames
Codec ID                       : V_MPEG4/ISO/AVC
Duration                       : 20mn 44s
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate mode                : Constant
Frame rate                     : 58.366 fps
Original frame rate            : 58.374 fps
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Writing library                : x264 core 148 r2638 7599210
Encoding settings              : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=15 / keyint_min=8 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=21.0 / qcomp=0.60 / qpmin=10 / qpmax=40 / qpstep=3 / vbv_maxrate=34000 / vbv_bufsize=50000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Default                        : Yes
Forced                         : No

Audio
ID                             : 2
Format                         : AAC
Format/Info                    : Advanced Audio Codec
Format profile                 : LC
Codec ID                       : A_AAC
Duration                       : 20mn 44s
Channel(s)                     : 6 channels
Channel positions              : Front: L C R, Side: L R, LFE
Sampling rate                  : 44.1 KHz
Compression mode               : Lossy
Delay relative to video        : 10ms
Language                       : English
Default                        : Yes
Forced                         : No

The changes in framerate are concerning. As I'm looking over this before posting, I'm wondering if "Frame rate mode: Constant" isn't describing exactly my issue. Maybe if it said something like "variable" then I wouldn't be having this issue.

MeGUI's generated AviSynth script

LoadPlugin("S:\MeGUI_2525_x86\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("W:\MassEffect3_2015_12_18_11_47_33_009.avi")
#deinterlace
#crop
#resize
#denoise

I've tried adding AssumeFPS(60.000), which corrected the displayed framerate in MediaInfo at the least, but ultimately that didn't solve the major issue.

What I want is to determine the exact cause of the problem if possible, how to correct the issue with my source video without editing tools if possible, and how to avoid this issue in the future. My end goal is to have a compressed, proper version of my source video that is ready for YouTube if I decide to upload it.

I get the feeling that my issue is not directly with x264's encoding process since the source and output video files appear to have the same number of frames in the same positions. I'm hoping the issue is a container problem or something that can be resolved by defining or changing a specific switch or field somewhere. Even better would be an issue with my configuration of MPC-HC. I just don't know. I'm not particularly experienced or knowledgeable with these matters beyond what I've shown in this post.

I'm hoping that someone with more experience reads this and notices "well, there's your problem", and that it's a simple enough oversight caused by ignorance.

Thanks for any help.

[manono]

Can't you just change the 30fps part to 60fps and rejoin it to the rest? Something like:

X=Last
A=X.Trim(0,654).AssumeFPS(60)
B=X.Trim(655,1368).ChangeFPS(60)
C=X.Trim(1369,0).AssumeFPS(60)
A+B+C

That doesn't really answer your question, but I know nothing (and care even less) about gameplay captures.

[Cornucopia]

Well there's your problem: you want to edit -without using editing software! I'm guessing it's because you assume you cannot edit without re-encoding. Depends on the source, but it should be very possible to do what you want with ffmpeg, AVISynth, and other tools (and yes it should take multiple steps).

In this instance, however, re-encoding wouldn't be the worst thing (if done smartly), so even less reason to avoid editors.

Scott

[hello_hello]

AVIs can only be constant frame rate, however if there's null frames L-Smash and FFMSIndex both treat them as variable frame rate and output the average frame rate. I don't understand the logic behind it when it comes to AVIs.

For L-Smash, try changing the script to this (it'll cause L-Smash to duplicate frames as required):

LWLibavVideoSource("W:\MassEffect3_2015_12_18_11_4 7_33_009.avi", fpsnum=60, fpsden=1)

Or, open the AVI using AVISource. It always assumes the correct constant frame rate (not the average frame rate). To do so using MeGUI, open the Script Creator and select the AVI as the input video. When MeGUI asks how you want to open it, select AVISource.
AVISource relies on you having the appropriate VFW codec installed to decode the video (ffdshow's VFW decoder should decode it).

If you were to remux the AVI as an MKV with MKVMergeGUI (without specifying a frame rate) and use that MKV as the source video, you could index with FFMSIndex and MeGUI should automatically add the frame rate conversion to the script (if it can determine the correct frame rate). It doesn't when indexing with L-Smash.
Alternatively you could re-encode the MKV (the one remuxed from your AVI) without adding frame rate conversion to the script, and use MKVCleaver or gMKVExtractGUI to extract the video timecodes from the source MKV. Then you could remux the encoded video with MKVMergeGUI while using the extracted timecodes for the video, which should then give your encoded video the same "variable" frame rate as the source.

Personally, I'd use the frame rate conversion method to output a constant frame rate, although every so often it can cause odd things to happen (frames dropped or repeated when they shouldn't be). If that happens, try a different indexer or try remuxing as an MKV first and then trying a different indexer etc.

[theewizard]

#1..why does the video contain (2) frame rates
Even if the source play's correctly on mpc, it's a total screw up for converting

#2 why record in mjpeg, if you do not intend to edit

[ctr2yellowbird]

Thanks for the responses.

Can't you just change the 30fps part to 60fps and rejoin it to the rest? Something like:

X=Last
A=X.Trim(0,654).AssumeFPS(60)
B=X.Trim(655,1368).ChangeFPS(60)
C=X.Trim(1369,0).AssumeFPS(60)
A+B+C

I tried that. MeGUI's video preview showed that 20 frames total would be added to the video, just during the loading scene, but the encoded video was still no good.

Well there's your problem: you want to edit -without using editing software! I'm guessing it's because you assume you cannot edit without re-encoding. Depends on the source, but it should be very possible to do what you want with ffmpeg, AVISynth, and other tools (and yes it should take multiple steps).

In this instance, however, re-encoding wouldn't be the worst thing (if done smartly), so even less reason to avoid editors.

Scott

It seems my use of the word "edit" was a misnomer. I was mostly referring to changes that happen before or after encoding rather than including changes that occur during encoding to my definition of "editing", but perhaps I shouldn't. I was trying to say that I want to avoid making changes such as cutting the problematic loading scene from my video and audio and compensating elsewhere if needed, since that seems more like treating the symptoms rather than the cause.

AVIs can only be constant frame rate, however if there's null frames L-Smash and FFMSIndex both treat them as variable frame rate and output the average frame rate. I don't understand the logic behind it when it comes to AVIs.

For L-Smash, try changing the script to this (it'll cause L-Smash to duplicate frames as required):

LWLibavVideoSource("W:\MassEffect3_2015_12_18_11_4 7_33_009.avi", fpsnum=60, fpsden=1)

Or, open the AVI using AVISource. It always assumes the correct constant frame rate (not the average frame rate). To do so using MeGUI, open the Script Creator and select the AVI as the input video. When MeGUI asks how you want to open it, select AVISource.
AVISource relies on you having the appropriate VFW codec installed to decode the video (ffdshow's VFW decoder should decode it).

That seems good. I want to try the AVISource route, but I'm getting an error upon selecting AVISource for indexing the source file:

Direct Show Error

Unable to render the file.
You probably don't have the correct filters installed.

I was getting a different error before installing ffdshow. How I can resolve this?

In the meantime, I'll try your L-SMASH suggestion. MeGUI's video preview shows 74648 total frames (+2023) with that script.

#2 why record in mjpeg, if you do not intend to edit

2. When I first tried out Afterburner's video capture options a few years ago, MJPG had a comparatively minimal performance impact (in-game) for its video quality and file size. Except for my CPU, my hardware has all improved an order of magnitude since then, so what would you suggest for recording 1080p at 60 FPS?

[manono]

I tried that. MeGUI's video preview showed that 20 frames total would be added to the video, just during the loading scene, but the encoded video was still no good.

You'll have to be a little more precise about 'the encoded video was still no good'. Frames are supposed to be added, but the length is supposed to remain the same after being reencoded through MeGUI. The audio should remain in synch. Is that what you mean by the video being no good? The audio still goes out of synch?

You said, though, the 30fps part was 713 frames so that count is supposed to have been doubled (713 new frames). Either the information you gave us is wrong (that part isn't 30fps, but much more), or you didn't follow the instructions.

[hello_hello]

Direct Show Error

Unable to render the file.
You probably don't have the correct filters installed.

I was getting a different error before installing ffdshow. How I can resolve this?.

Open the ffdshow VFW encoder/decoder configuration (the one with the yellow icon) and look for MJPEG in the list of decoders. Is it enabled?

[hello_hello]

You said, though, the 30fps part was 713 frames so that count is supposed to have been doubled (713 new frames). Either the information you gave us is wrong (that part isn't 30fps, but much more), or you didn't follow the instructions.

I'm not sure if it'd work as you described if the decoder is outputting the average frame rate. I assume all the 60fps frames are there, so it's just a matter of telling Avisynth to increase them to 60fps, but if for the 30fps parts, every second frame is effectively missing, and the average frame rate is being used for the whole video, they're probably whizzing by way too fast, so I think you'd need to first decrease the frame rate to 30fps, then change the frame rate to 60fps. ie:

X=Last
A=X.Trim(0,654).AssumeFPS(60)
B=X.Trim(655,1368).AssumeFPS(30).ChangeFPS(60)
C=X.Trim(1369,0).AssumeFPS(60)
A+B+C

I could be wrong, but without AssumeFPS(30) before ChangeFPS(60) I think you'd end up with the wrong number of frames. Maybe.....

[ctr2yellowbird]

AVIs can only be constant frame rate, however if there's null frames L-Smash and FFMSIndex both treat them as variable frame rate and output the average frame rate. I don't understand the logic behind it when it comes to AVIs.

For L-Smash, try changing the script to this (it'll cause L-Smash to duplicate frames as required):

LWLibavVideoSource("W:\MassEffect3_2015_12_18_11_4 7_33_009.avi", fpsnum=60, fpsden=1)

So that worked. With that line in the AviSynth script, the resulting encoded video played back the troublesome loading screen near the beginning of the video at the correct ~24 second duration. The rest of the video aligned with the audio, as it had in the source video. No desync. I would still like to get AVISource working though.

I tried that. MeGUI's video preview showed that 20 frames total would be added to the video, just during the loading scene, but the encoded video was still no good.

You'll have to be a little more precise about 'the encoded video was still no good'. Frames are supposed to be added, but the length is supposed to remain the same after being reencoded through MeGUI. The audio should remain in synch. Is that what you mean by the video being no good? The audio still goes out of synch?

The video was still badly desync'd from the audio, similar to my previous efforts.

You said, though, the 30fps part was 713 frames so that count is supposed to have been doubled (713 new frames). Either the information you gave us is wrong (that part isn't 30fps, but much more), or you didn't follow the instructions.

I provided my observations in the third longer paragraph in the OP that led me to believe that section of the video was 30 FPS. When I play back the source AVI, the loading screen takes ~24 seconds to pass, but that same loading screen only takes ~12 seconds to pass in my original encoded attempts, which was roughly what I saw from the encoded video produced from your suggestion. I agree that the actual result differed drastically from the expected result. Based on hello_hello's mention of null frames, I'm now assuming that the null frames weren't presented in MeGUI's video preview of the source AVI, which I was using to measure total frame counts and positions, and that they were somehow discarded or ignored during the normal x264 encoding process. Of course, I've not heard of null frames or the AVI behavior hello_hello described until today, but that explanation seems to make sense from my current perspective.

The script that I should have used based on your instructions was:

LoadPlugin("S:\MeGUI_2525_x86\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("W:\MassEffect3_2015_12_18_11_47_33_009.avi")
#deinterlace
#crop
#resize
#denoise
X=Last
A=X.Trim(0,654).AssumeFPS(60)
B=X.Trim(655,1368).ChangeFPS(60)
C=X.Trim(1369,0).AssumeFPS(60)
A+B+C

Is that not what you suggested I try? ...or was I supposed to perform each Trim() function as part of a separate script? I wouldn't have known if you meant the latter.

I must admit, however, that I did commit an oversight in describing the problem in the OP. I had in fact recorded two loading screens in my video, one near the beginning of the video as the game started and another near the end as the game finished. Both loading screens play back at about twice normal speed from my earlier video encode attempts. In the source AVI, the second loading screen starts at 20:24 and ends at 20:38 in MPC-HC, as opposed to starting at 20:31 and ending at 20:38 in my original desync'd MKV encode. According to MeGUI's video preview of the source AVI, the second loading screen appears at frame 71862 and ends at frame 72284.

I apologize for the omission. The second loading screen completely slipped my mind once I noticed the anomaly in the first.

[manono]

I could be wrong, but without AssumeFPS(30) before ChangeFPS(60) I think you'd end up with the wrong number of frames. Maybe.....

Worth a try.

[ctr2yellowbird]

Direct Show Error

Unable to render the file.
You probably don't have the correct filters installed.

I was getting a different error before installing ffdshow. How I can resolve this?.

Open the ffdshow VFW encoder/decoder configuration (the one with the yellow icon) and look for MJPEG in the list of decoders. Is it enabled?


[Attachment 34917 - Click to enlarge]

MJPEG was not enabled. It is now, but I'm still getting the same error. I restarted too if that matters.

I will try your other suggestions tomorrow.

Thanks again for the replies.

[hello_hello]

If you're really keen to see what's happening, one way is to remux the AVI as an MKV with MKVMergeGUI and extract the video timecodes as a text file with MKVCleaver. You'll have a text file like this:

# timecode format v2
0
17
33
50
67
83
100
117
133
150
167
183
200
217
233
250

For 60fps video the frames should be 1000/60 = 16.6667ms apart. They'd be rounded to the nearest millisecond but if you're keen you can look for larger than 17ms gaps between frames.

You could also try opening the video in a DirectShow player such as MPC-HC with ffdshow decoding and it's on screen display enabled. I'm not sure how informative that'll be in respect to telling you what's going on, but it might be worth a look. From memory DirectShow will (naturally) play the file correctly but if you step through the frames one at a time you might be able to see what's happening.

I've got a small AVI where I know there's a missing/null frame (not the sample pictured). When I checked with ffdshow and stepped through the frames, the frame count went from 413 to 415 in the OSD where that "missing" frame is.



PS You can navigate to a specific frame in MPC-HC using the Navigate/GoTo menu.

PPS If you enable "RAW Video" in the ffdshow DirectShow decoder configuration hopefully it'll run and you can use it's filters and OSD regardless of the video you play and without needing to fiddle with MPC-HC's internal decoder at all.

[hello_hello]

MJPEG was not enabled. It is now, but I'm still getting the same error. I restarted too if that matters.

I will try your other suggestions tomorrow.

Thanks again for the replies.

I'm not sure why. I just tested with an AVI containing h264 video and enabling/disabling h264 in the ffdshow VFW decoder configuration allowed me to open the AVI via AviSource.... or not.... but I'm getting a "couldn't locate a decompressor for fourCC h264" error message when the h264 decoder is disabled..




Try enabling "Other MJPEG" decoder too in ffdshow. You never know.....
Which version of Windows are you using? Not that I think it should matter (I'm using XP).

[ctr2yellowbird]

Some updates and documentation.

I got AVISource to work. I had Shark007 codec pack installed, but I uninstalled that, reinstalled ffdshow (and Haali), tried AVISource in MeGUI again, and it worked.

AVISource produced the following AviSynth script

AVISource("W:\MassEffect3_2015_12_18_11_47_33_009.avi", audio=false).AssumeFPS(60,1)
#deinterlace
#crop
#resize
#denoise

With that script, MeGUI's video preview shows 74648 total frames from the source AVI, same as L-SMASH with "fpsnum=60, fpsden=1". Now, the first loading screen starts at frame 658 and ends at frame 2097 (1439 total frames/60 FPS = 23.98s). The resultant video encoded with that is sync'd correctly with the audio.

MKVCleaver timecodes:

10700 <------ Normal
10717
10733
10750
10767
10783
10800
10817
10833
10850
10867 <------ ~30 FPS loading screen begins?
10917
10950
10967
11000
11050
11083
11100
11133
11167

53800 <------ Normal
53817
53833
53850
53867
53883
53900 <------ Framedrop noted in screenshots
53933
53967
53983
54000

I got ffdshow working in MPC-HC by adding ffdshow raw video filter through the External Filters section and setting that to Prefer. I've taken several screenshots as examples of framerate behavior observed in the source AVI, "bad MKV", and "good MKV" that I will link to at the bottom of this post. I'm seeing basically what I expected. The source AVI has a mostly stable 60 FPS, except during the loading screens which are 30 FPS, and some occasional framedrops. The bad MKV has a stable 58 FPS, regularly fluctuating to 57 FPS, throughout the video including the loading screens. The good MKV has a stable 60 FPS with a constant, very minor jitter throughout the entire video. Both the good and bad MKVs seemed to smooth out any framedrops (example framedrop in the screenshots).

Stepping through the source AVI frame by frame during the first 30 FPS loading screen, I noticed constant 1/60 second gaps between frames. By that, I mean ffdshow reports a timestamp of 00:00:20.000 - 00:00:20.016 with a frame duration of 16.6667ms for frame 1200. Using Ctrl + right arrow to see the next frame, ffdshow now reports a timestamp of 00:00:20.033 - 00:00:20.050 with a frame duration of 16.6667ms for frame 1202. That seems to be exactly what hello_hello described. Neither MKV skips frames like that.

Lastly, I cut a sample of my source AVI using Solveig AVI Trimmer+, encoded it using the automatic scripts generated from AVISource and then L-Smash, muxed the two video encodes with the same audio encode, and uploaded all three of those videos. Each video presents the same behavior that I observed from the full length versions, except the sample "bad MKV" version now only plays at ~50 FPS. Seemed like the thing to do.

The video samples, MKVCleaver timecodes .txt, and ffdshow screenshots can be downloaded individually here for reference.

Should I still try the altered AviSynth script on the source AVI?

X=Last
A=X.Trim(0,654).AssumeFPS(60)
B=X.Trim(655,1368).AssumeFPS(30).ChangeFPS(60)
C=X.Trim(1369,0).AssumeFPS(60)
A+B+C

[hello_hello]

Should the MediaFire link work? I couldn't find anything to download by following it.

I'm a bit confused by the encode you said plays at 50fps, but that aside, it seems you have it all sorted now.

If adding the frame rate conversion to the script or using Avisource is fixing the problem, I probably wouldn't worry about the other script (sorry manono). Best case scenario is probably for the output to be the same as when you're using L-Smash's frame rate conversion, but chances are that won't happen and you'll need to fiddle with it to get it right.

[ctr2yellowbird]

The link was working for me yesterday, but now my hyperlink seems incorrect. I don't know what happened. 2nd try

One more time, thanks again for all your help.

Edit: Well, it's working right now.

[ndjamena]

Update your version of MediaInfo, I hate seeing the Matroska Statistics Tags displayed like that, it brings back bad memories.

(MakeMKV adds statistics tags now too, now we just need FFMPEG on board.)