Muxing/Wrapping Problem using ffmpeg

[SameSelf]

I decided to start a new thread due to some problems I am having. Here is a quick summary. I am frameserving 1920x1080p29.97 ProRes422 content from PPro using DMFS to an avisynth script that I then encode to an elementary H.264 video stream using x264.exe:

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
ConvertToYV12()

x264.exe --preset slow --tune film --crf 20 -o "output.m4v" "signpost.avs"

Therefore, no additional filters, resizing, etc. The output.m4v is simply 1920x1080p29.97 video that has been compressed and encoded to H.264. For completeness, here is the MediaInfo:

General
Complete name                            : E:\video.m4v
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 4.36 GiB

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.0
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 5 frames
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 29.970 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Writing library                          : x264 core 146 r2538 121396c
Encoding settings                        : cabac=1 / ref=5 / deblock=1:-1:-1 /
analyse=0x3:0x113 / me=umh / subme=8 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 /
me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 /
fast_pskip=1 / chroma_qp_offset=-3 / threads=12 / lookahead_threads=2 /
sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 /
constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 /
direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 /
intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 /
qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Next, using AME, I export the audio out of PPro as AAC MP4. Here is the MediaInfo for the audio stream:

General
Complete name                            : E:\audio.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42
File size                                : 61.4 MiB
Duration                                 : 26mn 54s
Overall bit rate mode                    : Variable
Overall bit rate                         : 319 Kbps
Encoded date                             : UTC 2015-06-05 14:26:33
Tagged date                              : UTC 2015-06-05 14:26:33

Audio
ID                                       : 1
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 26mn 54s
Bit rate mode                            : Variable
Bit rate                                 : 317 Kbps
Maximum bit rate                         : 418 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Compression mode                         : Lossy
Stream size                              : 61.1 MiB (100%)
Language                                 : English
Encoded date                             : UTC 2015-06-05 14:26:33
Tagged date                              : UTC 2015-06-05 14:26:33

So now I just want to wrap/mux the video and audio into a single MP4. I have tried using ffmpeg:

ffmpeg -r 30000/1001 -i video.m4v -i audio.mp4 -c:v copy -a:c copy muxed.mp4

But I get the following warning:

ffmpeg -r 30000/1001 -i video.m4v -i audio.mp4 -c:v copy -c:a copy muxed.mp4
ffmpeg version N-65018-gad91bf8 Copyright (c) 2000-2014 the FFmpeg developers
  built on Jul 26 2014 22:09:15 with gcc 4.8.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      52. 92.101 / 52. 92.101
  libavcodec     55. 69.100 / 55. 69.100
  libavformat    55. 49.100 / 55. 49.100
  libavdevice    55. 13.102 / 55. 13.102
  libavfilter     4. 11.102 /  4. 11.102
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
[h264 @ 0000000000350700] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from 'video.m4v':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 1920x1080, 29.97 fps, 29.97 tbr, 1200k tbn, 59.94 tbc
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'audio.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42avc1
    creation_time   : 2015-06-05 14:26:33
  Duration: 00:26:54.61, start: 0.000000, bitrate: 318 kb/s
    Stream #1:0(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      creation_time   : 2015-06-05 14:26:33
      handler_name    : Mainconcept MP4 Sound Media Handler

Then when I continue it keeps repeating the following:

[mp4 @ 00000000058281e0] pts has no value
    Last message repeated 471 times
[mp4 @ 00000000058281e0] pts has no valueB time=00:00:15.71 bitrate=26948.3kbits/s
    Last message repeated 508 times
[mp4 @ 00000000058281e0] pts has no valueB time=00:00:32.69 bitrate=18653.5kbits/s
    Last message repeated 499 times
[mp4 @ 00000000058281e0] pts has no valueB time=00:00:49.38 bitrate=15964.2kbits/s
    Last message repeated 486 times
[mp4 @ 00000000058281e0] pts has no valueB time=00:01:05.63 bitrate=17328.7kbits/s
    Last message repeated 472 times
[mp4 @ 00000000058281e0] pts has no valueB time=00:01:21.41 bitrate=18697.2kbits/s
    Last message repeated 1 times
frame= 2443 fps=975 q=-1.0 Lsize=  186041kB time=00:01:21.45 bitrate=18711.3kbits/s
video:182816kB audio:3156kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.037520%

Are these errors related to the Stream #0: not enough frames to estimate rate; consider increasing probesize? And why would ffmpeg have a problem detecting rate? Is it talking about frame rate or bit rate? I tried specifying the frame rate, but that doesn't help. Given that this is VBR video I don't know what else to tell ffmpeg. Maybe I am asking too much of ffmpeg? Maybe I should be muxing to an MOV? Well, any help is appreciated. Thanks.

[poisondeathray]

-f h264 before the -i

ffmpeg -r 30000/1001 -f h264 -i video.m4v -i audio.mp4 -c:v copy -c:a copy muxed.mp4

[SameSelf]

thanks pdr for the quick reply. Tried that and still get the same Warning and pts message.

[poisondeathray]

But the output file is fine , plays ok ?

Or you could use mp4box instead, it's what most people use for muxing

[SameSelf]

Hmm, well, I managed to suppress the probesize error by inserting the following:

ffmpeg -probesize 50000000 -r 30000/1001 -f h264 -i video.m4v -i video.mp4 -c:v copy -c:a copy muxed.mp4

In case anyone is wondering, the default probesize is 5,000,000. So I just increased it to 50,000,000. So no more:

[h264 @ 0000000000350700] Stream #0: not enough frames to estimate rate; consider increasing probesize

But I still am getting the pts has no value error:

[mp4 @ 0000000005536c00] pts has no value
    Last message repeated 458 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:00:15.28 bitrate=27188.2kbits/s
    Last message repeated 504 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:00:32.13 bitrate=18815.1kbits/s
    Last message repeated 504 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:00:48.98 bitrate=15979.9kbits/s
    Last message repeated 488 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:01:05.29 bitrate=17284.2kbits/s
    Last message repeated 476 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:01:21.21 bitrate=18690.9kbits/s
    Last message repeated 464 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:01:36.72 bitrate=20751.7kbits/s
    Last message repeated 475 times
[mp4 @ 0000000005536c00] pts has no valueB time=00:01:52.61 bitrate=22018.0kbits/s
    Last message repeated 190 times

So I am guessing this error is related to the audio stream?

[SameSelf]

But the output file is fine , plays ok ?

Or you could use mp4box instead, it's what most people use for muxing

I haven't actually let it finish wrapping. I always kill it. I will play around a little more and let it finish to see if it plays ok.

[SameSelf]

OK, I let it finish encoding and while it plays, the video is jerky unlike when I play the m4v which is smooth. So something is clearly wrong. The audio sounds fine. I didn't realize mp4box was the standard tool for wrapping. I guess I will give that a shot.

[poisondeathray]

ffmpeg is the lowest on the list in terms of problem free muxing elementary streams, it's like the last resort. Very buggy and has been for many years

Use one of the other options. mp4box is the most common, but some people prefer l-smash

[SameSelf]

Wow, I had no idea ffmpeg was the tool of last resort. I was under the assumption that ffmpeg was the Swiss Army knife of all things video. Sadly, I can now truly attest to the bugginess of its wrapping functionality!

I am trying out mp4box. It looks pretty intuitive if one is use to ffmpeg. Doesn't seem to be much documentation out there though and I can't seem to get it to work. The following line just crashes mp4box, no errors or messages. Just a crash message.

mp4box.exe -add "E:\video.m4v:fps=29.97" -add "E:\audio.mp4" "E:\muxed.mp4"

How about ffmbc? Maybe it is a little better at wrapping?

[poisondeathray]

ffmpeg is last resort for elementary streams when muxing. It's fine when you use a container. For example , if you output a wrapped video from x264, you won't get those error messages in ffmpeg. It takes the timestamps and timing information from the container and copies them over. An elementary stream doesn't have that information to copy over, that's probably why it's complaining (even though the mux should be ok with ffmpeg)

On the opposite side of the spectrum, mp4box only accepts elementary streams, so "audio.mp4" is probably causing the crash . Try it with aac audio outside of the MP4 container

[SameSelf]

Ok great. This is definitely a learning process not having dealt directly with elementary streams before. I didn't realize what I was getting into. However, I didn't think you could output a wrapped video from x264 since it doesn't support audio? Or, do mean telling x264 to output the elementary video stream to a container format like MP4 instead of an M4V? Maybe that would solve my problems?

[poisondeathray]

"pure vanilla" x264 can't output to any container. The "official" version hosted by VLC (if you can call it that) can output to mkv container, but they removed MP4 support about a year ago IIRC

You can use x264 --fullhelp and it will list what is supported by that build. Different builds can be compiled with different patches and enable different functions, even audio

for example, komisar's x264 builds

Outfile type is selected by filename:
 .264 -> Raw bytestream
 .mkv -> Matroska
 .flv -> Flash Video
 .mp4 -> MP4 if compiled with GPAC or L-SMASH support (lsmash)
 .avi -> AVI if compiled with support (yes)

There should be no problem using elementary stream output with x264 then using a 3rd party muxer like mp4box. In fact almost all the usual GUI's use this process or some variation of it in the background because it's the most stable way, I'm going to say used by millions of people. It might be your audio doesn't have the proper headers or likely because it's in a container



Note, x264 uses .264 as the extension, not ".m4v" . "m4v" is an ambiguous extension, because it can indicate elementary stream (eg. Adobe uses this), or it can indicate modified MP4 container, usually with AC3 audio (e.g., Apple, Itunes uses this) . So I tend to use .264 or .h264 for AVC, .265 or .h265 for HEVC elementary streams to remove any ambiguity

[SameSelf]

That is a great tutorial pdr. Thanks so much. I will ditch the .m4v for .264. I am sure that isn't helping in this situation. Lots of things to test now! Thanks!

[SameSelf]

A HUGE round of applause to PDR for helping me through this. PHEW! So many little things that I was getting wrong.

I reran the X264 encode to write a .264 file versus the .m4v (yes, mp4box was choking on the .m4v extension). Then I checked my PPro audio export options and it turns out I had the multiplexer set to MP4 even though I was only exporting the audio. Yes, that was causing mp4box to choke too. So after turning that off and exporting the audio as an .aac, I was able to mux the .264 and .aac easily using mp4box into an .mp4!!!!!!

And best of all, the video plays beautifully! I now have a new workflow to add to my tool belt! Most importantly, I minimized the number of encodes for the audio and video and bypass the AME engine for the more favorable x264 encoder. All is well!