Mux elementary streams to MP4

[wotdefcuk]

I have .h264 and .aac elementary streams. They also have closed caption data in EIA 608 and 708 format. I used mp4box join the streams using the following command

mp4box -add nicu_cc.h264:fps=29.97 -add nicu.aac -new nicu_withcc.mp4
AVC-H264 import - frame size 720 x 480 at 29.970 FPS
AVC Import results: 23020 samples - Slices: 600 I 9018 P 13402 B - 23021 SEI - 599 IDR
Stream uses forward prediction - stream CTS offset: 2 frames
AAC import  - sample rate 44100 - MPEG-4 audio - 2 channels
Saving nicu_withcc.mp4: 0.500 secs Interleaving

This did the job but somehow the CC data was removed. Does anyone have any idea why? the original video was in 29.97 fps so nothing has changed.

The media info of the original video elementary stream is
General
Complete name : C:\nicu_cc.h264
Format : AVC
Format/Info : Advanced Video Codec
File size : 139 MiB

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Variable
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Writing library : x264 core 128 r2216 198a7ea
Encoding settings : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=40 / keyint_min=4 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=21.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Text #1
ID : CC1
Format : EIA-608
Muxing mode : SCTE 128 / DTVCC Transport
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

Text #2
ID : 1
Format : EIA-708
Muxing mode : SCTE 128 / DTVCC Transport
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

[Baldrick]

I don't think mp4 supports such closed captions. You could try convert the cc to srt subs or ttxt subs instead.

[wotdefcuk]

It does. I have done this before.

MP4 file with CC

[Selur]

sorry, don't know either.
What tool did you use to embed the CC data into a x264 video stream?

[wotdefcuk]

MP2TSME from Manzanita Systems.

[Baldrick]

Didn't know that. How did you made the last mp4? Also mp4box but it worked?

[wotdefcuk]

yes that worked but i am still in testing mode therefore that was my first file and there were a few errors in it so i abandoned it and started from a new file. Now mp4box started stripping the cc. I even make sure that the CC track is in the h.264 elementary stream before I try muxing it using MP4box


General
Complete name : C:\nicu_cc.h264
Format : AVC
Format/Info : Advanced Video Codec
File size : 139 MiB

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Variable
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Writing library : x264 core 128 r2216 198a7ea
Encoding settings : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=40 / keyint_min=4 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=21.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Text
ID : CC1
Format : EIA-608
Muxing mode : SCTE 128 / DTVCC Transport
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

[poisondeathray]

What was different between the 1st successful attempt and the 2nd unsuccessful one ? (how did you get the 1st one to "Work") ?

What software or hardware is capable of parsing CC in MP4 ? Are you able to see them ?

[Selur]

iirc the h.264 decoder of elecard can handle cc in H.264

[wotdefcuk]

The hardware used is a STB (enseo.com) and I am streaming to it using VLC. I havent found a player that can decode the CC in and play it. But as the CC data is in the video stream it self therefore the STB gets it and is able to play the file and toggle CC as need be.

Having said that VLC should be able to play this. http://mikecheat.com/disjrhd.ts
I am not sure why this happens because both of the captions look the same.

[wotdefcuk]

My fault. The captions are still there its just that I can't see them in media info. That is really weird behavior and I dont know why it is happening.
This is what I see but the captions can be played through the stream.

General
Complete name : C:\nicu_withcc.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 156 MiB
Duration : 12mn 48s
Overall bit rate mode : Variable
Overall bit rate : 1 709 Kbps
Encoded date : UTC 2013-01-17 21:35:06
Tagged date : UTC 2013-01-17 21:35:06

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 12mn 48s
Bit rate : 1 513 Kbps
Maximum bit rate : 5 019 Kbps
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 29.970 fps
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.146
Stream size : 139 MiB (89%)
Title : nicu_cc.h264:fps=29.97 - Imported with GPAC 0.5.0-rev4065
Writing library : x264 core 128 r2216 198a7ea
Encoding settings : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=40 / keyint_min=4 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=21.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2013-01-17 21:35:06
Tagged date : UTC 2013-01-17 21:35:10

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 12mn 48s
Bit rate mode : Variable
Bit rate : 192 Kbps
Maximum bit rate : 202 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Compression mode : Lossy
Stream size : 17.6 MiB (11%)
Title : nicu.aac - Imported with GPAC 0.5.0-rev4065
Encoded date : UTC 2013-01-17 21:35:09
Tagged date : UTC 2013-01-17 21:35:10

[wotdefcuk]

Is there a way to mux the elementary streams with ffmpeg?

[Baldrick]

Try

ffmpeg -i video.h264 -i audio.aac -acodec copy -vcodec copy video.mp4

[wotdefcuk]

yup tried that and the meta data about the captions is getting removed here too. Is there a way to maintain the metadata? It's in the format below

Text #1
ID : CC1
Format : EIA-608
Muxing mode : SCTE 128 / DTVCC Transport
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

Text #2
ID : 1
Format : EIA-708
Muxing mode : SCTE 128 / DTVCC Transport
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

[poisondeathray]

if you can see the actual CC, why do you care what mediainfo says ?

[wotdefcuk]

Because I am trying to do this all programmatically. So I can QC the file after everything is said and done.

[poisondeathray]

mp4box works ok when muxing elementary stream from your 1280x720p59.94 sample, CC preserved in mediainfo

Maybe something different about your other test file? Can you upload a sample from that file ?

[wotdefcuk]

That video was not created through mp4box. That was just a sample file to prove that CC works with mp4 container.

[poisondeathray]

That video was not created through mp4box. That was just a sample file to prove that CC works with mp4 container.

But extracting the elementary streams , then wrapping with mp4box works in that sample

How is that elementary stream different than the other elementary stream that doesn't work ? (Other than resolution, etc)

Maybe you can provide more information on what you did exactly. Maybe you are using a certain mp4box build that overwrites metadata

[wotdefcuk]

I am using this version.

https://www.videohelp.com/tools/mp4box

[Baldrick]

0.5.0 or latest 0.5.1 nightly/beta?

[wotdefcuk]

0.5.0