How to Demux avi with multiple video streams?

[Chojun]

Hello,

I registered with these forums because I've had a hard time finding an answer to this question.

I have a Fujifilm Finepix Real3D W1 camera. Essentially it is a digital camera with 2 lenses that has a specialized 3D LCD screen. It outputs video in .AVI format. The video is essentially two M-JPEG streams with an audio stream. I'm trying to (manually) convert the 3D AVI format to a side-by-side format that can be used by a 3D Head-mounted display. To do this, I need to demux the avi from the camera so that I can process it by hand to a side-by-side format in adobe premiere.

Anyone know of a good utility that can demux avis with multiple video streams? It seems like all the ones I've come across say they don't support avis with multiple video streams.


Thanks in advance!

Chojun

[jagabo]

Could you provide a short sample video?

[Chojun]

Thanks for the quick reply.

Here is a short (2 sec) video. The video codec is M-JPEG and I don't remember the audio codec off-hand but I'm more concerned about the video streams right now. I've tried a number of different tools to extract the streams.. but since I'm a n00b at this it'd be nice to have someone who knows what they're doing lend a hand.


Thanks a bunch!

[poisondeathray]

It seems the usual utilities "see" only 1 video stream, and can't process it properly

One way to do it is with ffmpeg , you basically stream copy the 1st video stream into a new video, and then do that for the 2nd, so you end up with 2 videos , left and right.

if you type (I renamed your file "input.avi"), it will give you the stream mappings:
ffmpeg -i input.avi

  Duration: 00:00:02.99, start: 0.000000, bitrate: 9676 kb/s
    Stream #0.0: Video: mjpeg, yuvj422p, 320x240, 30 tbr, 30 tbn, 30 tbc
    Metadata:
      strn            : FUJIFILM AVI STREAM 0200
    Stream #0.1: Audio: pcm_s16le, 11024 Hz, 2 channels, s16, 352 kb/s
    Stream #0.2: Video: mjpeg, yuvj422p, 320x240, 30 tbr, 30 tbn, 30 tbc
    Metadata:
      strn            : FUJIFILM AVI STREAM 0200

so to make video1, you want stream 0 for the video and stream 1 for the audio
for video2, you want stream 2 for the video (+/- audio, I disable audio for the 2nd video)

To do this, you use the -map function which correlates with the stream listing above

ffmpeg -i input.avi -map 0:0 -map 0:1 -vcodec copy -acodec copy output1.avi

ffmpeg -i input.avi -map 0:2 -vcodec copy -an output2.avi

"-vcodec copy" just copies the video , and "-acodec copy" copies the audio, and "-an" means disable video, "-vn" would mean disable video

In this zip file, output1 = stream0+audio1 , output2 = stream2+no audio

[Cornucopia]

You can use Graphedit to create a Directshow graph that'll take care of what you want.

FileSource outputs to Demuxer/Splitter, which outputs 2 video and 1 audio. Take 1 video and 1 audio and input to Muxer 1 then output to Filewriter1. Take video2 and input to Muxer 2 then output to Filewriter2.

Run the graph (setting options for files to be saved) and you should be done

Scott

[jagabo]

Not many tools can deal with the dual video streams. Here's a graph from GraphStudio that will split the video into two AVI files:



On the left is the File Source (Async.) filter. The output from that is routed to the AVI Splitter. The left video and both audio channels are routed to an AVI Mux filter, the right video output to a second AVI Mux filter. The two Mux filters are routed to separate File Writer filters, left.avi and right.avi. After "playing" this graph you will get two AVI files. Only the left one will have audio.

Here's an AviSynth filter that will create a color anaglyph:

left=AVISource("left.avi").ConvertToRGB()
right=AVISource("right.avi").ConvertToRGB()

red=ShowRed(left)
green=ShowGreen(right)
blue=ShowBlue(right)

MergeRGB(red,green,blue)

[Bergen]

I bought the same camera and was about to ask the exact same question when I found this topic. So nobody knows a way to extract the seperate streams in avisynth? I've been looking at all the internal avs functions but couldn't find anything suitable. It would be great if it can be done with avisynth.

Chojun: have you also seen this stereoscopic player software? http://www.3dtv.at/
It can load avi's with multiple video streams, or side by side, anything. And it can playback in all different kinds of formats, like red/cyan, green/purple, blue/yellow, or pageflip, etc. It also supports some hardware solutions. For example, it plays the avi's from the Fuji 3D camera to my Vuzix VR920 vr headset without the use of any conversion software (although it seems to have some vsync problems on one of my pcs). It's free, but then you can only play 5 minutes or so. I bought a license right away, great software.

[jagabo]

I believe you can build two capture graphs, one for the left stream, one for the right stream. Then use a pair of DirectShowSource() filters to read from the two graphs.

<edit>
Yes, I just tried it and it worked.

left=DirectShowSource("left.grf", audio=false).ConvertToRGB()
right=DirectShowSource("right.grf", audio=false).ConvertToRGB()
audio=DirectShowSource("audio.grf", video=false)

red=ShowRed(left)
green=ShowGreen(right)
blue=ShowBlue(right)

AudioDUb(MergeRGB(red,green,blue), audio)

Since DirectShowSource() requires only one pin be active in the GRF file I routed the unused pins to the Null Renderer filter. For example:


</edit>

[Cornucopia]

Excellent, jagabo!

[Chojun]

Wow, thanks for the responses!

Especially thanks to you all for showing me GraphEdit/Studio. That's seriously a cool tool. Using the graphs that you came up with, I've got what I need to get left and right videos into Premiere for manual conversion.

However, the thought had occurred to me: DirectShow is probably capable of squeezing the videos into the side-by-side format that I need, right? Well, here's a graph that I came up with (see pic):

I'm using the Video Mixing Renderer 9 with tweaked settings to mux the two video streams into a side-by-side format that is compatible with the HMD that I'm using. However, the question I have is that I'm not quite sure how to get the output rendered to a file since the Mixing Renderer doesn't have an output pin. Is there a decent alternative that I can use to get a mixed output to a file?

Thanks in advance

[Cornucopia]

If you've already DL'd Stereoscopic Player (which you should anyway, as it's invaluable in these situations), you might as well DL a trial of Stereoscopic Muxer from the same place. The DLL engine from that or SP can be exposed/used in Graphedit/GraphStudio ("Stereo Transformation Filter", IIRC) to do exactly what you need without using Video Renderers.
Video Renderers will ONLY output to screen, so you're right, they don't have an output node. They're (VMR) like WAVEOUT which renders to the soundcard.

Scott

[jagabo]

If the stereo muxer Conrucopia refers to doesn't work you can use AviSynth's StackHorizontal() filter:

left=DirectShowSource("left.grf", audio=false).ConvertToRGB()
right=DirectShowSource("right.grf", audio=false).ConvertToRGB()
audio=DirectShowSource("audio.grf", video=false)

AudioDub(StackHorizontal(left, right), audio)

[jmdajr]

this was incredibly helpful to me.. thank you!

[fdrdrive]

I'm trying to do something very similar to Chojun in this thread. However, my input files are .asf from a video conferencing platform. Running the file against ffmpeg -i, I see the following:

Input #0, asf, from 'demo.asf':
  Metadata:
    KeyFrameSpacing : 60
    DualVideoIsH264 : 0
    DualVideoFs     : 2
    DualVideoIsH263 : 0
    DualVideoNoData : 0
    WMFSDKVersion   : 10.00.00.3995
    WMFSDKNeeded    : 0.0.0.0000
    IsVBR           : 0
  Duration: 00:01:32.62, start: 0.588000, bitrate: 420 kb/s
    Stream #0:0(eng): Video: vc1 (Advanced) (WMVA / 0x41564D57), yuv420p, 320x240, 320 kb/s, 1k tbr, 1k tbn, 1k tbc
    Stream #0:1(eng): Audio: wmav2 (a[1][0][0] / 0x0161), 16000 Hz, 1 channels, s16, 16 kb/s
    Stream #0:2(eng): Video: vc1 (Advanced) (WMVA / 0x41564D57), yuv420p, 1024x768, 192 kb/s, 1k tbr, 1k tbn, 1k tbc

I can strip out Stream 2 without issue using this:

ffmpeg -i demo.asf -map 0:2 -vcodec copy -an video_stream_2.asf

HOWEVER - Stream 0 (Video) + Stream 1 (Audio) does NOT work!! Can't seem to strip that out.
I keep getting this error:

[asf @ 02119020] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1693 >= 1693   av_interleaved_write_frame(): Invalid argument

I'm attaching the demo file.

[Cornucopia]

For WMV/ASF, it's much easier to use the Windows Media Streams Editor (utility, comes with WME). Select each stream and demux separately. Done this MANY times.

Scott

[fdrdrive]

Thanks Cornucopia I've tried using the Windows Media Stream Editor, but it complains :

"The requested video codec is not installed on this system"

Running WinXP, and installed the CCCP codec pack and the K-Lite codec pack.

Also, trying to automate the DEMUX process via command line - was hoping to use FFMPEG

[fdrdrive]

AVI only seems to work, if I convert the following:

ffmpeg -i demo.asf demo.avi 

ffmpeg -i demo.asf -map 0:2 -vcodec copy -an video_stream_2.avi

FFMPEG complains:
"Seems stream 2 codec frame rate differs from container frame rate: 1000.00 (1000/1) -> 22.83 (137/6)"

[Cornucopia]

Sounds like you might STILL have a codec problem. What does MediaInfo say about your source files? (post text output here).

Scott

BTW, I know others here would disagree with me, but when I read: "installed the CCCP codec pack and the K-Lite codec pack", what that means to me is: "I tried the shotgun approach of using codec packs, in the hope that while junking up my computer with lots of stuff it doesn't need, I might have stumbled upon giving it something it MIGHT need."

Folks: LEAN MEAN FIGHTING MACHINE! Less is more. Only what you absolutely need.

[fdrdrive]

I am guilty of the shotgun approach... here is the output of MediaInfo

General
Format                                   : Windows Media
File size                                : 4.64 MiB
Duration                                 : 1mn 32s
Overall bit rate mode                    : Constant
Overall bit rate                         : 420 Kbps
Maximum Overall bit rate                 : 214 Kbps
Encoded date                             : UTC 2012-01-25 18:20:33.801
KeyFrameSpacing                          : 60
DualVideoIsH264                          : No
DualVideoFs                              : 2
DualVideoIsH263                          : No
DualVideoNoData                          : No

Video #1
ID                                       : 2
Format                                   : VC-1
Format profile                           : AP@L0
Codec ID                                 : WMVA
Codec ID/Info                            : Windows Media Video
Codec ID/Hint                            : WMV
Description of the codec                 : Windows Media Video 9 Advanced Profile
Bit rate mode                            : Constant
Bit rate                                 : 320 Kbps
Width                                    : 320 pixels
Height                                   : 240 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Variable
Nominal frame rate                       : 25.000 fps
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Language                                 : English (US)

Video #2
ID                                       : 3
Format                                   : VC-1
Format profile                           : AP@L1
Codec ID                                 : WMVA
Codec ID/Info                            : Windows Media Video
Codec ID/Hint                            : WMV
Description of the codec                 : Windows Media Video 9 Advanced Profile
Duration                                 : 1mn 32s
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Width                                    : 1 024 pixels
Height                                   : 768 pixels
Display aspect ratio                     : 4:3
Frame rate                               : 10.000 fps
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.024
Stream size                              : 2.12 MiB (46%)
Language                                 : English (US)

Audio
ID                                       : 1
Format                                   : WMA
Format version                           : Version 2
Codec ID                                 : 161
Codec ID/Info                            : Windows Media Audio
Description of the codec                 : Windows Media Audio 9.1 -  16 kbps, 16 kHz, mono 1-pass CBR
Duration                                 : 1mn 32s
Bit rate mode                            : Constant
Bit rate                                 : 16.0 Kbps
Channel(s)                               : 1 channel
Sampling rate                            : 16.0 KHz
Bit depth                                : 16 bits
Stream size                              : 181 KiB (4%)
Language                                 : English (US)

[Cornucopia]

OK, you should have the codec(s) (one is for AP @ L0 and the other is for AP @ L1), though there could be a disparity between having the decoder(s) and the encoder(s)...
(hard to say whether it's 1, 2, or 4 separate pieces of code)

One thing I notice is that your 2nd V stream, while HD, only plays at 10Fps instead of 25Fps (and with a lower bitrate than the main stream). Looks like that was supposed to be the "Powerpoint" channel, or back channel or something simlar that doesn't change often.
This WILL make your extraction difficult, because the framerates DON'T match, as ffmpeg said.

Well, most "codec packs" do NOT have WMV/VC-1 installers (because that is strictly overseen by Microsoft and they would get their butts kicked if they supplied that without permi$$ion).

You ought to see if you've got WME, and the various WMV codecs installed (specifically, the Advanced Profile ones). Those are more common with stock installs of Windows7, but not so much for XP.

Scott

[fdrdrive]

I did find this interesting article:

https://forum.videohelp.com/threads/319455-How-to-Demux-avi-with-multiple-video-streams?goto=newpost

Looks like the WMVA codecs are only supplied in XP 64bit / Server 2003

[fdrdrive]

Scott,

were you able to open up the samples I uploaded in the WM Stream Editor?

[Cornucopia]

Will DL and check tonite...

[gnznroses]

Have a similar problem. Trying to turn a dual-stream WMV into side-by-side format, and I installed the Stream Editor utility as recommended, but when I try to export one of the streams it throws an error, saying that the installed codec doesn't support maintaining interlacing.

Installed GraphEdit and GraphStudio and they both are only seeing one video stream it seems (or I simply don't know what I'm doing, but I notice it only labels "Raw Video 1" and "Raw Audio 2").

[Gixxer]

Hi, I'm trying to demux a mjpeg video from a Fujireal3dw3 to side by side.
Anyone found a good method / easy demuxer to do this ?

[agatek]

Hi, I have half related question to the OP one.
I use Stereo Movie Maker (SMM) to create 3D anaglyphs from the same Fuji camera (newer version) 3D AVIs. I am pretty pleased with the SMM output quality but what kills me is lack of any batch encoding. Using the camera I typically end up with a dozen or two of short movies that I had to process one by one in SMM. Is there any way, perhaps again utilizing ffmpeg to join first all the 3D 2 video stream AVIs into one bigger movie file?

If I do something like

cat n1.avi n2.avi .. nn.avi > all.avi
ffmpeg -i all.avi -map 0:0 -map 0:1 -map 0:2 -vcodec copy -acodec copy allindexed.avi

I end up with n1.avi only

[jagabo]

You can't just cat AVI files together. Each file's header has the properties of that file -- including its length. Any program will open the file, look at the first header, see the length of the first file you cat'd, and assume the rest is junk.

AviDemux can properly join AVI files and has a command line mode. You might be able to use that.
http://avidemux.berlios.de/doc/en/command.xml.html
Though I scanned that page quickly and didn't see any "add" or "append" commands.

[agatek]

Oh, I can for sure, but you need some extra processing after the cat command. I can give you an example with mancoder:

ffmpeg -i 1-a.avi -i 1-b.avi 
[..]
Input #0, avi, from '1-a.avi':
  Metadata:
    encoder         : Lavf54.59.107
  Duration: 00:00:30.00, start: 0.000000, bitrate: 26078 kb/s
    Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p, 1280x720, 24 tbr, 24 tbn, 24 tbc
    Metadata:
      title           : FUJIFILM AVI STREAM 0200
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
[avi @ 0x1750da0] non-interleaved AVI
Guessed Channel Layout for  Input Stream #1.1 : stereo
Input #1, avi, from '1-b.avi':
  Metadata:
    encoder         : Lavf54.59.107
  Duration: 00:00:25.00, start: 0.000000, bitrate: 27186 kb/s
    Stream #1:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p, 1280x720, 24 tbr, 24 tbn, 24 tbc
    Metadata:
      title           : FUJIFILM AVI STREAM 0200
    Stream #1:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s

cat 1-a.avi 1-b.avi > 1a1b.avi

mencoder 1a1b.avi  -o 1a1b-i.avi -forceidx -ovc copy -oac copy
[..]AVI file format detected.
[aviheader] Video stream found, -vid 0
[aviheader] Audio stream found, -aid 1
Generating Index:  99 %     
AVI: Generated index table for 3905 chunks!
VIDEO:  [MJPG]  1280x720  24bpp  24.000 fps  25026.4 kbps (3055.0 kbyte/s)
[V] filefmt:3  fourcc:0x47504A4D  size:1280x720  fps:24.000  ftime:=0.0417
videocodec: framecopy (1280x720 24bpp fourcc=47504a4d)
audiocodec: framecopy (format=1 chans=2 rate=48000 bits=16 B/s=192000 sample-4)
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.
Pos:  55.0s   1320f (100%) 1173.33fps Trem:   0min 174mb  A-V:0.000 [25044:1536]]
Writing index...
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.

Video stream: 25026.012 kbit/s  (3128251 B/s)  size: 172053824 bytes  55.000 secs  1320 frames

Audio stream: 1536.000 kbit/s  (192000 B/s)  size: 10560000 bytes  55.000 secs

ffmpeg -i 1a1b.avi
[..]
[avi @ 0xc62400] non-interleaved AVI
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from '1a1b.avi':
  Metadata:
    encoder         : Lavf54.59.107
  Duration: 00:00:30.00, start: 0.000000, bitrate: 48733 kb/s
    Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p, 1280x720, 24 tbr, 24 tbn, 24 tbc
    Metadata:
      title           : FUJIFILM AVI STREAM 0200
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s

ffmpeg -i 1a1b-i.avi
[..]
[avi @ 0x1914400] non-interleaved AVI
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from '1a1b-i.avi':
  Metadata:
    encoder         : MEncoder svn r34540 (Ubuntu), built with gcc-4.6
  Duration: 00:00:55.00, start: 0.000000, bitrate: 26568 kb/s
    Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p, 1280x720, 24 tbr, 24 tbn, 24 tbc
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s

Many sources say the same for ffmpeg that's why I mentioned it and because I am more familiar with ffmpeg than mencoder.
Mencoder can in principle handle 2 video streams via -vid switch but I have not managed to far to convince it to write 2 video streams using the above reindexing command.

[jagabo]

Oh, I can for sure, but you need some extra processing after the cat command.

I said, you can't just cat AVI files together, as described in your original post.

[agatek]

So what was your point of saying it if I included in the very next line the ffmpeg command?