Long (40 min +) AVI files created by Matrox RT.X2 in Premiere Pro CS5

[subwoofer]

To cut a long story short, I am struggling immensely trying to get useful video out of a set of AVI files I have received. They are the raw camera captures from a couple of concerts, each 40 to 55 minutes long. I really don´t want to bother the fellow I got them from more than necessary, but now I'm at wits end. I think I have tried virtually every available utility out there on the machine that would at least partially play the files after installing the Matrox free codecs.

Playing back the files on Windows playback invariably conks out at around the 10:20:00 mark - matching the 2GB file size point in the original file. I have tried VirtualDub in various versions, AviDemux, FFmpeg and a few more. All giving the same or worse results. Output from a repackaging/reindexing attempt in FFmpeg below:

ffmpeg -i BN2019_akt1.avi -c:v copy -c:a copy a1_test.avi
ffmpeg version 2020-12-01-git-ba6e2a2d05-full_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)

[avi @ 00000260c233df40] non-interleaved AVI
[mpeg2video @ 00000260c233f840] Invalid frame dimensions 0x0.
[mpeg2video @ 00000260c233f840] ignoring extra picture following a frame-picture
[mpeg2video @ 00000260c233f840] ignoring pic cod ext after 101
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, avi, from 'BN2019_akt1.avi':
  Metadata:
    software        : Matrox DSX AVI file . Format: 6. Build: 1.0.0.423
  Duration: 00:41:43.04, start: 0.000000, bitrate: 26684 kb/s
    Stream #0:0: Video: mpeg2video (Main) (M703 / 0x3330374D), yuv420p(tv, bt709, top first), 1440x1080 [SAR 4:3 DAR 16:9], 25000 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
    Side data:
      cpb: bitrate max/min/avg: 25000000/0/0 buffer size: 7340032 vbv_delay: N/A
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
Output #0, avi, to 'a1_test.avi':
  Metadata:
    software        : Matrox DSX AVI file . Format: 6. Build: 1.0.0.423
    ISFT            : Lavf58.64.100
    Stream #0:0: Video: mpeg2video (Main) (M703 / 0x3330374D), yuv420p(tv, bt709, top first), 1440x1080 [SAR 4:3 DAR 16:9], q=2-31, 25000 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
    Side data:
      cpb: bitrate max/min/avg: 25000000/0/0 buffer size: 7340032 vbv_delay: N/A
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[avi @ 00000260c235d5c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frame= 2481 fps=168 q=-1.0 Lsize= 1607647kB time=00:10:21.10 bitrate=21203.7kbits/s speed=  42x
video:1490411kB audio:116460kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.048305%

There's got to be a way around the problem, but my google mojo is apparently fading.

--
Joachim

[poisondeathray]

Does every video in the set have problems around 10:20:00 , or just that specific one of the set ?

Did you try repackaging with tsmuxer ?

[subwoofer]

It would appear that all attempts at repackaging ends with a file duration 10:21.240, but looking at the contents FFmpeg is not reading the stream correctly. VirtualDub2 reads (and displays) correct file length, but video has severe banding in the upper fifth after a little while and stops completely at approximately 10 minutes.

tsMuxeR can´t open AVI files.

I believe the footage was taken with a Sony camera, given that it is 1080i. I have audio files as separate tracks (Zoom recorder), so I only need to extract the video, but getting both streams out would be nice. I have so far no reason to suspect the integrity of the files at the binary level, I believe this all to be a format issue.

--
Joachim

[subwoofer]

And by the way: destination for the video is FCPX 10.5. I tried directly importing the files, but of course they are not recognized as video.

[DB83]

Well you will have to bother the fellow that gave you these. There is an important question to be answered. Did he actually check that the files were complete before he sent them to you.


Reported run-time means little since that could be calculated from size, bitrate or may even be in the video header.


My guess is that there is an export error and 10 mins is all that is there.


But for clarity sake can you supply a mediainfo report - text mode - of the video before you attempted any repair.

[subwoofer]

The originator claims he just copied the files from the PP project folders and that they are working on his computer. I also have an edited version of one of the concerts, edited by him and the 7GiB m2v file is absolutely fine.

Mediainfo for untouched file below. Bitrate is clearly off (should be 25Mbps as it is an HDV data stream). Also seeing the same "source duration" for all the files.

General
Complete name                            : D:\Barnas2018\footage\BN2018_akt1.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 9.11 GiB
Duration                                 : 48 min 53 s
Overall bit rate                         : 26.7 Mb/s
Writing application                      : Matrox DSX AVI file . Format: 6. Build: 1.0.0.423  

Video
ID                                       : 0
Format                                   : M703
Codec ID                                 : M703
Duration                                 : 48 min 53 s
Source duration                          : 10 min 21 s
Bit rate                                 : 174 Mb/s
Width                                    : 1 440 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Bits/(Pixel*Frame)                       : 4.465
Stream size                              : 12.6 GiB

Audio
ID                                       : 1
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : 1
Duration                                 : 10 min 21 s
Bit rate mode                            : Constant
Bit rate                                 : 1 536 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 114 MiB (1%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 480  ms (12.00 video frames)

--
Joachim, confused

[DB83]

Mediainfo is not always accurate but it seems the 'odd' bitrate is calculated by reference to the apparent erroneous run-length of 10 mins. You will see that earlier in the report a more accurate bitrate for the combined video and audio.


So it follows that if your friend can play these raw files then he has a codec installed in his computer that you are unable to access. But surely it is not a Matrox codec you need to obtain since they are for the video card - if you had one of those I am sure they would play. You need a Sony or compatable codec for the HDV source.


But sending raw files is cavalier in the extreme unless you do have identical systems. You state that he exported a fully playable edit so why oh why could he not have exported the un-edited video in this manner ?

[smrpix]

.avi is a weird wrapper for HDV. What happens if you change the extension to .m2t?

(I've done it in reverse on a current version of Premiere. It accepted the m2t file and threw an error when I tried to import an identical file as .avi.)

[jagabo]

Try adding "-fflags +genpts" before the input file on the ffmpeg command line. And send the output to an MPEG2 program stream (.MPG), MPEG2 transport stream (M2TS), or MOV file.

ffmpeg -fflags +genpts -i BN2019_akt1.avi -c:v copy -c:a copy a1_test.mov

[DB83]

Just to elaborate on my previous post, if that Matrox Card imported the video from the camera it is possible that it did something to it so was not a 1:1 transfer as you would get with other s/w. Put another way if that m703 is a Matrox codec then the video is not camera raw as you have been led to believe.

[DB83]

I'll throw a wild theory in to the mix.


The captures/transfers were done in 2gb chunks - the 10 minute mystery - and your friend messed up with trying to join them so only the first chunk is 'visible'

[subwoofer]

The +genpts did not change anything.

Uncertain why a card would recode the camera stream, but I guess it may have. I have the Matrox codecs (v2.0) installed on the Windows box, but only partly helps. Before that the files would not play at all.

It would be interesting to figure out what is going on here, as it could affect a lot of users going forward. The scenario is that the Matrox (or Canopus or whatever) card sits in a box that gets disused and or dies. Owner thinks nothing of it, having all his/hers material on disk, tapes long since recycled in either sense of the word. Then he needs some archive footage... This kind of issue will bite people, but it may not have yet. Massive collective amnesia ahead.

For myself, I can work from the bits that I can play for what I had in mind.

Resolve on the PC will not touch the files. Nor will FCPX.

--
Joachim

[subwoofer]

I'll throw a wild theory in to the mix.


The captures/transfers were done in 2gb chunks - the 10 minute mystery - and your friend messed up with trying to join them so only the first chunk is 'visible'

I highly doubt it, I believe what I have to be a copy of parts of the PP project file structure. Would be PP that messed it up in that case. The guy has 25 years experience in digital signal transport for broadcast, doubt he would b%&#ox it up like that. The structure looks like this:

<project name>
    <Export>
    <Footage>
        <Audio>
        <Clip>

The "Footage" folder also contain a few .prtl files, I believe those to be Premiere titles?

[DB83]

Well I did say it was 'wild' but the 2gb/10 min issue doth intrigue me.

[subwoofer]

I am also much intrigued by that one. That is when a 32bit signed integer overflows.

[aedipuss]

weird. if it were a captured HDV stream it would have mpeg audio, not pcm as it's not allowed with 1080 video.

[subwoofer]

So what were these cards really doing? We (as a community) need to figure out how to handle these files before we run into the issue of massive losses of archives. Most of all small businesses and personal users as the large organisations most likely have had generational archiving in place for some time - or maybe not.

[DB83]

Some text from the Wiki piece about HDV:


"Because HDV video is recorded in digital form, original content can be copied onto another tape or captured to a computer for editing without quality degradation. Depending on capturing software and computer's file system, either a whole tape is captured into one contiguous file, or the video is split in smaller 4 GB or 2 GB segments, or a separate file is created for each take. The way files are named depends on capturing software. Some systems convert HDV video into proprietary intermediate format on the fly while capturing, so original format is not preserved."

[smrpix]

There is a discussion of what appears to be a similar issue on the Matrox forums from 2014:
http://forum.matrox.com/mxo/viewtopic.php?t=13727
As well as links to older Matrox VFW codecs:
https://www.matrox.com/apps/video_drivers/pages/video/downloads/download/latest/?osID=...plicationID=44

[aedipuss]

Some text from the Wiki piece about HDV:


"Because HDV video is recorded in digital form, original content can be copied onto another tape or captured to a computer for editing without quality degradation. Depending on capturing software and computer's file system, either a whole tape is captured into one contiguous file, or the video is split in smaller 4 GB or 2 GB segments, or a separate file is created for each take. The way files are named depends on capturing software. Some systems convert HDV video into proprietary intermediate format on the fly while capturing, so original format is not preserved."

that's old file system info, from back in the FAT days. whole HDV tapes have been able to be captured intact for many years now. and capturing is different than transcoding to an intermediate format, that's kind of like using an easycrap in the middle to intentionally lose quality

[DB83]

It may be old file system info but we see audio that is not apparently supported by HDV (unless Wiki also got that wrong) and the OP's puzzlement of why the mediainfo report only reports 10 mins of audio.


And that video really does appear to be proprietory. Sure that the original creator can use them but anyone else ?

[subwoofer]

It may be old file system info but we see audio that is not apparently supported by HDV (unless Wiki also got that wrong) and the OP's puzzlement of why the mediainfo report only reports 10 mins of audio.

Well, not only the audio at 10m21s, there are two different durations on the video stream too...

There is a discussion of what appears to be a similar issue on the Matrox forums from 2014:
http://forum.matrox.com/mxo/viewtopic.php?t=13727
As well as links to older Matrox VFW codecs:
https://www.matrox.com/apps/video_drivers/pages/video/downloads/download/latest/?osID=...plicationID=44

It looks to be the exact same problem, but no solution beyond "get hold of a card", which in the day of the laptop is getting even more difficult. Those linked drivers are the ones I have installed. If you try to download them from Matrox you will be prompted for the serial number of your Matrox card.

I bet a whole lot of people just haven't found out they've been screwed yet.

[smrpix]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

[aedipuss]

probably not. it's a codec to be used with old, no longer made matrox hardware only. even their newer systems don't support their old proprietary HDV codec.

[aedipuss]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

it can't be standard HDV with pcm audio. that isn't in the HDV spec for 1080 video.

[subwoofer]

And I am not a Premiere user, I roll with FCPX and occasionally Resolve.

[subwoofer]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

I only have 8+GiB long files with this codec. So a sample may not be small.

[smrpix]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

it can't be standard HDV with pcm audio. that isn't in the HDV spec for 1080 video.

Yes, I understand that PCM is not legal in HDV, you set me right on that earlier. But HDV-compatible mpeg video only streams can be loaded into Premiere and Avid (the only ones I've tested.) If the streams can be separated they may be able to be individually loaded into Premiere ( -- I have less hope for FCPX based on Apple's moves as of a couple of years ago.)

[aedipuss]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

I only have 8+GiB long files with this codec. So a sample may not be small.

no problem if you can up it to a file sharing site.

[aedipuss]

Maybe subwoofer can post a sample. I still have a hunch that video is re-wrapped HDV. If it can be demuxed from the audio, I bet Premiere would recognize it.

http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2016-May/047072.html

it can't be standard HDV with pcm audio. that isn't in the HDV spec for 1080 video.

Yes, I understand that PCM is not legal in HDV, you set me right on that earlier. But HDV-compatible mpeg video only streams can be loaded into Premiere and Avid (the only ones I've tested.) If the streams can be separated they may be able to be individually loaded into Premiere ( -- I have less hope for FCPX based on Apple's moves as of a couple of years ago.)

yep. if we can get it out of the avi wrapper and back into a straight mpeg-2 anything should be able to use it.