ffmpeg choppy audio from large avi but not small avi source file

[cybertheque]

Hoping someone with ffmpeg experience can give me a clue: a 10 second mpeg clip created from a 37 GB huffyuv-
encoded avi file results in choppy audio (10 Hz chop); the same clip created from the first 256 MB of the same avi
file is OK. Here are the messages from each encoding run:

First, the full-size avi file:

================================================== =====

C:\tmp\video>c:\progra~1\ffmpeg\ffmpeg -i g:\vdubscratch.avi -t 0:0:10 -vcodec m
peg2video -pix_fmt rgb24 -acodec mp2 -ar 44100 -b 6000k c:\tmp\video\farpoint-g-
10sec.mpg
FFmpeg version SVN-r12665, Copyright (c) 2000-2008 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-postproc --enable-swscale --enable-avfilt
er-lavf --enable-pthreads --enable-liba52 --enable-avisynth --enable-libfaac --e
nable-libfaad --enable-libgsm --enable-memalign-hack --enable-libmp3lame --enabl
e-libnut --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid
--cpu=i686 --extra-ldflags=-static
libavutil version: 49.6.0
libavcodec version: 51.54.0
libavformat version: 52.13.0
libavdevice version: 52.0.0
built on Apr 2 2008 22:35:11, gcc: 4.2.3
Input #0, avi, from 'g:\vdubscratch.avi':
Duration: 01:31:42.0, start: 0.000000, bitrate: 51708 kb/s
Stream #0.0: Video: huffyuv, yuv422p, 640x480, 29.97 tb(r)
Stream #0.1: Audio: pcm_s16le, 96000 Hz, stereo, 3072 kb/s
Output #0, mpeg, to 'c:\tmp\video\farpoint-g-10sec.mpg':
Stream #0.0: Video: mpeg2video, yuv420p, 640x480, q=2-31, 6000 kb/s, 29.97 t
b(c)
Stream #0.1: Audio: mp2, 44100 Hz, stereo, 64 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
frame= 300 fps= 20 q=2.4 Lsize= 7516kB time=10.0 bitrate=6171.5kbits/s
video:7403kB audio:78kB global headers:0kB muxing overhead 0.471314%

================================================== ======

Now the first 256 MB of that file:

C:\tmp\video>c:\progra~1\ffmpeg\ffmpeg -i g:\farpoint-g.avi -t 0:0:10 -vcodec mp
eg2video -pix_fmt rgb24 -acodec mp2 -ar 44100 -b 6000k c:\tmp\video\farpoint-g2-
10sec.mpg
FFmpeg version SVN-r12665, Copyright (c) 2000-2008 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-postproc --enable-swscale --enable-avfilt
er-lavf --enable-pthreads --enable-liba52 --enable-avisynth --enable-libfaac --e
nable-libfaad --enable-libgsm --enable-memalign-hack --enable-libmp3lame --enabl
e-libnut --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid
--cpu=i686 --extra-ldflags=-static
libavutil version: 49.6.0
libavcodec version: 51.54.0
libavformat version: 52.13.0
libavdevice version: 52.0.0
built on Apr 2 2008 22:35:11, gcc: 4.2.3
Input #0, avi, from 'g:\farpoint-g.avi':
Duration: 01:31:42.0, start: 0.000000, bitrate: 390 kb/s
Stream #0.0: Video: huffyuv, yuv422p, 640x480, 29.97 tb(r)
Stream #0.1: Audio: pcm_s16le, 96000 Hz, stereo, 3072 kb/s
Output #0, mpeg, to 'c:\tmp\video\farpoint-g2-10sec.mpg':
Stream #0.0: Video: mpeg2video, yuv420p, 640x480, q=2-31, 6000 kb/s, 29.97 t
b(c)
Stream #0.1: Audio: mp2, 44100 Hz, stereo, 64 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
frame= 311 fps= 22 q=2.5 Lsize= 7786kB time=10.0 bitrate=6375.2kbits/s
video:7672kB audio:78kB global headers:0kB muxing overhead 0.464086%

================================================== ===============================


Quick web search didn't match the symptoms.

TIA,

Michael

[guns1inger]

Taking a wild guess based on the assumsption that your computer details are accurate, you simply do not have the grunt or the through put to read lossless video data in a sustained stream. Reading a small file is OK, but trying to keep a constant data rate high enough for Huffyuv is simply beyond the capabilities of this machine.

[cybertheque]

Taking a wild guess based on the assumsption that your computer details are accurate, you simply do not have the grunt or the through put to read lossless video data in a sustained stream. Reading a small file is OK, but trying to keep a constant data rate high enough for Huffyuv is simply beyond the capabilities of this machine.

Thanks for replying; why would processing speed have any impact on the contents of the encoded output?
This is just an offline avi -> mpeg conversion, and I have tried it with identical results on a number of machines
including an Athlon 2800. The odd thing is the reported avi file bitrate; it is accurate for the full sized file, but
ridiculously low for the short segment. Perhaps the structure of the avi requires seeking into areas not
present in the short version which confuses (favorably in this instance) ffmpeg?

Michael

[Chris K]

At first, I would try a more recent ffmpeg version. SVN-r12665 isn't the best one around.

http://tripp.arrozcru.com

From celtic-druid, SVN-r10464 is one of the most stable versions which is not to old.

The difference in reported bitrate is weird. I would also try to encode a few tests
with a somewhat longer duration to see what happens with the bitrate.

[guns1inger]

So the choppiness is seen in subsequent encodes of the large source, or only in playback ? If playback, the issue may not be CPU, but data throughput from the HDD. If you can't feed the data at a fast enough rate, the effect will be choppy or laggy playback.

As for the bitrate - it depends on how it is calculated. If it comes from the file header, the difference is odd. If this is an average bitrate for the file, then the clip you have extracted may simply have a lower average bitrate because of the content of that section.

[cybertheque]

Regarding the suggestion to try other ffmpeg versions, I did, and the results are the same.

To clarify the issue, the problem is audio choppiness (regular 10 Hz chops) in the encoded
output (time limited to ten seconds for practicality) from the 37GB input file. There is
no choppiness in the encoded output from the _head_ 256 MB of that _same_ 37GB
input file (dd if=vdubscratch.avi bs=1048576 count=256 of=farpoint-g.avi).

Here is virtualdub 1.8.6's view of the 37 GB avi file (which it produced in capture mode):

Video stream
Frame size, fps (us per frame) 640x480, 29.970 fps (33367 us)
Length: 164898 frames (1:31:42:09)
Decompressor: Huffyuv v2.1.1 (HFYU)
Number of key frames: 164898
Min/avg/max/total key frame size: 141188/202819/321236 (32660614K)
Min/avg/max/total delta frame size: (no delta frames)
Data rate: 48628 kbps (0.01% overhead)

Audio stream
Sampling rate: 96000Hz
Channels: 2 (Stereo)
Sample precision: 16-bit
Compression: PCM (Uncompressed)
Layout: 8060 chunks (0.00s preload)
Length: 528193668 samples (1:31:42:01)
Min/avg/max/total frame size: 156176/262130/262144 (2063257K)
Data rate: 3072 kbps (0.01% overhead)

The capture had no frame drops.
It plays back in virtualdub perfectly.

'gspot 270a' reports (bizarre):
Note: 27.1 GB unneeded bytes at end of file
Multipart OpenDML AVI (3 parts)
(10536 frames in first part, 154362 frames follow)

I am considering trying the 'HC' (HCenc) encoder but hate to have to frameserve from Avisynth and
convert to YV12; I would appreciate other recommendations for a _quality_ VBR and CBR mpeg-2
encoder, preferrably one that doesn't require manually demuxing the streams.

Michael

(Edited addition): FWIW, here is virtualdub's view of the _head_ 256 MB segment taken from
the larger avi file (after it reconstructed missing index data, etc.):
(only differences listed)
Video stream
Length 1211 frames (0:40.40)
Number of key frames: 1211
Min/avg/max/total key frame size: 141676/208669/237064 (246776K)
Data rate: 50031 kbps (0.01% overhead)

Audio stream
Layout: 60 chunks (0.00s preload)
Length: 3888858 samples (0:40.50)
Min/avg/max/total frame size: 88936/259257/262144 (15191K)

[AlanHK]

Audio stream
Sampling rate: 96000Hz

Why are you using 96k sampling rate?

What is the ultimate use/device for the file anyway?

[Chris K]

It has always been a rule to add -f vob to the command line for encoding mpeg2 program streams.

If I leave it off, MPC has a very long seek time and WMP only plays the audio.

[cybertheque]

It has always been a rule to add -f vob to the command line for encoding mpeg2 program streams.

If I leave it off, MPC has a very long seek time and WMP only plays the audio.

Thanks for the reply; I added the argument but it made no difference.

Does a 37 GB avi violate some spec (IIRC didn't MS impose a 2 GB limit in the container design)?

Also, if I need to frameserve from 'avisynth' in order to use other encoders, what are the
implications of colorspace conversion to YV12 (loss of precision, artifacts, etc.) It seems to me that
this is a retrograde solution (see http://www.animemusicvideos.org/guides/avtech/colorspace.html)

Michael

[cybertheque]

Audio stream
Sampling rate: 96000Hz

Why are you using 96k sampling rate?

What is the ultimate use/device for the file anyway?

Part of an oversampling experiment, however, I have now resampled the audio to 44100 Hz
and written a new avi (34 MB); ffmpeg continues to chop the audio but at a higher
frequency (about 20 Hz).

I did successful VBR and CBR encodes using "bbmpeg's" 'avi2mpg2' program; good quality
rendering, control of a large number of parameters, clean and simple GUI, options for
demuxing and specifying temp file locations, etc. but it is quite slow (I will post times for
various machines later).

Of all of the encoders that I have tried, this one has the most potential if it can be optimised.

Edited: I'm getting 7 fps for 6000 kbps CBR, encoding the large avi file described at the top of
this thread on an Athlon 2800

Michael