ffmpeg CPU usage

[diprotic]

I'm encoding with -threads 4 to use my quad-core processor, but ffmpeg maxes out at about 55% overall CPU usage at ~125fps. I've upped the threads to 8 and 16 with no improvement. Any thoughts on how to use ~100% CPU when encoding?

Note: I'm running ffmpeg 0.7.5 on Arch Linux; Core2 Quad Q9550/4GB

[aedipuss]

faster drives and more of them. a single raid 1 array is slower than a single drive as it has to make 2 copies of everything. add separate o.s., source, and destination drives. maybe use that raid 1 as the o.s only drive.

[intracube]

What's the full set of ffmpeg parameters you're using?

[diprotic]

ffmpeg -i infile -target ntsc-dvd -s 720x480 -b 2853k -pass 1 -aspect 16:9 -r 29.97 -vf yadif -y -mbd rd -trellis 2 -cmp 2 -subcmp 2 -an -threads 4 -f rawvideo -y /dev/null
ffmpeg -i infile -target ntsc-dvd -s 720x480 -b 2853k -pass 2 -passlogfile ffmpeg2pass -aspect 16:9 -r 29.97 -vf yadif -y -mbd rd -trellis 2 -cmp 2 -subcmp 2 -acodec copy -threads 4 outfile

Both passes run at max 60% CPU load & 125fps. I understand deinterlacing will slow things down, but that's no reason not to use ~100% CPU.

[intracube]

What program is giving you the figure of 60%?

I've just done a test encode based on your parameters. The only differences being the PAL framerate and resolution.

On my old dual core system I'm getting 30-40fps and 150% CPU usage reported by top. 200% would be the theoretical limit - 100% for each CPU.

I'd guess that if you ran top on your system it'd report ~240%.

I'd estimate that at 40fps my disk read access is ~6Mbps and writing ~4Mbps. Unless you're encoding from an uncompressed source, I can't see that this would be the bottleneck, even at your higher encoding speeds.

It might be down to the inefficiencies of the code. I don't mean to suggest that it's an ffmpeg specific problem, but from what I've read, multi threaded code often falls short of the theoretical limit. This thread echoes closely what we're seeing:
https://forum.videohelp.com/threads/299011-ffmpeg-and-quad-core

[diprotic]

You're spot-on: I'm estimating from top and it does occupy just over 200% CPU usage.

I'm also using modern disks with 32MB buffers that can usually sustain 60+ MB/s read/write. I know I'm not hitting anything like that with my sources. Also, the first pass is writing to /dev/null, yet another reason to not suspect disk bottleneck.

Worth noting: If I run 2 simultaneous ffmpeg batches, they each process uses ~160-180% CPU. That totals to 80-90% usage, which makes more sense.

[Calidore]

I'll generally run 3-4 ffmpeg conversions concurrently, each set to 8 threads, to maximize CPU (i7 here).

[jh443]

It is possible for a program to use all threads and still show less than 100% CPU usage. I use Process Explorer which shows individual thread usage on my i7 (2630QM), and I've often seen programs use all 8 threads with far less than 100% total usage.

http://technet.microsoft.com/en-us/sysinternals/bb896653

On the CPU usage display page, enable the "Show one graph per CPU" option.

[intracube]

Worth noting: If I run 2 simultaneous ffmpeg batches, they each process uses ~160-180% CPU. That totals to 80-90% usage, which makes more sense.

That sounds about right. Have you tried 4 simultaneous instances of ffmpeg, each set to use 1 thread?

That might give you the best utilization of your cores.

If you only want to encode a single video you might want to try splitting the source into 4 parts, encoding each part with a different ffmpeg instance, then recombine when authoring the DVD.

But you might have issues at the split points - GOP boundaries for example, which could cause visible glitches on playback. The source would have to be cut very carefully. I haven't tried any of this so don't know if it would work.

A 1 hour program might go from a 12 minute encode to 8 minutes, so probably not worth the extra hassle. But if you've got a load of videos to encode, I'd definitely have four ffmpegs running, each set to use a single thread.

[diprotic]

I'll generally run 3-4 ffmpeg conversions concurrently, each set to 8 threads, to maximize CPU (i7 here).

That's definitely the way to go. In this case I'm transcoding ten 1-hour videos, so I could easily split the job into 3-5 jobs. Thanks for the tip

[intracube]

ffmpeg -i infile -target ntsc-dvd -s 720x480 -b 2853k -pass 1 -aspect 16:9 -r 29.97 -vf yadif -y -mbd rd -trellis 2 -cmp 2 -subcmp 2 -an -threads 4 -f rawvideo -y /dev/null

One thing I forgot to mention - it's always preferable to de-interlace footage before any image resizing or other processing. If your source is interlaced, resizing can destroy the fields, cause artefacts and stop yadif or similar de-interlacing algorithms from working.

Something like this would work better:

ffmpeg -i infile -target ntsc-dvd -b 2853k -pass 1 -r 29.97 -vf yadif,scale=720:480 -aspect 16:9 -y -mbd rd -trellis 2 -cmp 2 -subcmp 2 -an -threads 4 -f rawvideo -y /dev/null

The '-s' option is the older method for resizing. FFmpeg now supports scaling within the -vf filter chain.

I'm not sure there's much point in choosing a high number of threads for each ffmpeg instance. If you've got four cores, I think running ffmpeg four times with -threads 1 for each will work just as well.

But by all means experiment, and post your findings if you see a benefit in increasing the number of threads.

[diprotic]

intracube, good to know about the video filters chain. I didn't know interlacing should come before resize. I also noted from the ffmpeg FAQ that -flags +ilme+ildct is a good option when dealing with interlaced sources. This option slowed my encoding by about 25% but did improve quality a bit.

[intracube]

intracube, good to know about the video filters chain. I didn't know interlacing should come before resize.

Another thing I should have noticed earlier; If you're authoring a DVD to play on a standalone player and TV, you shouldn't normally de-interlace the video. If the source is genuine 60i (59.94i), using yadif without any additional parameters will give you a video of half the framerate - 30p (29.97p).

I only use yadif to de-interlace footage that's going to end up on the net. Or possibly as part of a process to make it look more like film.

I also noted from the ffmpeg FAQ that -flags +ilme+ildct is a good option when dealing with interlaced sources. This option slowed my encoding by about 25% but did improve quality a bit.

Yes, AIUI these flags cause ffmpeg to encode each field independently to achieve better compression for interlaced content.

One other issue is that by default ffmpeg doesn't set the field order for the encoded video, even if it's correctly flagged in the source video and you use the +ilme+ildct flags. I have to set -top 1 (top field first) to get proper playback with 'PAL' authored DVDs. I think you want the same option for 'NTSC' DVDs.

[yetanotherlogin]

Hello

I'm running ffmpeg on Windows, and having the opposite problem: It's using so much CPU that Windows locks up (BSOD).

I read about BES, but it doesn't seem to handle command-line parameters.

Is there a way to tell ffmpeg to slow down?

Thank you.

----
Edit: Found it:
1. Open a DOS box, and run ffmpeg
2. Launch BSE
3. Click on "Target"
4. Click on "Limit This". Select cmd.exe process
5. Click on "Control"
6. Move the slider (right = less CPU time, left = more)

Not as good as having some limit embedded in ffmpeg, but it does the job.

---
Edit: The above doesn't work that well. CPU usage would still be so high that the CPU temperature would go beyond 70°C and cause a BSOD. Neither does using Windows' Task Manager to force the cmd.exe process to run with Low priority.

The solution is to be found within ffmpeg:

-threads 2

On a four-core CPU, it's still fast enough, and temperature won't go over 60°C.