Converting NTSC DV AVIs to H.264

[paleGreen1]

Please don't hurt me too much for asking a question that has surely been thoroughly discussed. I've been researching the best way to do this for a few days now and my head is spinning. I think I just need a shove in the right direction and I'll be off.

I have several thousand DV AVI's, almost all shot on a Sony DCR-TRV17, digitized using Premiere Pro, totaling about 1.8 TB. I would like to convert them to a more usable/compatible format (and also save space on my server) to be viewed on my TV using either our WD TV Live or Xbox 360 (also acting as Windows 7 Media Center Extender).

I'm a bit of a frame rate nut, so I don't want to lose the smooth motion of the original interlaced files (29.97 fps or 59.94 fields/sec). I looked into converting them to progressive 59.94 fps, but the loss in quality (although slight, using NNEDI2/3 and yadifmod), plus the extra cpu time, plus the higher encoded bitrate needed to maintain the same quality has lead me to believe that I should encode them interlaced. I realize interlaced encoding is not as efficient as progressive, but then it seems better than progressive 59.94 fps.

Like I said, I've been doing a lot of research and reading lots of posts, but these seemed most relevant:
https://forum.videohelp.com/threads/320394-Converting-DV-to-H-264-and-comparing-interla...-de-interlaced
http://forum.doom9.org/showthread.php?t=148187

So I'm looking for a decent AVS template for feeding these into an x264 encoder. Here's what I have working so far:

  DirectShowSource("dv-file.avi")
  global MeGUI_darx = 4
  global MeGUI_dary = 3
  AssumeBFF()
  ConvertToYV12(interlaced=true)

And then run it through MeGUI with this command line:

  program --crf 20 --bff --output "output" "input"

The output looks decent, although I've noticed a couple issues (maybe minor):
- it plays back 640x480 in WMP 12
- it plays back 720x540 in VLC 2.0.1
- it plays back at 720x480 *or* 640x480 in MPC-HC 1.6.1 (it seems to alternate every time I open the video)
- it won't open at all in QT Player 7.7.1 (says "Error -2002: a bad public movie atom was found in the movie")

Oh, I also tried just dropping it into Adobe Media Encoder CS5 and it did not keep the 59.94 fields/sec interlacing. I also looked at Handbrake, but it apparently doesn't support interlaced encoding. And RipStax wants me to install ffdshow (missing YV12 decoder) which has messed up my codecs before, so I don't want mess with that if I can help it.

Can anyone offer any advice or direction for me? Do I need to worry about that QT Player error? (I'd prefer the files be compatible with it and most other players, ideally.) Am I making this too hard? It seems like this should be a relatively easy, since I'm sure many people have "been there, done that."

Thanks for not hurting me too bad!

[_Al_]

http://avisynth.org/mediawiki/QTGMC . QTGMC is double frame deinterlacer.
Get whole package from overthere, author -Vit- put all needed dll's together for v3.32 and it is available for download.

About aspect ratio, you have to make an executive decision if to resize to square pixel like 640,480 or 720,540 or to go with aspect ratio, but not 4:3, DV avi is not square pixel , set it to 8:9, --sar=8:9

[paleGreen1]

Thanks _Al_. I haven't heard of that one before. But in this case I want to keep the interlacing. So I'm looking for a high-quality, high-compatibility, interlaced 59.94 frames/sec H.264 output.

I'm currently evaluating WinFF, which is looking promising. I'll update with my findings later.

In the meantime, I would appreciate any input or tips.

Thanks!

[manono]

But in this case I want to keep the interlacing. So I'm looking for a high-quality, high-compatibility, interlaced 59.94 frames/sec H.264 output.

No you don't. That even contradicts what you said in your earlier post. You want (I suppose) interlaced 29.97fps. It was a typo, right?

[paleGreen1]

Oops. Yes, that was a typo. But I do want to encode it interlaced, at 720x480. Basically, keeping everything as close as possible to the originals.

As I mentioned before, I am trying out WinFF, so I tweaked one of the MPEG-4 profiles, adding what I could find online for interlaced encodes, and after a few tries, I have something that looks decent to me. At least the interlacing is properly maintained, the aspect ratio is correct, I have audio, etc.

I started with the MPEG-4 H.264 (Very High Quality) Profile:

-crf 25.0 -vcodec libx264 -acodec libfaac -ar 48000 -b:a 160k -coder 1 -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method hex -subq 6 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 0 -qmax 69 -qdiff 4 -bf 3 -refs 8 -direct-pred 3 -trellis 2 -wpredp 2 -rc_lookahead 60 -threads 0

I changed these:
-crf 20.0
-flags +loop+ilme+ildct
-g 300
-keyint_min 29

...and added these:
-top 0
-pix_fmt yuv420p

...so it now looks like this:

-crf 20.0 -vcodec libx264 -acodec libfaac -ar 48000 -b:a 160k -coder 1 -flags +loop+ilme+ildct -top 0 -pix_fmt yuv420p -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method hex -subq 6 -me_range 16 -g 300 -keyint_min 29 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 0 -qmax 69 -qdiff 4 -bf 3 -refs 8 -direct-pred 3 -trellis 2 -wpredp 2 -rc_lookahead 60 -threads 0

So when I encoded my test file, this was the output. Unfortunately, I'm just learning what most of the parameters mean. Does this look OK? Anything jump out at you?

Thank you so much for your time!!

C:\Program Files (x86)\WinFF>chcp 1252
Active code page: 1252

C:\Program Files (x86)\WinFF>title Converting DV-901 12@2009.01.11_16.32.06.avi (1/1)

C:\Program Files (x86)\WinFF>"C:\Program Files (x86)\WinFF\ffmpeg.exe" -y -i "R:\Projects\Temp\DV to MP4\Originals\DV-90
1 12@2009.01.11_16.32.06.avi" -crf 20.0 -vcodec libx264 -acodec libfaac -ar 48000 -b:a 160k -coder 1 -flags +loop+ilme+i
ldct -top 0 -pix_fmt yuv420p -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method hex -subq 6 -me_range 16 -g
 300 -keyint_min 29 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 0 -qmax 69 -qdiff 4 -bf 3 -refs 8 -d
irect-pred 3 -trellis 2 -wpredp 2 -rc_lookahead 60 -threads 0 "R:\Projects\Temp\DV to MP4\WinFF\DV-901 12@2009.01.11_16.
32.06.mp4"
ffmpeg version N-38632-g4cda8aa Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar  8 2012 02:38:08 with gcc 4.5.0 20100414 (Fedora MinGW 4.5.0-1.fc14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=x86 --target-os=mingw32 --cross-prefix=i
686-pc-mingw32- --cc='ccache i686-pc-mingw32-gcc' --enable-w32threads --enable-memalign-hack --enable-runtime-cpudetect
--enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -lwinmm' --extra-cflags='--static -I/var/
www/users/research/ffmpeg/snapshots/build/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/snapshots/b
uild/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --enable-nonfree --enable-libx264 --enable-libspee
x --enable-libtheora --enable-libvorbis --enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-libopencor
e-amrwb --enable-libmp3lame --enable-libvpx --disable-decoder=libvpx
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 63.100 /  2. 63.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, avi, from 'R:\Projects\Temp\DV to MP4\Originals\DV-901 12@2009.01.11_16.32.06.avi':
  Duration: 00:00:54.08, start: 0.000000, bitrate: 30332 kb/s
    Stream #0:0: Video: dvvideo (dvsd / 0x64737664), yuv411p, 720x480 [SAR 8:9 DAR 4:3], 29.97 fps, 29.97 tbr, 29.97 tbn
, 29.97 tbc
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 2 channels, s16, 1536 kb/s
[buffer @ 0x19a6320] w:720 h:480 pixfmt:yuv411p tb:1/1000000 sar:8/9 sws_param:
[buffersink @ 0x19dcdc0] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x39f7c0] w:720 h:480 fmt:yuv411p -> w:720 h:480 fmt:yuv420p flags:0x4
[libx264 @ 0x19dc7c0] interlace + weightp is not implemented
[libx264 @ 0x19dc7c0] using SAR=8/9
[libx264 @ 0x19dc7c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
[libx264 @ 0x19dc7c0] profile High, level 3.1
[libx264 @ 0x19dc7c0] 264 - core 122 r2183 c522ad1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.o
rg/x264.html - options: cabac=1 ref=8 deblock=1:0:0 analyse=0x3:0x111 me=hex subme=6 psy=1 psy_rd=1.00:0.00 mixed_ref=1
me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 sliced_thread
s=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3
 weightb=1 open_gop=0 weightp=0 keyint=300 keyint_min=29 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf
=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'R:\Projects\Temp\DV to MP4\WinFF\DV-901 12@2009.01.11_16.32.06.mp4':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 720x480 [SAR 8:9 DAR 4:3], q=0-69, 30k tbn, 29.97 tbc
    Stream #0:1: Audio: aac (@[0][0][0] / 0x0040), 48000 Hz, 2 channels, s16, 160 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (dvvideo -> libx264)
  Stream #0:1 -> #0:1 (pcm_s16le -> libfaac)
Press [q] to stop, [?] for help
frame= 1621 fps= 18 q=-1.0 Lsize=   12019kB time=00:00:54.02 bitrate=1822.6kbits/s
video:10904kB audio:1057kB global headers:0kB muxing overhead 0.486223%
[libx264 @ 0x19dc7c0] frame I:12    Avg QP:20.54  size: 27632
[libx264 @ 0x19dc7c0] frame P:900   Avg QP:22.85  size:  9725
[libx264 @ 0x19dc7c0] frame B:709   Avg QP:24.18  size:  2935
[libx264 @ 0x19dc7c0] consecutive B-frames: 24.5% 41.2% 31.1%  3.2%
[libx264 @ 0x19dc7c0] mb I  I16..4:  3.7% 89.7%  6.7%
[libx264 @ 0x19dc7c0] mb P  I16..4:  1.2%  0.0%  1.1%  P16..4: 58.1% 23.2% 11.4%  0.0%  0.0%    skip: 5.1%
[libx264 @ 0x19dc7c0] mb B  I16..4:  0.6%  0.0%  0.1%  B16..8: 22.9%  8.4%  0.6%  direct:14.4%  skip:52.9%  L0:26.9% L1:
51.5% BI:21.6%
[libx264 @ 0x19dc7c0] field mbs: intra: 80.1% inter:59.2% skip:33.3%
[libx264 @ 0x19dc7c0] 8x8 transform intra:28.8% inter:74.9%
[libx264 @ 0x19dc7c0] direct mvs  spatial:99.6% temporal:0.4%
[libx264 @ 0x19dc7c0] coded y,uvDC,uvAC intra: 70.5% 83.9% 43.0% inter: 32.0% 46.4% 1.6%
[libx264 @ 0x19dc7c0] i16 v,h,dc,p: 10% 19% 13% 59%
[libx264 @ 0x19dc7c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 26% 15%  6%  8%  6% 13%  5% 13%
[libx264 @ 0x19dc7c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  9% 24% 19%  6% 11%  7% 13%  4%  8%
[libx264 @ 0x19dc7c0] i8c dc,h,v,p: 51% 26% 12% 11%
[libx264 @ 0x19dc7c0] ref P L0: 54.3% 29.8%  4.8%  3.0%  1.6%  1.8%  1.4%  0.9%  0.4%  0.4%  0.3%  0.3%  0.3%  0.3%  0.2
%  0.1%
[libx264 @ 0x19dc7c0] ref B L0: 61.2% 29.7%  3.5%  2.2%  1.0%  1.2%  0.7%  0.2%  0.1%  0.1%  0.1%  0.1%  0.0%  0.0%
[libx264 @ 0x19dc7c0] ref B L1: 69.3% 29.6%  0.5%  0.5%
[libx264 @ 0x19dc7c0] kb/s:1651.40

C:\Program Files (x86)\WinFF>pause
Press any key to continue . . .

[poisondeathray]

[libx264 @ 0x19dc7c0] 264 - core 122 r2183 c522ad1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.o rg/x264.html - options: cabac=1 ref=8 deblock=1:0:0 analyse=0x3:0x111 me=hex subme=6 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 sliced_thread s=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=0 keyint=300 keyint_min=29 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf =20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

That doesn't look right. DV-AVI should be bff . How does the video play ?

I don't see the flag in your commandline , in x264 it should be flagged --bff ; but ffmpeg libx264 encoding is slightly different syntax than x264.exe encoding . In some ffmpeg builds it's -top 0 , in others it's -bff

Also, your final bitrate seems low for a DV source. What kind of content was this clip? Was it clean, with little motion?


EDIT:

C:\Program Files (x86)\WinFF>"C:\Program Files (x86)\WinFF\ffmpeg.exe" -y -i "R:\Projects\Temp\DV to MP4\Originals\DV-90 1 12@2009.01.11_16.32.06.avi" -crf 20.0 -vcodec libx264 -acodec libfaac -ar 48000 -b:a 160k -coder 1 -flags +loop+ilme+i ldct -top 0 -pix_fmt yuv420p -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method hex -subq 6 -me_range 16 -g 300 -keyint_min 29 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 0 -qmax 69 -qdiff 4 -bf 3 -refs 8 -d irect-pred 3 -trellis 2 -wpredp 2 -rc_lookahead 60 -threads 0 "R:\Projects\Temp\DV to MP4\WinFF\DV-901 12@2009.01.11_16. 32.06.mp4"

So it looks like -top 0 was used for that commandline, but the log says it's tff .... not sure why there is a discrpancy

[paleGreen1]

Wow! You are good. (I've been reading your posts for a while now

Yeah, the "interlaced=tff" seems to be a bug in WinFF. I did specify "-top 0" (it's in there) which means bff, but it still output "interlaced=tff". I checked the playback and it looked fine to me. So I did a test, with "-top 1" and it still output "interlaced=tff" but the playback was awful. So it appears that "-top 0" is working, but that the output command is just wrong. <shrug>

And regarding the clip, no it wasn't clean at all. Low light, but with a harsh, blown out highlights in the center, and pretty shaky. (An excellent videographer, I'm not.) I thought it would make for a good test. So I guess I'll lower the the crf factor to get the bitrate up. What range would you think is right? I know that's difficult, not seeing the footage. Or should I do it another way?

Thanks again!

[poisondeathray]

Also, if you want it more compatible (e.g. maybe author SD blu-ray in the future) , I would use BD compatibilty settings

Not sure how of the ffmpeg syntax, but in x264 the --sar value should be 10:11 , not 8:9, because BD spec is based on inner 704px width (even though it's encoded as 720px). You would also add --blu-ray compat and VBV values .

You can see the basic commandline here
http://sites.google.com/site/x264bluray/home/480i-ntsc .

More info here
http://forum.doom9.org/showthread.php?t=154533

You can use CRF encoding with VBV instead of 2pass, it will give you compliant encodes

[poisondeathray]

Yeah, the "interlaced=tff" seems to be a bug in WinFF. I did specify "-top 0" (it's in there) which means bff, but it still output "interlaced=tff". I checked the playback and it looked fine to me. So I did a test, with "-top 1" and it still output "interlaced=tff" but the playback was awful. So it appears that "-top 0" is working, but that the output command is just wrong. <shrug>

So the log was just wrong?

How did you check the playback? What hardware/software? What was doing the deinterlacing?

And regarding the clip, no it wasn't clean at all. Low light, but with a harsh, blown out highlights in the center, and pretty shaky. (An excellent videographer, I'm not.) I thought it would make for a good test. So I guess I'll lower the the crf factor to get the bitrate up. What range would you think is right? I know that's difficult, not seeing the footage. Or should I do it another way?

Not sure, most people would use around 16-22. You used 20, but the log says it ended up with a bitrate ~1641kbps . That seems low to me for a DV source (even clean DV source with very little motion would usually consume a lot more bitrate)

If you don't need a set bitrate or filesize , I would use CRF encoding .

I'm not as familar with doing it though ffmpeg, so there might be something I'm missing

I guess the important thing is how does it look to you

[paleGreen1]

Yeah, I think it's just wrong in the log. I only did a quick playback test (I had to leave), playing it back in WMP 12 and VLC 2.0.1. WMP's deinterlacing looked correct with "-top 0" and horrible with "-top 1". VLC's deinterlacing (on auto) looked correct both ways(!).

Thanks for the suggestion regarding BD compatibility. That's a good idea and I'll definitely check out those links and tweak the encoding profile. I'll post it back here when I decide what to use on all 1.8 TB.

I thought the bitrate was low too. It looked OK, but I didn't do a very close comparison to the source. So I'll take a closer look and try a lower crf if necessary. I need to test some different footage too (outside, well lit, etc).

Thanks again!

[Cornucopia]

I concur with all that pdr said. Should get you a decent end result.

Scott