m2ts to xvid or h264 always gets blockiness

[ubulord]

I have a Sony HDR-CX550VE and use it with the maximum resolution (1920 x 1080 max bit rate). The resulting .m2ts files play fine but it's hardly a very useful format. Whenever I try to convert it into another format (xvid or h264) I run into 2 problems:
1) The image is very jumpy
2) compression blocks are very visible, regardless of bit rate (I've tried everything up to 12000) and compressor (Xvid or H264).
I can solve the first problem if I (for instance) use TMPGEnc 4 Express and choose "deinterlace always (double framerate)" and then output at 50 fps progressive.
I have never found a way of solving the second problem. Blue skies, for instance, invariably get full of blocks. I've tried TMPGEnc, Adobe Premiere, AVS Video Converter, Magix Video Pro but the blockiness is always there. I've tried an infinite number of presets and custom modes, both with a wide range of bit rates.
Also, If I try to convert to Xvid or H264 at 25 fps the movie is invariable jumpy.
Am I missing something? How can I convert .m2ts to 1280 x 720 at 25fps in xvid or h264 without blockiness and with smooth movement?
Keeping the videos in m2ts is not an option, as nowadays (contrary to the old times of DV) when one edits a movie one is forced to output to a recompressed format and AVI - XviD or AVI/MP4 - h264 are the formats with wider compatibility.

[jagabo]

Use Target Quantizer encoding in Xvid with a quantizer of 2 or 3. Or CRF encoding in x264 with a value of 18 to 20.

25 fps will never be as smooth as 50 fps. But I don't know how jumpy your video is so I can't say whether what you're seeing is to be expected or if something is going wrong in your conversions.

[budwzr]

Can't you just record in 720p?

[ubulord]

Thanks for your suggestions Jagabo. I did 12 second samples, one in XviD and one in avc. They both have high bit rates (Xvid 7300, avc 15100) and still the blue sky looks awful in both of them. In fact, it looks a lot worst in avc, inspite of the higher bit rate. From my experience, areas with blocking are always worst in avc than in XviD. Is it too much to ask that you look at my samples? I'm not even sure if I can post a link in the forum, but the fact remains that if no one looks at my samples, I'll never know if I'm just too obsessed with quality or if there is something wrong with my compressions.
Regading budwzr comment, it's irrelevant if I record in 720p or not. To edit the video I must output it to some format, and I always get this awful blocking in one-colour areas like the sky, walls, etc.
The only format in which I get better results is HDV Mpeg. A 10000 bit rate gives me a decent blue sky. Unfortunately, it's a format my friends won't know what to do with it. People want avi's or mp4's. And isn't h264 supposed to be the best thing around? It's the format in which I get worst results .
Link to my xvid and avc samples: http://www.mediafire.com/file/57uro4cbjm33m11/Video%20Tests.rar
Link to my HDVMpeg sample: http://www.mediafire.com/file/41mdvauqci3kcac/HDVMpeg.rar

[poisondeathray]

Thanks for your suggestions Jagabo. I did 12 second samples, one in XviD and one in avc. They both have high bit rates (Xvid 7300, avc 15100) and still the blue sky looks awful in both of them. In fact, it looks a lot worst in avc, inspite of the higher bit rate. From my experience, areas with blocking are always worst in avc than in XviD. Is it too much to ask that you look at my samples? I'm not even sure if I can post a link in the forum, but the fact remains that if no one looks at my samples, I'll never know if I'm just too obsessed with quality or if there is something wrong with my compressions.
Regading budwzr comment, it's irrelevant if I record in 720p or not. To edit the video I must output it to some format, and I always get this awful blocking in one-colour areas like the sky, walls, etc.
The only format in which I get better results is HDV Mpeg. A 10000 bit rate gives me a decent blue sky. Unfortunately, it's a format my friends won't know what to do with it. People want avi's or mp4's. And isn't h264 supposed to be the best thing around? It's the format in which I get worst results .
Link to my xvid and avc samples: http://www.mediafire.com/file/57uro4cbjm33m11/Video%20Tests.rar
Link to my HDVMpeg sample: http://www.mediafire.com/file/41mdvauqci3kcac/HDVMpeg.rar

The short answer for your poor avc results is that you're using poor settings and a not-so-great encoder .




You're using a low quality "baseline" profile. You aren't using any b-frames (impairs compression efficiency) , no cabac, 1 reference frame - these are very inefficient settings.

Beware of broad based generalizations like "AVC is better". There are different profiles , and different types of AVC encoders. Some profiles are used for low quality devices and scenarios, and some for high end mastering (even 10-bit 4:4:4). So there is a wide range of quality. There is even a wide range of AVC encoders. Some are great, some are worse than xvid or mpeg2.

Banding/blocking in gradients is a difficult task to tackle in general.

You can actually see blockiness/banding in all the samples in the skies, and this is a common issue for MPEG2/MPEG4-ASP/MPEG-AVC compression. Could you upload a sample of the source as well ?

The encoder you're using probably isn't using any adaptive quantization - Many encoders have their quantization such that gradients and dark macroblocks to having higher quantizers (lower quality). The thinking is the backgrounds and shadows, skies etc... are not as noticable to the human eye as the main subject

You cannot customize or adjust many features with Mainconcept based encoders (unless you have the higher end versions, or the SDK, but even the SDK is not as good as x264)

You can redistribute bits to flat and dark areas using features like adaptive quantization (AQ). This is a delicate balance because it detracts from other areas. If the strength is adjusted too high, you impair other parts . I've illustrated this before if you dig up some old threads and are interested to learn more

HAQ (Haali AQ) specifically lowers quantizer on blue and flat gradients (like blue skies). Some x264 branches have both HAQ and AQ that can be used (not included in the main branch).

Also, to "encourage" encoders to allocate more bits to these problem areas, many people use dithering filters to pre-process. This is expensive , and takes a lot of extra bits, but it really helps with gradients, even if there is some banding in the source

Another thing is you can lower deadzones (disable trellis) - again this is a bitrate expensive option. This limits the amounts of fine detail that the encoder drops (retains dithering and fine grain, which helps with banding/blocking in gradients)

[Cauptain]

Try MEGUI with PS3 profile 17-22 CQ

[budwzr]

Try AutoGK with 75% quality, and set the width to whatever you want. If you still have a problem then there's no solution.

[Cauptain]

Try AutoGK with 75% quality, and set the width to whatever you want. If you still have a problem then there's no solution.

Yes, I agree.

[ubulord]

You are a godsend, the answer to my prayers. I've been trying to solve this for months. It's 3 am here now so I'll have to digest your comments tomorrow but basically, you say I have to learn more about x264 settings and I'm using a low quality encoder.
1) Where do I get a high quality encoder? I thought these programs like Premiere Pro, TMPGEnc or Magix had good encoders...
2) Where do I learn all the details about costumizing x264's features? But anyway, you say I can't because the encoders I use don't allow costumization...
3) Regarding Xvid, I have access to all the settings that exist (I suppose). I tried turning on adaptive quantization and switching off trellis and the results weren't very noticeable. Appart from editing the matrix (which I don't know how to do) and messing with the B-VOP section (which I also don't know how to do) I don't see much more I can do.
Here is a link to the original file as came out of the camcorder
http://www.mediafire.com/file/c93kw70n2qrv135/Blue%20Sky.rar
You mentiodithering filters to pre-process. In what programs do I get them and is there somewhere on the net where I can learn to use them?
Pegasys's website says that on Nov 25, "it has formally signed an agreement with x264 LLC for the use of the x264 H.264/AVC encoder under a commercial license" and it will be incorporated in future products. Does that mean that TMPGEnc is going to have what you call a decent encoder?
Sorry if I'm asking too much from a total stranger but I really have no one else to turn to.

[ubulord]

Regarding the AutoGK suggestion, I've changed the m2ts to a ts using tsmuxer (because AutoGK didn't accept the m2ts) but when I tried to start the conversion I got a message saying "no video sequence header found". And then another one saying "no data, check your PIDS". I'll try the MEGUI suggestion tomorrow. Thanks to you all.

[budwzr]

If there's no timecode, then you need to use the camera software to stitch the clips together.

EDIT: This camera shoots HD in AVC and Mpeg2 for SD, so you can use AutoGK for the SD files, and AVIDemux for the AVC ones.

I took off my rant, sorry.

[poisondeathray]

but basically, you say I have to learn more about x264 settings and I'm using a low quality encoder.

Yes. I'm 100% sure this is mainconcept's fault. It typically blurs everything , dropping detail especially dark areas and flat areas (like skies). It's a way of "cheating" for improving compressibility. Many encoders do this to some extent (for the rationale mentioned earlier about background vs foreground, etc...), but very few encoders allow (good) control over it.

If you look closely at the source, there is a fine grain or noise in the sky. This noise acts as a natural dither. The encoder you used (and settings used) were insufficient to retain the grain, and those macroblocks have been assigned high quantizers. You see a similar thing with your xvid and mpeg2 encodes, they all exhibit it - a "smeary pastel" along the sky, and the gradients have sharp blocky demarcations instead of fine gradations (look at the exagerrated view screenshots attached to see more clearly). If you search, I've posted on many other threads similar to this topic illustrating this (especially with mainconcept's avc encoder)

Basically it boils down to a more efficient encoder and more efficient settings allow you to retain the noise/grain the same bitrate. More efficient implies higher quality at a given bitrate. HAQ (and it's derivatives like VAQ allow you to control "moving more bits" to those dark areas like shadows (preseving shadow detail instead of becoming blocky typical of xvid encodes) and flat gradient areas like skies.

I've included some screenshots to illustrate in a zip file. The x264 encode ended up being 9.5Mb/s (so roughly 2/3 the bitrate of the mainconcept encode). And how do I know it's a mainconcept based avc encoder? Because I can recognize the typical artifacts made by it. Anyways, the x264 was a b-frame (lower quality), and about 1/2 the size of the mainconcept p-frame, but still it retains more detail, more grain. (I puposely picked it so x264 encode would be shown in the worst light, an even p-frame to p-frame comparison would show even more difference). I've also included an exaggerated luminance view, for those who aren't used to visualizing these typical artifacts - the banding and blocking should be very evident. x264 will ALWAYS deliver better results when used properly. It's better in every way.

1) Where do I get a high quality encoder? I thought these programs like Premiere Pro, TMPGEnc or Magix had good encoders...

They're OK, if you use decent settings. But x264 is better in every way - price, quality, configurability, speed. If you notice your avc encode also has "keyframe popping". (The quality of the I-frame fluctates greatly compared to the P frames and you get a sort of "pulsing"). And guess what? You can't control I:P or P:B (if you were using b-frames) using Mainconcept's encoder (unless you have the SDK or high end versions)

2) Where do I learn all the details about costumizing x264's features? But anyway, you say I can't because the encoders I use don't allow costumization...

There aren't any good guides, but you can start here.
http://mewiki.project357.com/wiki/X264_Settings


You need a bit of experience to use what settings under what circumstances or what sources benefit from particular settings for example

3) Regarding Xvid, I have access to all the settings that exist (I suppose). I tried turning on adaptive quantization and switching off trellis and the results weren't very noticeable. Appart from editing the matrix (which I don't know how to do) and messing with the B-VOP section (which I also don't know how to do) I don't see much more I can do.

xvid doesn't do well in these circumstances, and the AQ was ported from x264, but the strength is not adjustable in the xvid version. A different matrix will definitely help, but you would need huge filesizes compared to a well encoded x264 version

You mentiodithering filters to pre-process. In what programs do I get them and is there somewhere on the net where I can learn to use them?

avisynth . There is a bit of a learning curve, but it basically has filters for just about everything. Gradfun2dbmod is a great diterhing filter

Pegasys's website says that on Nov 25, "it has formally signed an agreement with x264 LLC for the use of the x264 H.264/AVC encoder under a commercial license" and it will be incorporated in future products. Does that mean that TMPGEnc is going to have what you call a decent encoder?

Yes , but you can use x264 for free . Right Now. There are many free gui's available now. Some of them are easier to use than others, some allow more customization and have open commandline boxes (to customize more)

[ubulord]

Very interesting and thanks to all for your answers. I have a lot of stuff to investigate and study. But the main problem remains unchanged. All these suggestions about x264 encoders are just standalone encoders. My problem is that I want to edit videos (combine clips, apply transitions, etc) and then output the final result to a decent format without blocking and banding. These x264 GUIs are not video editors (or only very basic ones). The only solution would be to edit the videos, output the final video to an uncompressed format and then compress that with a standalone x264 GUI. It's not practical and I'm not even sure my computer would be capable of handling an uncompressed file for a 50 minute video. So the big question remains - isn't there a good quality video editor that allows me to output to x264 with configurable features?

[poisondeathray]

Sure you can. x264vfw will integrate into most NLE's like vegas, premiere for example. (It will show up as an option under "AVI" export). There are 2 versions of it , and the menus are a bit different but you can experiment
http://sourceforge.net/projects/x264vfw/
http://komisar.gin.by/

Debugmode frameserver (works with vegas 9 , not sure about 10, and earlier versions of premiere, not CS5) will frameserve out of your editor (essentially it makes a "dummy AVI" than you can use for input into other programs - it allows you to connect your NLE to other programs without a big lossless intermediate)

HDD are inexpensive now, uncompressed or lossless compression (e.g. lagarith, huffyuv, ut video codec) works well (this is what many people use, myself included), but you expressed your reluctance

I wouldn't be surprised if NLE's very soon license x264 like TPMG has

[jagabo]

I ran a few test with the M2TS file. I found I had to use CRF 15 (no fine tuning) to get near transparency in the sky (Yadif deinterlaced, Bilinear resized to 1280x720, in AviSynth, x264 CLI). The resulting MKV file was about ~40 MB. Even then there was a little posterizeation. Results were better with grain tuning but the files were much larger. If I use a higher CRF plus grain tuning to get about the same quality as the untuned CRF 15 encode, the file size is about the same.

[ubulord]

Ok, thanks a lot. I've understood how it's done. It'll probably take me some time to be able to actually manage to do it, but that's another story.
As you are all so kind and willing to help me, I could abuse my luck and try to move to my second (and last) problem: why can't I get smooth movement from a 25 fps compression?
I've uploaded 2 clips with 3 seconds each. One is compressed at 50 fps and is acceptable, the other is compressed at 25 fps and is unwatchable. This happens quite often with the videos I make.
They were both compressed in TMPGEnc and in the filters section I chose "Deinterlace Mode - deinterlace always (double framerate).
This is the link to the 2 cips: http://www.mediafire.com/file/f5tz9nd8jv80np6/Jumpy.rar
If someone wants to view the original they were made from, it's here. It's in 3 files because I have size limitations in Mediafire and the m2ts file is 264 MB:
http://www.mediafire.com/file/dar49lb0a8h0974/Jumpy.part1.rar
http://www.mediafire.com/file/csmc32w19y28587/Jumpy.part2.rar
http://www.mediafire.com/file/pietej1owl70l8p/Jumpy.part3.rar

[jagabo]

Unless you're having some kind of playback problem the "jumpy" 25 fps file looks normal to me. It's no worse than film at 24 or 25 fps. The high contrast (bright windows in the background) makes it more obvious. One thing that will make less smooth than it could be: if your monitor isn't running at 50 Hz refresh (or some other multiple of 25) you will get some frame repeat judder in addition to the inherent jerkiness of 25 fps.

If you are running at 60 FPS try the video in this post as a comparison:
https://forum.videohelp.com/threads/307004-Best-framerate-conversion-%28eg-23-97-to-30-...=1#post1888926

Here's a post with a 24 fps clip from 2001 a Space Odyssey:
https://forum.videohelp.com/threads/307004-Best-framerate-conversion-%28eg-23-97-to-30-...=1#post1888926

And the same clip with motion interpolation to make a 60 fps clip:
https://forum.videohelp.com/threads/305001-Current-HD-standard-is-a-FAIL?p=1872955&view...=1#post1872955

Regarding the blocky h.264 videos, there's one other possible problem: when there is insufficient CPU power h.264 decoders will skip the deblocking process during playback. That will leave the videos more blocky than normal.

[ubulord]

Ok, thanks a lot to you all. I have checked your video samples and reached the conclusion that only video at 50 or 60 fps looks normal to me. Either I'm fussy about it or something in my system is making jerkiness more obvious. I think I'm fussy because on my tv / media players I also see the jerkiness (in home made movies, in commercial ones I only do sometimes).
I started experimenting with MeGui and the results are really much better than what I'm used to.
Thanks again.

[jagabo]

Good movie makers know how to minimize the jerkiness of 24 fps film. They use shallow depth of field so the background is out of focus, motion blur (long exposure), low contrast, etc.

[Cauptain]

All these suggestions about x264 encoders are just standalone encoders. My problem is that I want to edit videos (combine clips, apply transitions, etc) and then output the final result to a decent format without blocking and banding.

Use Power Director 9. Easy learn, efficient and impressive. For any PC output MPEG2 25000 bitrate.

Vegas 10 is powerfull ditor, but require time to learn and good PC.

[Bonie81]

ubulord

hope u r using windows as OS, and all experts here had already replied you in details.
just from you nick name ubulord ---> if u r using Ubuntu, just in case, Linux has way better choices for videos.

@jagabo
i wonder, if you can help on my new thread HERE

As you said "Good movie makers know how to minimize the jerkiness of 24 fps film. They use shallow depth of field so the background is out of focus, motion blur (long exposure), low contrast, etc."

[ubulord]

Sure you can. x264vfw will integrate into most NLE's like vegas, premiere for example. (It will show up as an option under "AVI" export). There are 2 versions of it , and the menus are a bit different but you can experiment

This is a great suggestion. Works ok and turns up on the export / encode menus of the video editors / converters. Spares me all that hassle of frameserving or intermediate lossless files.

Now, as human beings are never satisfied, there is this very tempting box that says "Extra command line (for advanced users)". Is there some site where I can learn what command lines I can use and how to understand the various variables that the command line gives me access to? Never really tried command lines in anything except one or 2 situations when trying out linux, but then I was just copying from somewhere else, not really understanding it.

Use Power Director 9. Easy learn, efficient and impressive.

Tried the 64 bit version. Definitely not convinced. Regardless of the merits of the editor itself, when one gets to the export phase (called PRODUCE on PD9) one is faced with a closed system with a handfull of choices and which does not recognize the codecs that are installed on the computer. What's the use of a good video editor if in the end you are not allowed to freely choose the codecs, the width and height and the framerate?

Unless you're having some kind of playback problem the "jumpy" 25 fps file looks normal to me.

Having been clearly established that I have some wiring in my brain that makes me accept only video with extreme smoothness, I have a problem. I keep using TMPGEnc 4 Express for the simple reason that it has a deinterlace mode called "deinterlace always (double framerate) that (followed by a conversion to 50 fps) I credit (maybe erroneously) with making my videos smooth. Everytime I try other editors (vastly superior in all other departments) the video never seems to be as smooth as the ones coming out of TMPGEnc. Any suggestions on how to achive this smooth result on other NLE, like MAGIX Video Pro, Premiere, whatever?

[jagabo]

x264vfw... there is this very tempting box that says "Extra command line (for advanced users)". Is there some site where I can learn what command lines I can use and how to understand the various variables that the command line gives me access to?

Start here: http://mewiki.project357.com/wiki/X264_Settings

[edDV]

Keeping the videos in m2ts is not an option, as nowadays (contrary to the old times of DV) when one edits a movie one is forced to output to a recompressed format and AVI - XviD or AVI/MP4 - h264 are the formats with wider compatibility.

AVCHD is already h.264. If you recompress h.264 to a lower bit rate you will reduce quality. Your edit master should be at least the same bit rate as the camera master if you want similar quality. You may find x264 constant quality mode results in higher bit rates than the camcorder original depending on motion and noise issues in the content. Xvid would normally require higher bit rates since it is a less efficient codec.

From spec sheet these are your shooting options for bit rate ...

FX (1920x1080i, 24Mbps)
FH (1920x1080i, 16Mbps)
HQ (9Mbps)
LP (5Mbps)

[Cauptain]

Tried the 64 bit version. Definitely not convinced. Regardless of the merits of the editor itself, when one gets to the export phase (called PRODUCE on PD9) one is faced with a closed system with a handfull of choices and which does not recognize the codecs that are installed on the computer. What's the use of a good video editor if in the end you are not allowed to freely choose the codecs, the width and height and the framerate?

For me work...

[ubulord]

[QUOTE=Cauptain;2042774]

For me work...

Let me clarify. Everything's fixed. I can't choose 720 x 400 (or 416), for instance, which actually is something I use a lot. No XviD. I installed x264vfw and it doesn't show up on the compression choices, etc. Basically, it's a closed system. If you're ok with the options that are there, fine, if you aren't, the whole thing becomes useless. Which actually is a pity, why ruin an otherwise good editor by limiting the output options?

[ubulord]

[QUOTE=Cauptain;2042774]

For me work...

Ooops, just noticed that you do have all the options. I don't. Are you using the 64 bit version or the 32 bit version? I've been reading some posts on Powerdirector forums and maybe the closed system is a problem of the 64 bit version. Not sure as I haven't tried the 32 bit version. Which one are you using?

[ubulord]

Keeping the videos in m2ts is not an option, as nowadays (contrary to the old times of DV) when one edits a movie one is forced to output to a recompressed format and AVI - XviD or AVI/MP4 - h264 are the formats with wider compatibility.

AVCHD is already h.264. If you recompress h.264 to a lower bit rate you will reduce quality. Your edit master should be at least the same bit rate as the camera master if you want similar quality. You may find x264 constant quality mode results in higher bit rates than the camcorder original depending on motion and noise issues in the content. Xvid would normally require higher bit rates since it is a less efficient codec.

From spec sheet these are your shooting options for bit rate ...

FX (1920x1080i, 24Mbps)

Just to make sure. You aren't saying that if my edit master is FX 1920x1080i, 24Mbps (which are the settings of the original files out of the camera) the edit master will only have been recompressed in the areas with transitions, effects, etc, like it happens with DV?
Everything I've read says that h264 is always recompressed when edited, even the untouched parts.

[poisondeathray]

Just to make sure. You aren't saying that if my edit master is FX 1920x1080i, 24Mbps (which are the settings of the original files out of the camera) the edit master will only have been recompressed in the areas with transitions, effects, etc, like it happens with DV?
Everything I've read says that h264 is always recompressed when edited, even the untouched parts.

You're referring to "smart rendering"

So far, no software does it properly for h.264 . I think eventually it will be possible (for limited profiles and subsets)

Vegas claims to be able to do it for AVCHD, but it doesn't always work and is very buggy

Videoredo beta is very buggy too, but has improved the last few releases

I think nero claims to be able to do it as well, but if you search their forums, it doesn't work properly either.

Always backup your original files




Regarding power director or any other program - you need to match 64-bit to 64-bit and 32-bit with 32-bit programs and .dlls. x264vfw has 2 versions. If you're using 64-bit , then you need 64-bit xvid as well for it to show up as an option. You can use 32-bit programs in a 64-bit OS

[edDV]

Keeping the videos in m2ts is not an option, as nowadays (contrary to the old times of DV) when one edits a movie one is forced to output to a recompressed format and AVI - XviD or AVI/MP4 - h264 are the formats with wider compatibility.

AVCHD is already h.264. If you recompress h.264 to a lower bit rate you will reduce quality. Your edit master should be at least the same bit rate as the camera master if you want similar quality. You may find x264 constant quality mode results in higher bit rates than the camcorder original depending on motion and noise issues in the content. Xvid would normally require higher bit rates since it is a less efficient codec.

From spec sheet these are your shooting options for bit rate ...

FX (1920x1080i, 24Mbps)

Just to make sure. You aren't saying that if my edit master is FX 1920x1080i, 24Mbps (which are the settings of the original files out of the camera) the edit master will only have been recompressed in the areas with transitions, effects, etc, like it happens with DV?
Everything I've read says that h264 is always recompressed when edited, even the untouched parts.

As poisondeathray says, h.264 "Smart Render" is still a work in progress. Next best is x264 constant quality re-render which will generate similar or larger bit rates. Smart Render bit-rate will by definition be similar to camera master except for re-rendered transition or filtered frames. Lower bit rate settings will lower quality.

Note that AVCHD FX and FH modes are VBR which complicates a smart render design.

BTW, Vegas smart render works fine for DV/HDV/XDCAM-EX and HDCAM.