Blocky conversion DV-->Mpeg2 @ 9000Kbps using TMPGEnc 4.0

[kamaleon]

Hi folks

I'm trying to encode some DV captures from my Panny NV-GS400. I don't seem to be able to get any better results than this:



To the right, the original DV captured. On the right, one of many essays of encoding it to mpeg2, using TMPGEnc 4.0

If you pay close attention to the girl on the right, she's very very blocky. Am I doing something wrong or is this just what it's ever going to look?

In the exemple given, it's a CBR @ 9000Kbps 10bit precision high motion search on. I've tried all sorts of things, VBR average, VBR constant, lower bitrates, etc, etc, but I keep on getting these blocks all over the place.

Can someone give me some hints please? I'd really appreciate it.

Thanks!

[jagabo]

That's normal for TMPGEnc. CCE delivers fewer macroblock artifacts. I don't know about the free MPEG2 encoders but you might try one of them.

[kamaleon]

hi there. So do you reckon i'm not doing anything wrong and this is just how it's supposed to be with TMPGEnc? Gosh i should say i'm very disappointed with such a highly recommended encoder. I'll have a look at alternatives. Care to specify any of those free encoders you mentioned? I didn't even know such a thing existed.

Anyway, thanks ever so much for your time.

[lordsmurf]

That's nothing more than the nature of MPEG compression. Other encoders will only vary slightly.

[kamaleon]

That's nothing more than the nature of MPEG compression. Other encoders will only vary slightly.

You mean you're not kidding? You mean the best encoding possible will always have blocks and artifacts?

[yoda313]

Well remember you changing the original source to a new source. You're going from dv to mpeg. You can't expect it to look exactly like the original. That is what compression is all about. You just try to get it to look decent at a smaller size since dvd isn't the same as dv video.

[kamaleon]

Hi mate.

Yes, but DV is compressed too, and it looks fine. I don't think that because you say "compressed" you have to have bad looking quality.

I thought that mpeg2 could be free from artifacts, but maybe not at a "standard" bitrate of 9Mbps max?

[manono]

I'm not convinced it's the "nature of MPEG compression" that's responsible. It could be you. Could it possibly be because of a bad colorspace conversion? You haven't detailed exactly how you're getting the video into the encoder. Just dropping it in? Is DV AVI by definition RGB24? I don't know. Just asking. Because TMPGEnc will convert to RGB24 before encoding if the source isn't already RGB24. And is that blockiness happening only with the red/orange colors? I've seen that sort of thing in reds and oranges resulting from inferior colorspace conversions.

As for free encoders, HCEnc is about the best, I think. And is said to block up less than some others (maybe not less than Procoder, though). It will require an AviSynth script file, and a conversion to YV12, if it's not YV12 already. I think I agree with jagabo that you might get better results with a different encoder, HCEnc, Procoder, CCE.

And personally, just in case the fast movement is causing the blocks, I'd raise the max bitrate as high as possible without going over the limit of video+audio+subs+overhead=10080. 9500, at least.

[Dosdan]

I'm no expert but I would suggest three things:

1. Try HC encorder. It's free and very well thought of in comparison to the commercial MPEG-2 encoders.

2. Next try noise reduction. You feed HC via an Avisynth script file (scripfilename.AVS) and it's easy to run a sequence of powerful freeware avisynth filters on the DVtransferred.AVI file. I use VdubMod, which also accepts .AVS input, to develop, edit and preview scriptfiles before using them as the input for either the HC or TMPGenc Plus v2.5 MPEG-2 encoders.

3. Try some different matrices in HC. Different matrices vary the balance between bitrate allocation in high-freq (high detail - sharp) and low freq (little detail - smooth) parts of the clip. I believe that one of the reasons MPEG-2 encoding in commercial DVDs looks so good is that they spend skill, time and money selecting different matrices for different scenes in a movie ("hand tuning"). Other reason would be the careful attention to lighting and panning that is used in the composing of the shots.

The lighting flare, smoke and any fog effects creates a lot of extra detail for the encoder to deal with. Noise reduction helps here.

[kamaleon]

You haven't detailed exactly how you're getting the video into the encoder. Just dropping it in? Is DV AVI by definition RGB24? I don't know. Just asking.

Yes, i was just dropping it in... i've never paid attention to colourspaces, thanks for mentioning that!

Is RGB24 what I should be looking out for then?

And personally, just in case the fast movement is causing the blocks, I'd raise the max bitrate as high as possible without going over the limit of video+audio+subs+overhead=10080. 9500, at least.

I'll try and raise the max bitrate to 9500 too then, ok!

[kamaleon]

I'm no expert but I would suggest three things (...)

Great! I'll need a bit of time to digest all that and see what I can come up with, as i'm not very familiar with using avisynth. Thanks for the insights, though

[manono]

Is RGB24 what I should be looking out for then?

If it's not already RGB24, then TMPGEnc will convert it. And I don't much trust an encoder doing the conversion. There are probably a bunch of ways to find out the source colorspace. I use AviSynth, though.

AVISource("C:\Path\To\Video.avi")
Info()#will give you lots of info, including colorspace

Then open that script (placed inside a renamed .txt file renamed as, perhaps, Video.avs), adjusted for your name and paths, in VDub(Mod). If it's something besides RGB24, then remove the Info line and replace it with ConvertToRGB24(). And if it was and is to be encoded as interlaced, then you use ConvertToRGB24(Interlaced=True). Maybe GSpot can give you that information. But as I said, I don't know much about DV video.

If you decide to try out the HCEnc, then the script goes:

AVISource("C:\Path\To\Video.avi")
ConvertToYV12()# or ConvertToYV12(Interlaced=True)

[45tripp]

why not upload a sample of a few seconds to a file sharing site?
You'll get specific and definite feedback then.

gl

[lordsmurf]

Yes, but DV is compressed too, and it looks fine. I don't think that because you say "compressed" you have to have bad looking quality.

#1 - It's not the same compression.
#2 - It only looks "bad" when you freeze frames like that.
#3 - TMPGEnc is a good encoder, but it's far from the best. Procoder and MainConcept are better, but not without their own quirks too.
#4 - All MPEG will have some degree of noise in it because it's a high-compression format. JPEG and MJPEG are in the same boat.

[trhouse]

DV AVI is compressed but still about 13 GB/hour. Mpeg2 lossy compression is going to reduce it to about 4.7 GB/hour at the highest bitrate. Adjusting that bitrate about 10% is not going to get back AVI performance.

[jagabo]

Noisy handheld camcorder footage is especially problematic for MPEG encoders.

[vhelp]

@ kamaleon

fwiw, I thought that the clip looked ok for that scene. At least the pic wasn't so
blocky to my current video workstation.

I didnt' think that it was too bad. And I can see more video details than most here,
because of my graphics card capabilities. And, because you pointed out the exact
area, I could spot them real quickly. But I think I would have had to search for it
if you hadn't -- and still found something.

It would be interesting to see what others would get using the same scene that you are
having trouble in.

Can you please post a short (few [5-10] seconds) dv avi of this. It would be interesting
to see how it comes out on my end as well as others, here. Make sure its not in any
codec (compression) format. Just keep it in its original dv avi format.

Now, I'm not sure if your PAN NV-GS400 (in that scene of the pic you posted) is interlaced
or progressive. At least the pic leaves me the impression that it was in progressive,
though you may have de-interlaced it. I don't know. I'm just guess, now.

Thanks,

<<EDIT>>
hehe, I didn't finish typing at the start.

-vhelp 4339

[thecoalman]

Well if you want some comaprisons here's some from Uleads version of mainconcept, note that this clip will stress any encoder. The lights are flashing and moving very rapidly:

DV:


MPEG 8000CBR:




8000CBR MPEG, 10MB : http://www.nepadigital.com/reencode/8000cbr.mpg

The DV-AVI file if you want to see what results you get. It's only about 10 seconds and no audio, 36MB .

http://www.nepadigital.com/reencode/avidv.avi

Note that it's NTSC

[manono]

Thanks for making the clips available, thecoalman. I seem to remember a couple of times you posted blowups of part of a frame, a before and after, showing the problems with the encode. Can you post that again, so I (we) can see the kinds of problems we're looking for? And can you say which frame it is, and explain how you took the pics, so if we post some of our own, the differences won't be because of how the pics were made? Thanks.

Also, similar clips from kamaleon, one of the source, and one of his encode of the same part, would be welcome.

[jagabo]

I believe thecoalman's images were from frame 212 of the video. I downloaded both the DV AVI file and the Ulead/Mainconcept encoded MPEG file. I also encoded the DV AVI file with TMPGEnc and CCE at 8000 kbps CBR. I used DGIndex to create D2V files and the following AVISynth script to open all four files, crop and enlarge a small section:

V1 = AVISource("dv.avi").ConvertToRGB(interlaced=true). Crop(300, 144, 60, 80)
V2 = MPEG2Source("mc.d2v").ConvertToRGB(interlaced=true ).Crop(300, 144, 60, 80)
V3 = MPEG2Source("cce.d2v").ConvertToRGB(interlaced=tru e).Crop(300, 144, 60, 80)
V4 = MPEG2Source("tmp.d2v").ConvertToRGB(interlaced=tru e).Crop(300, 144, 60, 80)
StackHorizontal(V1, V2, V3, v4)
PointResize(720, 320)

Here is frame 212 from the four videos (via VirtualDubMod):



The order is of course DV, MC, CCE, TMPGEnc.

Keep in mind that one particular frame is not a fair comparison of the codecs. Frame 212 was an I frame in CCE, but a B frame in MC and TMPGEnc. If you scan through the four videos you'll find varying quality in all three enocoders.

That said, you can see that in this particular frame TMPGEnc introduced the most artifacting.

The CCE and TMPGEnc files if anyone wants to look at them (~19 MB):

http://www.badongo.com/file/3668376

[towtruck]

My understanding is that all miniDV cameras produce interlaced, BFF video. In anycase, my Panny does. Try it in GSPOT.

Also, if you use HCenc/AviSynth, you can use the canopus codec which supports pixel_type = YV12, which may minimize colorspace conversion.

I'd be real interested in seeing what kind of results HC will give in your case. If you try it, please share the results. Thanks!

[Jim44]

My understanding is that all miniDV cameras produce interlaced, BFF video. In anycase, my Panny does. Try it in GSPOT.

Be aware, that Gspot may not show interlacing state correctly. Not sure if the developer has addressed it, but I did bring it up to him a few months ago (also noted in one of the comments on Gspot tool page).

A more accurate way to test it is to use AviSynth & VDub as mentioned by manono here:
https://forum.videohelp.com/topic317903.html#1631782

Jim

[towtruck]

Thanks Jim

I'm sure you are right, in that it MAY NOT show TFF/BFF correctly. I originally tested using manomo's trick. In my particular case, that trick indicated BFF, as did GSPOT. I don't know how field order is detected in GSPOT, but I wonder if the reliability of the indication varies with source material, in other words, could GSPOT give good results for DV-avi as opposed to some other format?

In any case, I have heard that DV from a miniDV camera is always BFF. Can nyone contest that??

[Jim44]

but I wonder if the reliability of the indication varies with source material, in other words, could GSPOT give good results for DV-avi as opposed to some other format?

My understanding from my (brief) conversation with the developer is that the problem was specific to DV-AVI. Other formats that I've tried appear to be reported correctly.

Jim

[jagabo]

I don't know what this has to do with the original subject, but GSpot is not reliable as far as field order is concerned.

It reports all DV AVI files as interlaced and BFF. I have progressive DV AVI files from my camcorder and intentionally created TFF DV AVI files. It reports them as as interlaced and BFF. I don't know if DV has interlace/field order flags and GSpot is simly reporting them, or if it simply assumes interlaced and BFF when it sees DV AVI.

Here's a discussion involving MPEG files and what several different programs (including GSpot) report about field order:

https://forum.videohelp.com/topic330966.html#1713538

And more to the point, this post:

https://forum.videohelp.com/topic330966-30.html#1713720

[thecoalman]

Thanks for making the clips available, thecoalman. I seem to remember a couple of times you posted blowups of part of a frame, a before and after, showing the problems with the encode.

I was actually going to post that above but can't seem to find it, If you want to maintain accuracy the best way that I know of is to save the frame in lossless format such as .tif or .bmp from your editor. Then zoom in on them in an image editor and take screenshot, note I said zoom. If you resize the image it will distort the results. The image I had may not be entirely accurate because it was generated from the screenshots I posted above so jpg compression may play a small part in it.

[manono]

Thanks, thecoalman,

I did as you suggested, by taking a small cropped BMP pic in VDubMod, and then using a picture editor to zoom it, rather than resizing it in either the script or in the picture editor. I swear, when doing a Windows Slideshow between the zoomed one and the one resized in the script, I could see no difference at all. And since the resized one made a PNG when capped in VDubMod about 25% the size of the one made by my picture editor, I used it instead.

There was a request to throw HCEnc into the mix. As I was curious about it myself, I decided to do it. I made videos from the source using both CCE and then HCEnc. I took a pic using the exact same script as jagabo's. From left to right, I go Source DV AVI/Main Concept/CCE/HCEnc. You can tell some of the differences by just looking at the pic. And you can surely tell the difference between the TMPGEnc encode and the HCEnc encode by looking at the 4th one over in jagabo's above, and mine. What I would suggest, for anyone curious, is to save both pics to a folder and run the Windows Slideshow (or some other slideshow) on them. You'll see not only the huge difference between the TMPGEnc one and the HCEnc one, but also subtle differences between the 2 CCE encodes. I don't know if one is better than the other, but they are different, attributable to the different settings used. By the way, I purposely made the sizes of my 2 encodes just slightly less than that of the Main Concept encode that thecoalman uploaded.

I would also submit that both the CCE and HCEnc encodes look very slightly better than the Main Concept encode. Using Frame 212, both the CCE and HCEnc encodes fall on I-Frames, while the MC encode is a B-Frame. "Okay", you say, "that's unfair. How does a Main Concept I-Frame look compared to the 2 others?" I have no interest in TMPGEnc, so I didn't do an encode using it. And after not much looking around, I couldn't find a frame that used an I-Frame for all 3 encodes. So, I used one that is an I-Frame for MC, and B-Frames for both CCE and HCEnc, frame number 195:

I submit that maybe - maybe - the Main Concept one is slightly better this time. But the differences are much less than between CCE/HCEnc and TMPGEnc, and even between CCE/HCEnc and Main Concept in the earlier pics. If I had to rank them, I'd say the differences are hardly noticeable, but in order from best to worst, MC, HC, CCE. Again, from left to right, Source/MC/CCE/HC.

Edited Later: After a discussion with, and after receiving an education from, jagabo in the posts after this one, I've remade the pics, replaced the 2 important ones, and removed the third, along with the discussion of it. None of my conclusions have been changed, though, and they remain the same.

[jagabo]

In quite a few frames, the colors of the Main Concept encode are way off. This is frame number 173:... You can easily see the yellow stripes in the light at the top, and the blue stripes for most of the way down (2nd pic over), where none of the other 3, not the source, nor the CCE and HCEnc encodes, show this aberration.

Actually, all four of my images look like the MainConcept image:



DV, MC, CCE, TMPG

I believe your DV decoder is converting to YV12 as if the source is progressive. I used Cedocida and forced YUY2 output.

By the way, I just noticed that I upscaled incorrectly in my original AVS script. I originally had 3 60x80 samples side by side (180x80 total) and upscaled by 4x to 720x320. Later I added the 4th sample but forgot to change the scaling. It should have upscaled to 960x320. So the images that are posted have incorrect aspect ratios.

[manono]

So the images that are posted have incorrect aspect ratios.

Yeah, I knew that already, wanted mine to be able to match yours, and didn't think it important for our purposes.

I believe your DV decoder is converting to YV12 as if the source is progressive. I used Cedocida and forced YUY2 output.

Sorry, but none of that means anything to me. I used the ConvertToRGB(Interlaced=True) as you did in your script, and I open at my default Fast Recompress. Are you saying I'll get different results using Full Processing?
Edit: No, Full Processing doesn't change anything.

[jagabo]

I believe your DV decoder is converting to YV12 as if the source is progressive. I used Cedocida and forced YUY2 output.

Sorry, but none of that means anything to me. I used the ConvertToRGB(Interlaced=True) as you did in your script, and I open at my default Fast Recompress. Are you saying I'll get different results using Full Processing?
Edit: No, Full Processing doesn't change anything.

I believe your DV decoder is putting out YV12 but it is using progressive algorithms rather than interlaced. So the chroma subsampling is being performed incorrectly. This results in blending the two fields' chroma channels. Here's the same files and script with Cedocida set to output progressive YV12:



Notice the DV sample (leftmost) looks just like yours. What VFW DV decoder are you using?