what output resolution to choose

[codemaster]

I want to convert a video, from a lossless format, to a lossy format, but I don't know what resolution to choose.

I've got some mpeg files, from a camcorder, that I want to edit, and then export to something lossless like lagarith, and then encode with x264.

If the source is mpeg2, 576i (720 x 576 px), 25 fps, then, from the options below, what resolution would you recommend for the output (and why):

Option 1: 720 x 404 px. Height is resized in avisynth, from 576 px to 404 px, to maintain an aspect ratio as close as possible to 16:9. Vertical resolution is reduced, 172 px are lost.

Option 2: 852 x 480 px. Width and height are resized in avisynth. Width is interpolated from 720 px to 852 px, and is mod2, not mod16. Height is resized from 576 px to 480 px. Vertical resolution is reduced by 96 px, less than Option 1.

Option 3: 720 x 576 px. Width and height are not resized in avisynth. Aspect ratio has to be corrected automatically by video players and the mkv container. There are no vertical and horizontal black bars in the video, so cropping is not required.

[Krispy Kritter]

Assuming you want TV playback, you generally don't want to change the resolution (if possible) to maintain quality.

[Selur]

I agree you normally want to keep the resolution and the pixel aspect ratio the same for maintaining max quality.

[Keyser]

I agree. Option 3 (720x576) is by far the best. If, for some obscure reason, you were converting to a container that wouldn't support non-square pixels, I would advise you to resize to 1024x576, thus mantaining the number of scan lines.

[hello_hello]

In theory I'd go for 720x576 or I'd resize to 1024x576. These days I'd mostly do the latter as there's a TV (built-in USB media player) and a Bluray player in this house that don't obey aspect ratios in MKV or MP4 files etc, so I gave up and now I resize everything to square pixels instead.

In practice I'd be likely to do something different anyway. I don't have any experience with Camcorder video (only PAL DVDs) but I often resize progressive 16:9 PAL to 960x540. It's 16:9, there's a little less video to encode than 1024x576 (smaller file sizes) and I can't see any loss of picture detail. In fact the resized to square pixels with spline36resize encode tends to look a little sharper running fullscreen on my TV than the original DVD video.

If it's interlaced, I'd de-interlace to 50fps rather than 25fps, preferably with QTGMC doing the de-interlacing. I mostly resize interlaced PAL DVDs to a height of 480. I guess the theory is similar to de-interlacing 1080i while resizing it to 720p. With a decent de-interlacer you'd probably have difficulty claiming one has more picture detail than the other, especially if the 720p version is 50fps. My experience when de-interlacing and resizing 576i video to 480p at 50fps is quite similar, but that's my experience with DVDs. Camcorder video might be a different story for some reason and my resizing theory could be completely silly. If it is, I'm sure someone will point it out.

If I was thinking of resizing to 720x404 I'd try to forget that and resize to 704x396 instead. It's exactly 16:9.
If I was thinking of resizing to 852x480 I'd seriously consider 832x468 as an alternative. It's exactly 16:9.

Is Camcorder video 16:9 or does it use the ITU aspect ratio, making it slightly wider?

[Skiller]

Is Camcorder video 16:9 or does it use the ITU aspect ratio, making it slightly wider?

SD Camcorders (DV and DVD ones, and all analog ones) follow ITU aspect ratio (tested a few myself over the years, never came across one that doesn't). Generally, native SD equipment almost always does. Downconverting HD equipment usually does not in my experience.
So PAR is 16:11 here.

[hello_hello]

Is Camcorder video 16:9 or does it use the ITU aspect ratio, making it slightly wider?

SD Camcorders (DV and DVD ones, and all analog ones) follow ITU aspect ratio (tested a few myself over the years, never came across one that doesn't). Generally, native SD equipment almost always does. Downconverting HD equipment usually does not in my experience.
So PAR is 16:11 here.

Thanks for the info. I had no idea so I thought it'd be worth asking. It'd annoy me to edit/re-encode a large amount of camcorder video only to discover sometime down the road the aspect ratio was a tad off. Although I guess the aspect ratio can be changed at any stage if need be.

According to MeGUI that makes the aspect ratio for the video in question 1.823361, or it'd be resized to 1050x576 in square pixels dimensions. It appears when calculating resizing and aspect ratios for DVDs, if you tell it to use an ITU aspect ratio, MeGUI uses the "almost exact and commonly used ITU-PAR" from the second list of PAR's here. (resulting in 1050x576 resizing for PAL 16:9).

[Skiller]

In my opinion the ones in the second list ("almost exact and commonly used ITU-PARs") should not be used because they got the half lines thing wrong. Brother John also mentions this.

They ignore the fact that the active analog picture is made up of 575 lines rather than 576 so the error with those is 576:575. The MPEG-4 PARs stick very close to the real ITU ones (much closer than the "almost exact ones") so I'd suggest to use those or the exact ones (personally I prefer the MPEG-4 ones because they are easy to remember).

MPEG-4: 720 × (16:11) = 1047.27*
Exact ITU: 720 × (4600:3159) = 1048.43
Half lines wrong ITU: 720 × (512:351) = 1050.26

* I would round this up to 1048 rather than down to 1047.

[hello_hello]

In my opinion the ones in the second list ("almost exact and commonly used ITU-PARs") should not be used because they got the half lines thing wrong. Brother John also mentions this.

I think you're correct about that. I thought about it after posting, and the mpeg4 PAR is definitely closer to the exact ITU aspect ratio.
I did the math and I'm pretty sure MeGUI uses the PAR's in the second list. I might suggest it should be changed in the MeGUI thread at doom9.

[Cornucopia]

@Skiller, you are mis-calculating WRT # of lines.
Both NTSC & PAL analog signals record on a (gentle) diagonal, so the 525 & 625 signals, which start and end in the middle of the screen (due to sweep of CRT beam starting from point of origin). Then there is blanking vs. active picture. But even with active picture, one is left with 2 fields of (for PAL) 287 + 1/2 lines. Since it is IMPOSSIBLE to create a straight rectangle using 1/2 a line on either end (which would make it "jagged"), one has to either round UP to 288 or round DOWN to 287. Rounding down loses actual "video information", so engineers have always rounded UP. 2x 288 = 576. I believe you were mistakenly adding 1/2 + 287 + 287 + 1/2 = 575 (instead of Rnd 1/2 => 1 for 1 + 287 + 287 + 1 = 576), or you were quoting non-technical websites that also mis-calculated (including that linked German site of Brother John's, which is also mistaken in the same way). A check with SMPTE & ITU documents will verify what I demonstrated.

The return of your calculations to a 576 base, and an understanding that:
1. 702 might be as close an integer approximation for the digital sampling of the analog line, but it did not work with the coding & electronics of that era, so 704 was elected as the transfer equivalent. This gave you a 0.285% error in overall AR from the perfect, one that would never be noticeable by anyone. Remember, 702 is still an approximation also, there is never any EXACT, PEFECT equivalent, so using the industry-standard 704 makes just as much sense.
2. 704 has a greater error for NTSC, but is workable there also, giving engineers a common digital horizontal with which to work. That is the base you should be working from.
3. The 704 -> 720 conversion is never considered to be done using a stretch, but with padding/cropping. This maintains the existing (if imperfect) AR.
4. Using these units for your base (704 & 576) makes calculations simple: for 4:3 DAR you use 12:11 PAR, for 16:9 DAR you use 16:11 PAR. Done.
5. All these calculations never expected square pixel equivalents. That is a recent development accompanying the rise of HD & computer-originated imaging.

In answering the OP's original question:
While horizontal sampling is a spread function, vertical sampling is discreet/quantized. So a resize on its axis is more detrimental than one on the horizontal axis. PARTICULARLY if you were dealing with interlaced material.

I would: resize to 768x576 for 4:3 material (12:11 PAR), and to 1024x576 for 16:9 material (16:11 PAR) - if assuming 704-based image core and digital-friendly PAR values. Assuming you could not use non-square pixels (if you could, you wouldn't need to resize at all).

Scott