Some strange video

[SameSelf]

I was given some video that was shot using a Canon P&S camera, no idea the model number. The Mediainfo is kind of strange. Is this really exactly 30.000 fps? Would that make it Drop Frame video? I thought that frame rate didn't really exist for NTSC, iow everything is really 29.97 fps Non Drop frame. On another note, given that it is SD with square pixels in a MPEG-4 format, is blu-ray the best optical destination or DVD? Since DVD only supports 29.97 fps and non-square pixels, it seems that I would have to do too much to it to burn it to DVD. But the more I think about it, the more it seems this video is already perfectly encoded for web or PC playback with the exception of maybe the audio which is 44.1 vs 48. Please feel free to straighten me out on any of these points. Thanks.

General
Complete name                            : D:\MVI_1024.MOV
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt  
File size                                : 156 MiB
Duration                                 : 2mn 2s
Overall bit rate                         : 10.6 Mbps
Encoded date                             : UTC 2015-06-18 20:38:39
Tagged date                              : UTC 2015-06-18 20:38:39

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Baseline@L3.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 1 frame
Format settings, GOP                     : M=1, N=15
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 2mn 2s
Bit rate                                 : 9 901 Kbps
Width                                    : 640 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 30.000 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 1.074
Stream size                              : 145 MiB (93%)
Language                                 : English
Encoded date                             : UTC 2015-06-18 20:38:39
Tagged date                              : UTC 2015-06-18 20:38:39
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Audio
ID                                       : 2
Format                                   : PCM
Format settings, Endianness              : Little
Format settings, Sign                    : Signed
Codec ID                                 : sowt
Duration                                 : 2mn 2s
Bit rate mode                            : Constant
Bit rate                                 : 705.6 Kbps
Channel(s)                               : 1 channel
Channel positions                        : Front: C
Sampling rate                            : 44.1 KHz
Bit depth                                : 16 bits
Stream size                              : 10.3 MiB (7%)
Language                                 : English
Encoded date                             : UTC 2015-06-18 20:38:39
Tagged date                              : UTC 2015-06-18 20:38:39

[Cornucopia]

Many P&S cams create SD video which is not "industry standard broadcast-compatible" but more "IT compatible". They know their market. This is likely how it is presented to the audience: "post it directly onto Youtube...". Low end, throwaway type.

However, you are mistaken, it would NOT take much to "rehabilitate" this into something workable for a DVD (here, I am going to assume you mean the "DVD-Video" authored spec for CE playback).
It's AVC, so it's not supported anyway and will need to be re-encoded. As will the audio (because it isn't 48kHz).

I would:
Use ffmpeg to demux/remux the Audio stream into a WAV file.
For the video:
Decode into AVISynth.
Use "AssumeFPS(ntsc_video)" - slows down by 0.0999%, not noticeable now just need to slow down audio.
Use "BicubicResize(clip, 704, 480)" and then "AddBorders(clip,8,0,8,0)" - stretches the horiz correctly, then pads to full 720 expected for most DVDs.
Then, load this into HCEnc and encode to fit your DVD, using 4:3 DAR and NTSC DVD-compliant settings (except the Progressive encoding + Progressive Sequence header flag).
Then, add audio (see below), author & burn

For the audio:
Open in Audacity
Use "Change Tempo" and use that percentage change from above. This will stretch the audio very slightly to maintain synch.
Use "Resample" to resample it to 48kHz.
Export to mono 48kHz WAV. (this is 768kbps, so easy enough to leave as-is for bitrate budgeting and not re-compress to AC-3, etc)

That's it.

Scott

[SameSelf]

Thanks, Scott! I appreciate the workflow tips. I will probably be posting back here as I dig deeper into this process. I tend to prefer using After Effects cuz I like to process my video with denoising, sharpening, color correction and grading, and filmic blending before bringing into PPro. So quick follow up question.

Do you recommend decoding and processing the video in AVISynth before bringing into AE? I want to minimize the generation loss of course. Or, can most of that be done in AE (I need to dig into the effects). I will frameserve out of PPro to HCenc, so thanks for those workflow tips.

[newpball]

...with denoising, sharpening, color correction and grading, and filmic blending before bringing into PPro.

Filmic blending?

What do you mean by that?

Also:

Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Before you get to color grading you might want to clean up this 'Canon signature' mess first.

[SameSelf]

Also:

Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Before you get to color grading you might want to clean up this 'Canon signature' mess first.

I am not sure what you mean. What is the problem exactly and how do you suggest I clean the "Canon signature mess"?

[SameSelf]

OK, after spending some time thinking through Scott's recommendations, I have tried the following workflow:

1. Keep everything in its 640x480p30.000 square pixel format as I render from AE for PP import
2. Set up a PP project matching the video exactly and build timeline
3. Frameserve out of PP via DMFS to AVISynth and apply the resize and border adjustment as outlined above
4. Encode using HCenc

However, I am having trouble with HCenc recognizing the AVS file. Here is my AVISynth script:

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
BicubicResize(704,480)
AddBorders(8,0,8,0)

Am I doing this right?

[poisondeathray]

What is the error message or hcenc log file say ?

Can you preview the avs script in vdub or avspmod, and is there an error message ?

[newpball]

Also:

Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Before you get to color grading you might want to clean up this 'Canon signature' mess first.

I am not sure what you mean. What is the problem exactly and how do you suggest I clean the "Canon signature mess"?

If it is a mess depends on who you ask and under what circumstances.

In one extreme you will get an answer that HD as we know would be a complete disaster without Rec.709 while in other cases you will get 'a bit of gamma difference or primary mismatch will not break the bank'. It's mostly a job security thing in my opinion. Wait till you get smpte170m and bt470m in the mix. The soup concocted by brilliant engineers, after all NTSC = Never Twice the Same Color.

Of course some engineers can explain it all and will tell you this whole zoo is one stroke of brilliance and that the critics are just trolls!

Needless to say that converting with the wrong coefficients will give noticeable differences.

However I remain to be intrigued by your filmic blending comment. If that means you are going to blend your 480p30 into 24p then please don't, you will butcher the video.

[hello_hello]

Also:

Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Before you get to color grading you might want to clean up this 'Canon signature' mess first.

I am not sure what you mean. What is the problem exactly and how do you suggest I clean the "Canon signature mess"?

If it is a mess depends on who you ask and under what circumstances.

They don't need to always be the same do they? Which would make sense. If they all had to be the same, why would you have more than one colour option?

https://en.wikipedia.org/wiki/SRGB
sRGB uses the ITU-R BT.709 primaries, the same as are used in studio monitors and HDTV, and a transfer function (gamma curve) typical of CRTs.
https://en.wikipedia.org/wiki/SRGB#Background
Much software is now designed with the assumption that an 8-bit-per-channel image file placed unchanged onto an 8-bit-per-channel display will appear much as the sRGB specification recommends. LCDs, digital cameras, printers, and scanners all follow the sRGB standard.

I'm not sure I've quite got my head around it, but I think it's basically showing the video should be converted to RGB on playback using BT.601 (matrix coefficients) I assume because it's standard definition, and the resulting RGB will display most accurately on a BT.709/sRGB calibrated monitor (colour primaries)..... or something to that effect. Maybe someone who understands it properly will come along, but I'm not sure there's necessarily any mess involved.
https://en.wikipedia.org/wiki/RGB_color_space#Specifications

The way I interpret the Transfer Characteristics shown in the pdf here, BT.709 and BT.601 are exactly the same so that doesn't seem very messy either.
I'm pretty sure Handbrake always writes the Transfer Characteristics as BT.709. Maybe that's why? It doesn't matter.

In one extreme you will get an answer that HD as we know would be a complete disaster without Rec.709 while in other cases you will get 'a bit of gamma difference or primary mismatch will not break the bank'. It's mostly a job security thing in my opinion. Wait till you get smpte170m and bt470m in the mix. The soup concocted by brilliant engineers, after all NTSC = Never Twice the Same Color.

smpte170m and bt470m are the same as BT.601 aren't they? At least in respect to the "matrix coefficients". They seem to be according to the pdf I linked to above.
I've not sure I've ever seen MediaInfo display smpte170m or bt470m. It just displays BT.601.

Of course some engineers can explain it all and will tell you this whole zoo is one stroke of brilliance and that the critics are just trolls!

Could you argue the critics don't always have their facts straight?

Needless to say that converting with the wrong coefficients will give noticeable differences.

How do you convert with the wrong coefficients if you're just re-encoding and there's no conversion to/from RGB taking place during the conversion process?

Using x264’s color coefficient flagging properly.

[SameSelf]

OK, after spending some time thinking through Scott's recommendations, I have tried the following workflow:

1. Keep everything in its 640x480p30.000 square pixel format as I render from AE for PP import
2. Set up a PP project matching the video exactly and build timeline
3. Frameserve out of PP via DMFS to AVISynth and apply the resize and border adjustment as outlined above
4. Encode using HCenc

However, I am having trouble with HCenc recognizing the AVS file. Here is my AVISynth script:

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
BicubicResize(704,480)
AddBorders(8,0,8,0)

Am I doing this right?

LOL, I figured out my problem. The signpost was sitting on D:\ not E:\. HCenc wasn't saying anything, no error, no logfile, nothing. It just refused to open the AVS file, which is why I was confused. After fixing the path, I encoded a 15 sec clip following the workflow above, but I had to add ConvertToYV12(). For some reason, when I choose YUY2 in DMFS, HCenc will go through the motions but doesn't output anything. After choosing RGB24 and adding ConvertToYV12(), it was encoded fine and Encore accepted the clip without triggering the transcode flag. So success?

The only question I have now is, Scott said:

load this into HCEnc and encode to fit your DVD, using 4:3 DAR and NTSC DVD-compliant settings (except the Progressive encoding + Progressive Sequence header flag).

What exactly is the Progressive encoding + Progressive Sequence header flag? I poked around and found under 'Interlace Options' a progressive radio button and clicked that. Then also under 'chroma downsampling', a progressive radio button and clicked that as well. But Scott sounds like he is referring to something else.

Here is the log file:

              -----------------------------------------
              | HCenc - MPEG2 encoder - rel. 0.27.0.1 |
              -----------------------------------------

MPEG profile@level:    MP@ML
input:                 d:\signpost.avs
output:                D:\Test.m2v

              -----------------
              | Avisynth file |
              -----------------

file: d:\signpost.avs

      AVISource("D:\signpost.avi", audio=false).AssumeFPS(30000,1001)
      BicubicResize(704,480)
      AddBorders(8,0,8,0)
      ConvertToYV12()

              --------------------
              | encoder settings |
              --------------------

profile:               BEST
frames:                0 - 463
framerate:             29.970
aspect ratio:          4:3
chroma format:         4:2:0
bitrate:               NA
max. bitrate Kb/s:     9000
pass:                  1 (Constant Quant)
constant Q:            7.000
pulldown:              no
closed gops:           no
VBV check:             yes
VBV size (bits):       1835008
scene change det:      yes
interlaced:            no, BFF
goplen,B-pic:          15 2
dc_precision:          10
intraVLC:              auto
scan method:           zigzag
bias:                  0
chapter frames:        1
time code:             0  0  0  0
CPU:                   MMX,SSE2,SSE3,SSSE3,SSE4.1,SSE4.2,AVX
priority:              high
SMP active:            yes
matrix:                MPEG
luminance gain:        no
adaptive quantization: yes, strength 2
deadzone quantization: auto

              ------------------
              |  source stats  |
              ------------------

nr. of frames in source:         464
width*height:                    720x480
color space:                     4:2:0 (YV12)
fps:                             29.970
nr. of frames to encode:         464
frames to encode:                0 - 463

              ---------------------
              | encoding - pass 1 |
              ---------------------

pass 1 encoding time:            0:00:05 (5.27 s)
fps:                             88.0

              ------------------
              | encoding stats |
              ------------------


intra matrix used
   8  16  19  22  26  27  29  34
  16  16  22  24  27  29  34  37
  19  22  26  27  29  34  34  38
  22  22  26  27  29  34  37  40
  22  26  27  29  32  35  40  48
  26  27  29  32  35  40  48  58
  26  27  29  34  38  46  56  69
  27  29  35  38  46  56  69  83

non-intra matrix used
  16  17  18  19  20  21  22  23
  17  18  19  20  21  22  23  24
  18  19  20  21  22  23  24  25
  19  20  21  22  23  24  26  27
  20  21  22  23  25  26  27  28
  21  22  23  24  26  27  28  30
  22  23  24  26  27  28  30  31
  23  24  25  27  28  30  31  33

nr. of gops:          32
nr. of frames:        464
nr. of I-frames:      32
nr. of P-frames:      125
nr. of B-frames:      307
average bitrate:      2459
minimum bitrate:      1997
maximum bitrate:      8239

bytes in bitstream:   4762009
bits in bitstream:    38096072

average Quantizer:    12.580

total CPU time:       0:00:05 (5.23 s)
total elapsed time:   0:00:05 (5.39 s)

[jagabo]

Oops, wrong thread.

[SameSelf]

DVD uses MPEG DAR, not ITU rec.601. You should resize your 640x480 4:3 DAR video to 720x480, not 704x480 and adding 8 pixel borders to the left and right.

Really? My understanding of NTSC video is that even though it is 720x480, the active picture area is only 704 pixels and that the extra 8 pixels on either side can either have video in them or be blank depending on the camcorder. But since 704x408 isn't a 4:3 aspect ratio (and neither is 720x480 for that matter), it is by squeezing the horizontal resolution by 0.909 that you get to the 4:3 display ratio of 640x480. So if starting from 640x480, you would need to repeat the process as outlined by Scott: resize to 704x480, then add borders.

If I resize as you suggest than that leaves part of my picture out of the active picture area.

Also, I am puzzled by your ITU rec.601 comment. My knowledge of colorspaces is very rudimentary at best. Can you please elaborate how this relates to the aspect ratio?

Anyone care to clarify? Thank you.

EDIT: I missed your edit as I was typing my response.

[jagabo]

DVD uses MPEG DAR, not ITU rec.601. You should resize your 640x480 4:3 DAR video to 720x480, not 704x480 and adding 8 pixel borders to the left and right.

Really? My understanding of NTSC video is that even though it is 720x480, the active picture area is only 704 pixels and that the extra 8 pixels on either side can either have video in them or be blank depending on the camcorder.

That's the ITU rec.601 spec for digitizing analog video. The DVD spec refers to the MPEG 2 spec regarding aspect ratios. The MPEG 2 spec is very clear: the full frame comprises the aspect ratio, not the inner 704x480 as in the ITU spec.

If I resize as you suggest than that leaves part of my picture out of the active picture area.

An upscaling DVD player will include the entire frame at the digital outputs.

[SameSelf]

Good news, I have sorted out the video portion of the stream unless anyone has some further clarifications. There has been some chatter about the colorspace, but someone with more expertise needs to weigh in. I am just not qualified to understand where I might be messing things up.

As for the audio, what a mess! Not only is it 44.1 kHz, but it is also mono! So I gotta stretch it, resample it to 48 kHz, as Scott suggests, and also simulate some stereo imaging.

[newpball]

... and also simulate some stereo imaging.

Why?

[SameSelf]

Why not?

[jagabo]

There has been some chatter about the colorspace, but someone with more expertise needs to weigh in. I am just not qualified to understand where I might be messing things up.

Basically, you have two common ways of converting between YUV and RGB in digital video: rec.601 and rec.709. The way they handle color is slightly different.

https://forum.videohelp.com/threads/329866-incorrect-collor-display-in-video-playback?p...=1#post2045830

Generally, you should use rec.709 for HD, rec.601 for SD. MediaInfo is showing rec.601 for the conversion matrix, the same as DVD so you shouldn't need to change that. If you camera shot rec.709 you would need to use ColorMatrix(mode="rec.709->rec.601") to get proper colors.

Beyond that there are two different ranges for the luma channel (Y) with both systems. DVD uses the limited range of 16-235. Some cameras use the full range of 0-255 to get better granularity. If your camera uses the latter you need to reduce the range with ColorYUV(levels="PC->TV") or you will get crushed blacks and blown out brights when you watch the DVD.

[jagabo]

Why not?

Fake stereo from a mono source sounds like crap. I'd rather hear the original mono any day.

[Cornucopia]

You won't get BACK to stereo from mono (at least not satisfactorily). But DVDs work with mono streams just fine, so just author/mux it that way. And your cam probably only has a mono mic, so that really is as good as it could get anyway.

Since the cam was originally a digital cam, it probably DOES make more sense to go the direct 640->720 route (rather than the 640->704, pad->720 preferred for analog sources), but the difference is 2% and not really enough to worry or quibble over.

Scott

[newpball]

On the other hand if the sound source is relatively anechoic and phase coherent, impulse response convolution could place him in any virtual building he wants, he then could capture a full 'living stereo' image.

[SameSelf]

Since the cam was originally a digital cam, it probably DOES make more sense to go the direct 640->720 route (rather than the 640->704, pad->720 preferred for analog sources), but the difference is 2% and not really enough to worry or quibble over.

I am not quibbling over anything. Just trying to understand completely. If I go 640->720 then won't the part of the picture be excluded on a TV? Or do most TVs display the full 720 pixels now? Just want to make sure I fully understand.

[newpball]

Since the cam was originally a digital cam, it probably DOES make more sense to go the direct 640->720 route (rather than the 640->704, pad->720 preferred for analog sources), but the difference is 2% and not really enough to worry or quibble over.

I am not quibbling over anything. Just trying to understand completely. If I go 640->720 then won't the part of the picture be excluded on a TV? Or do most TVs display the full 720 pixels now? Just want to make sure I fully understand.

No way, there is this 'brilliant' feature called over scan. It made made sense a gazillian years ago but, guess what, it's still there!
Necessary and a brilliant feature according to some that just got to stay.

[_Al_]

What exactly is the Progressive encoding + Progressive Sequence header flag? I poked around and found under 'Interlace Options' a progressive radio button and clicked that. Then also under 'chroma downsampling', a progressive radio button and clicked that as well. But Scott sounds like he is referring to something else.

That will make progressive stream, but chroma down-sampling is not happening if you feed HcEncoder with YV12 (done in Avisynth), so in that case perhaps does not matter what is set there, no YUY2 to YV12 conversion (if HcEncoder supports it), therefore no chroma downsampling. Not sure why YUY2 input did not work, never actually tested feeding HcEncoder with YUY2.

[Cornucopia]

What jagabo & I were discussing has NOTHING to do with overscan. Has to do with digital sampling from analog vs. creation from digital, and reconciling the 2 keeping in mind MOD16 math (very important at the time, less so now).

Overscan is just another of newpball's hotbuttons, and an inaccurate & derogatory red herring (as usual).

re:Overscan,
There are still a few new (as well as millions of existing) monitors & TVs that have some sort of bezels, but 95% of newer models have negligable to no overscanning (and this is sometimes dependent upon which input & res settings chosen). Regardless, any decently professionally produced visual would have taken overscan into consideration and avoided placing important static visual items outside of title-safe, let alone action-safe. Even without the overriding need to account for possible overscan, it is still a good design practice so I don't think that is going to change any time soon (nor should it IMO). For a non-pro show from a consumer cam, this may not have been followed, however, so YMMV.

Scott

[SameSelf]

Many thanks to everyone who has helped me think through this. I have come to the conclusion that the only way to be entirely certain about these details is to start testing various options. I am a huge newbie in this realm, but I guess everyone has to learn at some point. Fortunately, I have some DVD-RWs to avoid making a bunch of coasters because ultimately it comes down to how the picture looks on a television. I have learned the hard way numerous times that WYSINWYG in terms of PC to TV. I will try to post back here with what I find.

[SameSelf]

I am wondering if there is a way to avoid resizing the image and just mapping the 640x480 square pixel image into a 720x480 non-square pixel image? I tried a couple of different resizing BicubicResize(704,480) and LanczosResize(704,480) and tried to compare them to the original image. Here are some ~400% zoomed in crops. As you can see, the difference between Bicubic and Lanczos is minimal, and there are some strange artifacts in both. Is this an example of ringing?

Original Image 640x480p30.000 square pixels


Resized Image 720x480p29.97 non-square pixels BicubicResize


Resized Image 720x480p29.97 non-square pixels LanczosResize

[jagabo]

Those look like MPEG blocking artifacts, not resizing artifacts.

[newpball]

I wonder, is what we see the original 9901 Kbps 640x480 or some processed 'improved' version?

[hello_hello]

I wonder, is what we see the original 9901 Kbps 640x480 or some processed 'improved' version?

It's labelled "original".

[manono]

I am wondering if there is a way to avoid resizing the image and just mapping the 640x480 square pixel image into a 720x480 non-square pixel image?

For MPEG, yes, as it can be 1:1. DVD, no, as it allows for only 2 DARs - 4:3 and 16:9.