Improve comprimibility in SD broadcast stream

[elmuz]

Hi all,
my company wants to broadcast via DTT (digital television).
Problem is the bitrate is very low... here television is only PAL (720,576) encoded in MPEG2 at around 2Mbit/s (this is because we can't pay much more)...
We noticed that full hd source - once they are resized and then encoded - become really damaged... you can't see details... the annoying thing is that other video (music clip) originally sent to us in SD, result not as bad as our shootings...
My idea is that during donwscale image become "full of details" and so less compressible (which is the right word?), maybe we need to apply a strong denoise filter before the encoding phase... I try with fluxsmooth but I can't see much difference.
Any idea?

[jagabo]

Use a lower resolution like 544x576. Downscale with a bilinear filter rather than a sharper filter (to reduce noise). Use a strong spacial/temporal filter -- annoying but necessary if you want low bitrates. Keep in mind that some video compresses better than other. Noise, detail, high contrast, motion, flickering, fog, smoke, etc. all consume more bitrate. Basically, anything that causes more pixels to be different from the pixels around them, spacially or temporally, will consume more bitrate.

[elmuz]

thank you jagabo, could you suggest a strong spacial/temporal filter? I tried severals (Avisynth talking) like FluxSmooth, Deen, Hiq3Denoise etc...
I found all of them quite weak, but maybe it was just a question of values...
I found eDeen() very aggressive, maybe too much... We need to test.
How do you consider this pipeline of filters?

DECODE --> DEINTERLACE (Yadif) --> RESIZE (Bilinear) --> DENOISE (one of them)

Would it make sense change the order between Resize and Denoise? Consider it's from 1080 to 576... I think doing denoise first wouldn't be visible considering so much downscale... isn't it?

Something else... is progressive better than interlaced when encoded at low bitrates?

[poisondeathray]

If you're optimizing for low bitrate and compressibility, avoid yadif . Try QTGMC with low sharpness settings. You can use a faster preset e.g

QTGMC(preset="faster", sharpness=0.3)


Using quantizer encoding, the filesize will be significantly smaller using QTGMC (partially because it denoises a bit, partially because yadif leaves residual artifacts, which consume more bitrate)

Would it make sense change the order between Resize and Denoise? Consider it's from 1080 to 576... I think doing denoise first wouldn't be visible considering so much downscale... isn't it?

You cannot resize vertically before deinterlacing

[elmuz]

Would it make sense change the order between Resize and Denoise? Consider it's from 1080 to 576... I think doing denoise first wouldn't be visible considering so much downscale... isn't it?

You cannot resize vertically before deinterlacing

I meant after deinterlacing... just change the order between denoise and resize...

[poisondeathray]

It depends on the type of noise, and the type of denoiser

Resizing first before denoising might be much faster to process, but some types of noise maybe resized, and the denoiser won't work as effectively

[jagabo]

When converting 1080i to progressive SD just throwing out one field looks reasonably good and is very fast:

WhateverSource()
SeparateFields()
SelectEven()
Filter...()

[elmuz]

This time I went with this...
v=MPEG2Source(".....d2v", cpu=0)
v=v.QTGMC(preset="faster", sharpness=0.3)
v=v.SelectEven()
v=v.BilinearResize(720,576)
v=v.deen() # or v.edeen() which is much stronger

and it goes between 1-2 frames per second...
Do you think QTGMC will give benefits? It takes a lot of cpu...

[edDV]

Hi all,
my company wants to broadcast via DTT (digital television).
Problem is the bitrate is very low... here television is only PAL (720,576) encoded in MPEG2 at around 2Mbit/s (this is because we can't pay much more)...

Can you be more specific on the type of DTV transmission? Most have defined codecs and bit rates for SD and HD source. You should be encoding to that target or your signal will suffer serious recode loss. Usually broadcasters request programming source at higher standard allowing them to apply the final transmission encode. For example they may apply statistical multiplex (VBR shared by several channels).

[elmuz]

Hi all,
my company wants to broadcast via DTT (digital television).
Problem is the bitrate is very low... here television is only PAL (720,576) encoded in MPEG2 at around 2Mbit/s (this is because we can't pay much more)...

Can you be more specific on the type of DTV transmission? Most have defined codecs and bit rates for SD and HD source. You should be encoding to that target or your signal will suffer serious recode loss. Usually broadcasters request programming source at higher standard allowing them to apply the final transmission encode. For example they may apply statistical multiplex (VBR shared by several channels).

We are one step earlier...

Basically my company produces their own videos (shooting, editing and so on... everything you do with Premiere/After Effect). Next we send them to a company which ingest them in big storage/database and let us control playlists and everything we want air (commercials, bumper, id, video titles and so on). This second company provides a complete video stream (they downscale videos, overlay logo, popup and so on).
A third society take care of digital broacast... and everything waste in this step because of low bitrates... (only in some regions, where we have to share mux with others channels)..

RADIO COMPANY
LAUNCHING TV
[Video production]
(I am here)
|
|
"PLAYOUT" COMPANY
[they get our contents
and they provide the
the complete stream
|
(until here quality is good)
|
BROADCAST COMPANY
[they receive high quality
signal and send to all
the country]

My idea is to avoid give the second company full hd videos... maybe if we send SD file very denoised maybe something gonna improve... don't you think?

[edDV]

Hi all,
my company wants to broadcast via DTT (digital television).
Problem is the bitrate is very low... here television is only PAL (720,576) encoded in MPEG2 at around 2Mbit/s (this is because we can't pay much more)...

Can you be more specific on the type of DTV transmission? Most have defined codecs and bit rates for SD and HD source. You should be encoding to that target or your signal will suffer serious recode loss. Usually broadcasters request programming source at higher standard allowing them to apply the final transmission encode. For example they may apply statistical multiplex (VBR shared by several channels).

We are one step earlier...

Basically my company produces their own videos (shooting, editing and so on... everything you do with Premiere/After Effect). Next we send them to a company which ingest them in big storage/database and let us control playlists and everything we want air (commercials, bumper, id, video titles and so on). This second company provides a complete video stream (they downscale videos, overlay logo, popup and so on).
A third society take care of digital broacast... and everything waste in this step because of low bitrates... (only in some regions, where we have to share mux with others channels)..

RADIO COMPANY
LAUNCHING TV
[Video production]
(I am here)
|
|
"PLAYOUT" COMPANY
[they get our contents
and they provide the
the complete stream
|
(until here quality is good)
|
BROADCAST COMPANY
[they receive high quality
signal and send to all
the country]

My idea is to avoid give the second company full hd videos... maybe if we send SD file very denoised maybe something gonna improve... don't you think?

Will these programs be broadcast as 16:9 or 4:3?

Does the "playout company" specify a format? In the old days this would have been a tape format (e.g. HDCAM, DigiBeta or Betacam). Today programs tend to be distributed as high bit rate MPeg2 or DVCPro (35-100 Mb/s for HD or ~ 20-50 Mb/s MXF container for SD).

Playout server company EVS has some good case studies that you may find useful. The RTBF (Radio-
Télévision Belge Francophone) case study seems close to what you describe.
http://www.evs.tv/America/English/Products/Doc-Center/Case-Studies/page.aspx/3058

[elmuz]

[QUOTE=edDV;2067895]

Will these programs be broadcast as 16:9 or 4:3?

Yes PAL 16:9, sharing a mux with other channel... We know it should be a "final-stage company" problem... I mean: they should denoise as a last step before broadcast everything... we were just wondering if we could do something to help them preventing that horrible loss...

Thanks even for your link

[edDV]

[QUOTE=elmuz;2067902]

Will these programs be broadcast as 16:9 or 4:3?

Yes PAL 16:9, sharing a mux with other channel... We know it should be a "final-stage company" problem... I mean: they should denoise as a last step before broadcast everything... we were just wondering if we could do something to help them preventing that horrible loss...

Thanks even for your link

I think if this is to be broadcast SD 16:9, that is what you should feed them even if the master is HD. That way you have control of the downscale and denoise if necessary. Premiere Pro isn't the best for downscale. Avoid deinterlace. Search the site for AVIsynth scripts for 1080i to 576i downscale.

[jagabo]

1080i to 576i:

WhateverSource() #assuming 16:9 DAR source
QTGMC() #or Yadif(mode=1)
WhateverResize(720,576)
WhateverNoiseFiltering()
SeparateFields()
SelectEvery(4,0,3)
Weave() #576i output

[edDV]

The EVS site has some good white papers on tapeless workflows
http://www.evs.tv/America/English/Products/Doc-Center/Products/page.aspx/3060

This is the introductory white paper.
http://www.evs.tv/01/MyDocuments/WP_STUDIO_PRODUCTION_effective_transition_to_tapeless.pdf

Common SD transition formats.

SD codecs M-JPEG – User selectable bitrate 8-100Mbps
IMX D-10 (SMPTE 356M) 30/40/50Mbps
DVCPRO 50

Modern SD Formats

XDCAM MPeg2 or DVCPro in MXF wrapper

[Gavino]

1080i to 576i:

WhateverSource() #assuming 16:9 DAR source
QTGMC() #or Yadif(mode=1)
WhateverResize(720,576)
WhateverNoiseFiltering()
SeparateFields()
SelectEvery(4,0,3)
Weave() #576i output

I suspect using QTGMC is a waste of (CPU) time here, and a much simpler bobber could be used without any appreciable loss. Any benefits gained from QTGMC are probably lost on downsizing and re-interlacing.

[elmuz]

I suspect using QTGMC is a waste of (CPU) time here, and a much simpler bobber could be used without any appreciable loss. Any benefits gained from QTGMC are probably lost on downsizing and re-interlacing.

I suspected the same...

[jagabo]

1080i to 576i:

WhateverSource() #assuming 16:9 DAR source
QTGMC() #or Yadif(mode=1)
WhateverResize(720,576)
WhateverNoiseFiltering()
SeparateFields()
SelectEvery(4,0,3)
Weave() #576i output

I suspect using QTGMC is a waste of (CPU) time here, and a much simpler bobber could be used without any appreciable loss. Any benefits gained from QTGMC are probably lost on downsizing and re-interlacing.

Buzzing edges from simpler bobbers still buzz after downsizing. If the video doesn't have sharp horizontal edges it's less of an issue. It's up to you whether the time investment is worth it. And I did indicate Yadif was an option.

[edDV]

Will these programs be broadcast as 16:9 or 4:3?

Yes PAL 16:9, sharing a mux with other channel... We know it should be a "final-stage company" problem... I mean: they should denoise as a last step before broadcast everything... we were just wondering if we could do something to help them preventing that horrible loss...

I suspect the losses you are seeing are from repeated re-encoding. The goal should be to avoid re-encoding until the final re-mux for broadcast transmission.

Most broadcast facilities have an in-house standard and prefer to convert everything to that standard at ingest. In modern facilities, for SD this may be IMX/XDCAM in an MFX wrapper or DVCPro 50 in an MXF wrapper. You should contact the facility engineer and find out how to encode directly to their in-house standard including bit rate. This would hopefully avoid a recode at ingest to the play server.

[Gavino]

Buzzing edges from simpler bobbers still buzz after downsizing.

Won't they (usually, if not always) disappear on the subsequent re-interlacing (576p->576i)?

[jagabo]

Buzzing edges from simpler bobbers still buzz after downsizing.

Won't they (usually, if not always) disappear on the subsequent re-interlacing (576p->576i)?

They're reduced. They don't disappear. Moire patterns (a related issue) are also a problem.

[Gavino]

Can you point me to some examples?
I am not doubting what you say, but I would like to see what kind of difference it makes in practice.
It may also depend on whether you view the final output on an interlaced device (CRT) or are relying on a deinterlacer to view it on a LCD display.

Certainly I think if you removed the noise filtering from your script, there would be little point in using QTGMC, and the author (-Vit-) seems to agree with this.
http://forum.doom9.org/showthread.php?p=1488711#post1488711
(See also Didée's post #558 just below that.)

[jagabo]

Can you point me to some examples?

I don't have any real video handy right now.

[[_chef_]]

2mbit/s. Madness. Unwatchable with such a res.