Identifying commercials

[erik1958]

Is there any interrest in a utility to identify commercials in a mpeg recording?

[ebenton]

I think there would be some interest in a tool that would detect and remove commercials, at the discretion of the user.

It seems to me that there were VCRs a few years ago that claimed to have firmware that was/is able to pause recording during commercials by detecting "fade to black" both before and after commercial breaks. I think that it used to be (and may still be today) that the networks would fade to black at the beginning of a commercial break and then again before resuming programming. This "black" is/was supposedly "blacker" than the fade to black used between scenes within movies or TV programs, and therefore relatively easy to detect.
I don't know if this technology ever really came to pass, or if it is very reliable, but I do remember reading about it years ago.

[erik1958]

Comskip is a free commercial detector
It is a windows console application that reads a mpeg file and analyses the content based on a large amount of configurable parameters.
After analysis it generates a file in various possible formats containing the location of the commercials inside the mpeg file.
The formats include input files for interactive video editors (VideoRedo, Cuttermaran,...) command line video cutters (mpgtx, cuttermaran, mencoder) and video players (zoomplayer, mplayer.
Included is a crude cutpoint editor to review the results of the analysis and modify where needed.
Out of the box Comskip may work for you but due to the large variation of broadcasting styles tuning may be required.
A manual describing a tuning approach and the tunable parameters is included.

The Comskip support forum, including links to the downloads, can be found at: http://mk.kaashoek.com/comskip

[wwjd]

erik1958 wrote:

Is there any interrest in a utility to identify commercials in a mpeg recording?

The short answer is YES!

I am interested in cutting commercials and I normally use Videoredo for this. I have noticed more and more shows with commercials that are nearly impossible to accurately detect. It is going to take something a lot more sophisticated than Comskip or Add Detective to accurately find commercials. Dark frames and logos will not cut it.

[Baldrick]

I have seen it but i want a one click solution.....yes....i'm lazy and i hate console applications.


Added comskip to our tools list.

[erik1958]

If you did try Comskip and it failed to work for you I would like to learn from you, PM me or use the approach outline here

Comskip uses:
Dark frames
Logos
Sound
Aspect ratio
Scene schange
Extensive heuristics

and it is being succesfully used for automated deletion of commercials in many places such as USA, Canada, Europe, Australia, New Zealand, Singapore.

Its a console application because it is intended to be integrated into a PVR without any user interface

[wwjd]

It has been a while since I tried Comskip so I will give it another try. I am with Baldrick in that I want a single click solution that is very reliable. Reliable enough to have it find and automatically remove the commercials.

What I am seeing more and more is a blending of the show and the commercials. They mix in upcoming previews of the same show, they have funky lead outs and lead ins, they mix many Dark frames within a string of commercials, and so on....

I have recently run into several shows that were quite challenging to find the actual start and finish of the commercial segment and I was doing this manually.

[wwjd]

BTW can Comskip be easily integrated into BeyondTV Version 4? If so, how?

[erik1958]

Comskip can be integrated in any recording environment that supports the starting (either during or after the recording is finished) of an executable with as a parameter the fully qualified name of the recording.
After finishing comskip will generate a text file with the same name and path as the mpeg recording it analysed but of course with a dedicated file extension.
This allows for processing during recording, directly after or batchwise at a later moment. The processing can be re-run at any given time.

[erik1958]

With respect to the reliability question:
It depends on your broadcast conditions.
At home we used it in our networked PVR to always remove the commercials without manual intervention (we use mpgtx controlled by something called comclean) and we can not remember when it had a false positive.
Of course sometimes it will fail and you will see the odd 10 seconds of commercial but its much better then manually skipping through the commercials.
When you want to convert to DVD and you need the perfect deletion than Comskip is giving you a VERY good start.
Comskip has seen substantial improvements during the last year.

[reboot]

I use comskip (version 0.79 from Erik) with VideoReDo to automatically detect and cut commercials, on every recording I make.
With a little patience, you can tune it to detect virtually perfectly. You can even have different profiles based on channel (depending on PVR software).
If your PVR software has the ability to run parallel or post recording addons, it's as automatic as you can get. There's virtually nothing to do. Record something, comskip runs and detects, then videoredo (or comclean/mencoder or comclean2/cuttermaran) runs, and outputs a (usually) commercial free mpg.
For burning to DVD, I always check the files first, and rarely do I need to do further editing, depending on the channel. Because of the way comskip does it's detections, it simply works better on some channels than others, but again, this is tuneable.

[wwjd]

reboot,

Could give details on how you get comskip to work with videoredo? I would like to set this up and make it automatic. I am currently evaluating BeyondTV and will soon evaluate GBPVR, how do you get comskip to work with these PVRs?

[hhhhbk]

it would be great if there was a program that would remove commercials


are there any programs for AVIs which can have multiple start and endpoints

instead of the current way i have to make a seperate file for each segment between advert breaks

then append avi segment to create 1 file

[erik1958]

You can set any or all of these output options in the comskip.ini file.
output_videoredo=0
output_womble=0
output_cuttermaran=0
output_mpgtx=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=0

For videoredo a videoredo project file is generated that can be used in a manual or batch conversion run.

[erik1958]

How I do it:

I use a batch file that contains two lines

comskip %1
call comclean %1

Comskip identifies the commercials
Comclean (can be downloaded from http://mk.kaashoek.com/gbpvr ) deletes the commercials
It ONLY works for mpeg files with the .mpg extension.
And it cuts on GOP boundaries without retiming the PTS, some players don't like that.
Instead of comclean you could use comclean2 that uses cuttermaran to do the actual clean and repairs the timeline by remuxing.
Or use a small retime utility that can be found on the same website to do the retiming but that utility has problems with DTV recordings causing sometimes audio and video to run out of sync.

[wwjd]

erik1958,

Thanks for the information. I will mess with it this evening.

[reboot]

Here's a copy of my parallelprocessing.bat, used with GBPVR. I have no idea how to use this with Sage, Beyond or any other, unless you want to use DirMon ( see www.dragonglobal.org )
I do everything during the recording, instead of after, because it's faster.
Note the line that calls videoredo.
Also note that I use an addon to rename the recording to "Showname - Episodename.mpg
Rem that line if you don't.

@echo off
Rem ParallelProcessing.bat
Rem This script will be run automatically by GBPVR when found after a recording is finished
Rem To enable it put this file inside your GBPVR default program directory (C:\Program Files\devnz\gbpvr\)
Rem together with Comskip.exe, Comskip.ini and Comskip.dictionary
Rem The script is called with two parameters
Rem %1 is the fully qualified name of the recorded mpeg file.
Rem %2 is the channel number of the recording
echo Prostprocessing.bat invoked on %1 recorded from channel %2 >>postprocessing.log

Rem The part below can be used to not run Comskip on certain channels, please modify for your conveniance.
Rem The channel numbers that should be skipped are the numbers between the double quotes ("").
Rem You can add "if" lines as much as you want
Rem To disable this behaviour use channel numbers outside the normal range, put a "rem" before the "if" or delete the "if" lines

Rem list of channels to not run comskip on
Rem if "%2" == "27" goto eof:
Rem if "%2" == "36" goto eof:

Rem this trick also makes it possible to have dedicated comskip.ini file for certain channels.
if "%2" == "7" goto channel7:
REM if "%2" == "4" goto channel7:
REM if "%2" == "6" goto channel7:
if "%2" == "15" goto channel15:

Rem use the default comskip.ini for all other channels
goto generic:

:channel7
comskip --ini=comskipcbs7.ini %1
goto continue:
:channel15
comskip --ini=comskipshcase15.ini %1
goto continue:

:generic
Rem The actual run of Comskip. Only one parameter, the name of the mpeg file to process.
Rem Comskip will read its settings from the Comskip.ini found in the same directory as Comskip.exe
Rem Both are supposed to be in the GBPVR default program directory. (C:\Program Files\devnz\gbpvr\)
comskip %1
:continue

Rem Once the commercials have been identified you can run comclean to delete the commercials from the recording.
Rem Do this only when you are not using the Skip function in the GBPVR viewer.
REM the following calls videoredo to edit the file, then renamerecordings.
cscript //nologo "C:\Program Files\VideoReDoPlus\vp.vbs" "%~dpn1.VPrj" "%~dpn1_Cleaned.mpg" /t1 /q
Rem RENAME the old .mpg file and rename the new one to the old name
move "%~dpn1.mpg" "%~dpn1_original.mpg"
move "%~dpn1_cleaned.mpg" "%~dpn1.mpg"

RenameRecordings -t %1 > renamerecordings.log

If you have videoredo, you also need this .vbs script in the videoredo folder.
vb.zip