Help needed converting HD h.264 Freeview recordings (New Zealand)

[Chopmeister]

I have a Visione VE4000 set top box, that decodes Terrestrial High Definition Freeview broadcasts in New Zealand, and also records/plays these broadcasts using an external USB hard drive. I am after help converting the recordings it makes to more normal (Divx/MKV) files playable on my other equipment, and for storage etc. Being able to edit them to remove leader, trailer, and commercial segments would be a bonus.

I understand that the transmission format is MP4 h.264. The Visione box seems to save each show recorded into a separate sub-directory off the disc root directory, named using a numerical code that is clearly a date/time stamp of when the recording was created. Within this directory there is an .ifo file that is 2KB in size, and when opened with a text editor contains some junk and a description of the show from the epg, as well as a further sub-directory with the same name. The further sub-directory contains a small (1MB) file with an .idx extention, as well as a series of .ts files in numbered sequentially that are clearly the recording itself broken up into 1.46GB chunks. The .idx file is binary not text.

GSpot tells me that the .ts files are MPEG-2 transport streams with video type mp2t (which is odd as the broadcast isnt MPEG2). The .ts files won't play with VLC, or in my WD HD TV media player which is supposed to be able to handle .ts files. Also:

• Super (version 2010 build 37) won't convert them (and gives an error message as soon as I hit go)
• MediaCoder (version 0.7.3) won't convert them (it gives error code 17, the explanation of which is in Japanese on the website http://wiki.mediacoderhq.com/index.php/Error_Codes#017)
• Handbrake (version 0.9.4) won't convert them (it also crashes)

The vendors aren't particularly helpful - their response is that they haven't had much experience of this, and suggest that I try Super (which, as per above, doesn't work).

Do any of the gurus out there have any idea how these chunks can be combined and converted to a more normal file format? This is driving me to distraction.

Help and suggestions greatly appreciated.

[guns1inger]

I would try mediainfo instead of gspot and see what it says, then look at the beta of the new videoredo to see if it can handle the files.

[Chopmeister]

Opening the first .ts chunk using Mediainfo shows that:

- The container is MPEG-TS (No PAT/PMT)with 1 video stream (AVC) and 1 audio stream (MPEG Audio).

- The video stream is 7,148 Kbps 1280*720 (16:9), at 50.000 fps, AVC (Component) (Main@L4.0) (CABAC /4 Ref Frames).

- The audio stream is192 Kbps, 22.05 KHz, 2 channels, MPEG Audio (Version 2)(Layer 1) (Dual mono/ Intensity Stereo + MS Stereo).

Videoredo opens the appears to video file, but only for each chnuk (about 26 minutes long) rather than the whole recording. However, while it will open the video, it plays no audio. When I try to open multiple chunks, the join option doesn't seem to join them and it dives a message that the audio format isn't supported. When I follow its recommendatio to do a "quick stream fix" it thrashes the machine, plays a few seconds of sound, then crashes.

[Chopmeister]

This is the full output in case it's useful

[guns1inger]

You might be able to join the files in TSMuxer. I am surprised at the choice of audio encoding. I would have expected AAC at that bitrate. You might be able to take the joined chunks, open the resulting file in Xvid4PSP 5.0.x (not version 6) and withthe video set to copy mode, re-encode the audio to AAC and output to either an M2TS or MP4 container, then try editing that in VideoRedo. PITA, but it might work.

[Chopmeister]

I have made limited progress on resolving this.

Firstly, after a lot of trial and error, I joined the chunks of the file in the logical order and renamed the file from .ts to .trp (which seems to matter for some reason), using this method:

copy /b chunk.ts + chunk.001.ts + chunk.002.ts videofile.trp

Then I loaded the file into H.264 TS_Cutter (v111) using the Tools "Convert TRP/REC -->TS" option and trimmed the start and end of the recording to length.

This now gives me a single .ts file, consisting of all of the recording. Mediainfo tels me this is a1280x720 AVC .ts file with a video bitrate of 6994kbps. Furthermore, the video now works - VLC will play it fine (and it looks great).

However, it has no sound. Mediainfo reports the auido as:
ID: 320 (0x140)
Menu ID: 129 (0x81)
Format MPEG Audio
Format Version: Version 2.5
Format profile: Layer 1
Bit rate mode: Constant
Bit rate: 64.0 Kbps
Channel(s): 2 Channel
Sampling rate: 11.025 Khz

Nothing I have tried will convert this file to xvid.MP4, or play it. I think the video aspect is no longer a problem (since it plays in VLC etc), and it is now down to the sound that is tripping me up.

Suggestions welcome?

[guns1inger]

If VLC can play it, VLC can probably transcode it to Xvid in an AVI container. From there you can repackage it if necessary.

Start VLC, then click on Media -> Convert/Save, and go from there

[Chopmeister]

VLC won't quite play it. As noted above, it will play the video but not the audio part of the file.

Your suggestion works for transcoding, but while the transcoded file has video it has no audio. Same with Handbrake, it converts the file but outputs a result that has video but no audio in it.

It is the audio, not the video, part of the problem that I am asking for help with.

[guns1inger]

Just appending chunks of file may not produce a playable output because not all the correct header information may be in the correct place. Mpeg-1 Layer 2 audio is pretty simple, standard stuff, so if VLC won't plat it there is something else going on.

[netmask56]

Is it possible to post a small except? Across the ditch we use MPEG2 + MPA or AC3 for digital TV, but with the advent of the recent 3D trials this transmission went out as
General
ID : 3330
Complete name : E:\BeyonwizTransfers\3D Trial.ts
Format : MPEG-TS
File size : 254 MiB
Duration : 2mn 27s
Overall bit rate : 14.5 Mbps

Video
ID : 101 (0x65)
Menu ID : 13107 (0x3333)
Format : AVC
Format/Info : Advanced Video Codec
Format version : Version 2
Format profile : High@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Format settings, GOP : M=1, N=24
Duration : 2mn 27s
Bit rate : 13.4 Mbps
Maximum bit rate : 14.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.258
Stream size : 235 MiB (92%)
Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Audio
ID : 102 (0x66)
Menu ID : 13107 (0x3333)
Format : AC-3
Format/Info : Audio Coding 3
Format profile : Layer 2
Mode extension : CM (complete main)
Duration : 2mn 27s
Bit rate mode : Constant
Bit rate : 384 Kbps
Maximum bit rate : 432 Kbps
Channel(s) : 5 channels
Channel positions : Front: L C R, Side: L R
Sampling rate : 48.0 KHz
Video delay : -511ms
Stream size : 6.73 MiB (3%)
Language : English

Menu
ID : 258 (0x102)
Menu ID : 13107 (0x3333)
List : 101 (0x65) (AVC) / 102 (0x66) (AC-3, English)
Language : / English
Maximum bit rate : 14475200


The Beyonwiz native format (.tvwiz) breaks the file up into 32MB chunks and we have bespoke software to join them up and convert to TS.

I'd be happy to have an experiment with your file as I believe eventually (don't hold your breath) Oz will change the standard to something similar to NZ or near enough to give more bandwidth for broadcasters and the possibility of 3D transmissions. But then there are the pollies....

[guns1inger]

It's not the pollies, it's the TV stations. The pollies just bend over when the pressure is applied. Our standards are what they because the TV stations went for the cheapest (and crappiest) option available at the time.

[netmask56]

Not in Australia - very different politics to USA or Europe, the standards here were determined by a bunch of techno nerds far removed from reality. Like our FM service - ever so different standard from the International etc. Analog TV had a channel 0 and a 5a! and as for cooperating between Oz and NZ for agreed common standards that would be of mutual advantage, a market of 22 million + 4 million - nah we both have to re-invent the wheel everytime. We even have 3 different railway guages... I wish the corporations would lean on the politicians to make sensible decisions.

[guns1inger]

The pollies agreed to it because of pressure from the commercial stations. Same reason they knobbled others from coming into the market, and the same reason it took years before secondary channels came into play. At that stage it was Kerry Packer and Kerry Stokes primarily driving it. They have a lot less influence now, but then we also have a much more disinterested group of pollies. No-one cares that our TV broadcasts are such low quality. Less than half the freeview badged set-top boxes currently on sale have the ability to receive and decode H.264 broadcasts, but anything sold prior to the freeview initiative, or without a freeview badge, cannot - which makes up the bulk of receivers on sale or already sold. The samples of 720p broadcast material from the US that I have seen are orders of magnitude better than the 720p material broadcast here, and substantially better than the 1080i broadcasts.

[poisondeathray]

chopmeister - something is wrong with your joined sampling rate. It began as 22.05KHz, then became 11.025KHz after you joined

can eac3to identify the audio (either in the original or joined version) ?

can you demux the audio track (maybe in tsmuxer or eac3to or dgavcindex) and play in winamp, or foobar2000, or other software ?

[Chopmeister]

If people will excuse the thread gravedigging, I have been having another play with this.

I still can't seem to convert the files, or play them with VLC. However, in a fit of curiosity I installed the K-Lite codec pack and found that the .ts streams recorded by the STB will play fine in Media Player Classic Home Cinema ver 1.5.3.3611 (MPC). The playback quality in MPC rock solid and much better quality than the actual STB will play them back. However, MPC doesn't seem to have a conversion option that I can find, and none of the converters I have tried (incl VLC, MeGUI, Mediacoder, Handbrake) will work for conversion.

MPC reports the info on the .TS chunks as:

Video: MPEG4 Video (H264) 1920x1080 25.00fps [Video]
Audio: AAC(LATM) 48000Hz stereo [Audio]

Still in the market for any thoughts on what will touch this. I want to get it to a more orthodox format & container, so that I can join the chunks, cut out the commercials, and view on a stand-alone media player (xvid is my plan). Any wisdom greatly appreciated.

[stars]

Hi. I think it will be hard to get the video into xvid format because of the size of the
video picture 1920x1080.

The best thing is to demux the .ts chuncks seperatly.

If you want to cut out the commercials you probely have to change the format of
the video and audio to get correct cuts. I use Mpeg2snitt for .ts streams with mpeg2 video
and mp2 audio but it dosent work with h264 and AAC.

If you have a .mkv file you can do this...
I use vitualdub with a plugin direct show inputdriver http://forums.virtualdub.org/index.php?act=ST&f=7&t=15093&

Haali Media Splitter and ffdshow. In this way I can easely edit the
video file but the you must rencode it after editing. If you want you could always framserv the
video to a encoder of your liking. I sometimes frameserv to MeGUI. If the audio plays fine
in Virtualdub you can choose under the audio menu full processing and save the audio as
a .wav file. If you have edited the video the audio cuts will be there to. So when you
have a .wav file you can do almost anything.

If you dont cut the commercials you can remux your audio and video into a .mkv file using
mkvtoolnix. When you have remuxed all your chuncks join them toghter into 1 file
using the same program. If you do it this way you wont have problems with
sync and headers.

Hope it helps

regards stars...

[Chopmeister]

No, video is not the problem. I have been playing around a bit more and have made some progress. Firstly I have discovered the properties can be found in Media Player classic, and are as listed below. Possibly more usefully, I have discovered that both video and audio will play in Monogram Graphstudio, and it gives me a graph "which puzzles me a bit" of the decoders needed - and seems to show that a combination of LAV audio decoder, ffdshow audio filter and an AC3 filter is what I need - but I am at a loss as to how to use this to create a conversion process.

Both the data from MPC and a screenshot of the graph are as per below.

I'm really hoping that some of you with more technical skill than me can advise how to use this information to convert the video into a more mainstream & editable format. I have pretty much exhausted my meager knowledge, and this is really starting toi irritate me.


Media Player Classic "Properties" Info
General
Complete name : I:\serve\1201010310413341.ts
Format : MPEG-TS
Format profile : No PAT/PMT
File size : 59.5 MiB
Duration : 1mn 10s
Overall bit rate : 7 084 Kbps

Video
ID : 270 (0x10E)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Duration : 1mn 10s
Bit rate : 6 734 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.130
Stream size : 56.5 MiB (95%)
Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

[midnightsun]

Hi, I have a couple of set-top DVR's that record transport streams in what looks to be a similar way to yours. I use dgavcindexnv to join the split chunks. Dgavcindexnv is not free but comes at a very affordable price. In any case the files can be joined with a simple copy command such as "copy a.001++a.002++a.00n a.TS", since the files are just split in a binary fashion. Once you have the larger joined transport stream, this can be manipulated directly by several different tools, BUT you still may be slightly out of luck IF the recorded transport stream doesn't contain PAT/PMT tables, which may well be your case. Videoredo tv suite h.264 should still be able to open the file, but in case it complains that the format is not supported, you're going to have to feed it a more standard transport stream. A tiny application like tsremux (look for it) will allow you to open the PAT/PMTless TS and convert it to a more orthodox TS (all it does is fix the PAT/PMT tables, it doesn't convert the video and audio streams inside the file in any way, it just repackages them in a more friendly way).
At this point this is a 100% usable transport stream and you can make what you wish out of it!

[midnightsun]

I forgot... there's also another way to join the split files without resorting to dgavcindexnv or command line, and that is through TSSplitter (llok for it), which, despite its name, also has a "join mode"

[poisondeathray]

If your graphstudio graph looks like that now, any program that uses directshowsource() in avisynth should be able to convert it now

If you right click the first box and look at the properties, check if it's haali or lavf splitter - your recently installed codec pack probably installed the correct splitter that you were missing

So something like xvid4psp, megui should be able to convert the file . or You can make up your own 1 line script and use vdub or any program that accepts avs

[Chopmeister]

Nope, xvid4psp and megui still won't work - I get an output file with video and no audio.

It plays fine with video and audio in graphstudio, and I can save the .grf file. However, when I try to use this in MeGUI I get the following error:

DirectShowSource: GRF file does not have a compatible open video pin.
Graph must have 1 output pin that will bid RGB24, RGB32, ARGB, YUY2 or YV12

Which I can't seem to get past.

[jagabo]

Remove the last filter (renderers) in both streams. But if I recall correctly, DirectShowSource() on a GRF file will only get video. You may have to use two GRF files, one for audio, the other for video. Send the unused stream to the Null Renderer.

[poisondeathray]

jagabo is correct; if you are manually constructing a graph, you need to remove the last pin (renderer) and use a separate grf for audio & video

use audiodub in the script

e.g

vid = DirectShowSource("video.grf", audio=false)
aud = DirectShowSource("audio.grf", video=false)
AudioDub(vid,aud)



Did you just try DirectShowSource("video.ts") on the video (without a .grf)?

[Chopmeister]

Still stumping me, with error messages that I don't know how to fix. I'm afraid I have strayed two far out of my zone of competence.

It has just occurred to me to do what I should have done some time ago, and upload the sample clip to see if some kind soul with more knowledge that I can would be willing to solve its conversion. The file is a recording of about 1 minute, recorded off free to air TV, and is 59MB in size. The video is straight forward. It is getting the LATM audio converted and into a more user friendly format that still seems to be the stumbling block.

My first go at using Mediafire, but I have put the sample clip here.

http://www.mediafire.com/download.php?842dm6twechy74d

Which hopefully will work

[poisondeathray]

Your sample works fine with graphstudio, and you can convert both video & audio with the method outlined above . (feed the .avs into any program that accepts avs scripts e.g vdub, megui, xvid4psp, etc. etc...)

You need lav splitter in the chain to split the transport stream, and lav audio decoder to decode latm aac audio . But your screenshot shows you already have that . If you right click the 1st box and select properties, I am willing to bet it is Lav splitter

Delete the last renderer box (DirectSound Device), and everything after lav audio decoder as well as the entire video chain for the audio.grf . It should just show the 1st box and lav audio decoder with empty output pin

Delete the entire audio chain and the last renderer box for the video.grf chain . It should just show the 1st box and ffdshow video decoder with empty output pin

Post your error message if any

[Chopmeister]

Error:
AviSynth Encoder Error: Abnormal encoder termination (exit code = 1)

Unsupported number of channels: 6

[Chopmeister]

Update: Never Mind.

The error only appears when I try to use MeGUI. It doesn't appear in Xvid4PSP, and the file converts.

Many thanks to those who put in the effort to help me with this - it has been irritating me for ages!