reducing image quality/file size of .SUP (PGS) subtitles

[Interpolator]

Most of the discussion around subtitles revolves around conversion from one format to another, which is typically the most useful task, especially for reducing file size (i.e. SUB2SRT, SUP2SRT, etc).

On that note, I have two questions:

1. Are there any tools which allow a simple quality-reduction of .SUP (PGS) files, keeping the same format?
2. Can someone point me to a detailed format specification (so I can write my own)?

At this writing (July 2015), x265 and AAC codecs permit such impressive compression that inclusion of the original .sup files is an "affordable" cost to file size, but I'd still like to reduce them in quality, while still keeping the same format. I'm going on the theory that it's probably a single (or minimal) color lossless format like .png, and that the resolution is scalable.

OCR to SRT is great, but error prone, and there's limited usefulness in converting to other image formats if .SUP is supported by my preferred muxer (MKV).

[_Al_]

While muxing sup into MKV there can be "zlib" compression used. I do not set in in command line, but it appears to be default anyway, perhaps unless there is specific: --compression 0:none set using command line or using mkvmerge GUI.

But anyway, this can be helpful to investigate what that "zlib" compression is based on. https://en.wikipedia.org/wiki/Zlib

[Interpolator]

Thanks, this is a start.

I suspect what I really need is more info on the Blu-Ray .SUP (PGS) specification itself. I am somewhat surprised this compression strategy hasn't been considered and utilized already, but perhaps when I get ahold of the specs I'll understand.

I'm thinking back to the early 80's 8-bit computers when character sets were simple 8x8 definitions and everything was easily readable (at least in English). It doesn't take high resolution to produce perfectly acceptable subtitles in most languages.

While muxing sup into MKV there can be "zlib" compression used. I do not set in in command line, but it appears to be default anyway, perhaps unless there is specific: --compression 0:none set using command line or using mkvmerge GUI.

But anyway, this can be helpful to investigate what that "zlib" compression is based on. https://en.wikipedia.org/wiki/Zlib

[KarMa]

When dealing with storage capacity like a bluray, this is a none issue. I have not seen any way to compress these files, like you can with a png, so the best alternative would be to just OCR to an .srt text file.

[Interpolator]

When dealing with storage capacity like a bluray, this is a none issue. I have not seen any way to compress these files, link you can with a png, so the best alternative would be to just OCR to an .srt text file.

Thanks for your response, though offline optical media storage capacity and OCR solutions aren't really relevant to the discussion.

BDSup2Sub does allow for resolution changes and rescaling, and allows export to the same format. Unfortunately, there does not seem to be any option for adjusting image compression settings or color depth, just the colors in the "palette" and dithering.

With one or two .SUP files of a total of 40MB or so, that's pretty small potatoes on a 2GB 1080P x265 recode and two AAC audio tracks of, for example, 384kbps 5.1 English and a secondary language of perhaps 96bps stereo. On the other hand, some films have literally ten or more audio tracks, and 15 or more subtitle tracks. This is where it starts to be come material. That's where I'd go with the secondary audio tracks of something like 24kbps or 32kbps stereo, especially when they're probably not going to get used much anyways, if at all.

I did find a less than ideal option with BDSup2Sub: Compression of less-commonly-used language subtitles can be scaled to, say, 70% width and 80% height. Yes, it makes smaller text, but not miniscule.

Many transcoders seem to opt for just omitting most additional audio and subtitle tracks entirely. I don't use 'em, personally, except in the rare cases where I watch a foreign film, but that's also irrelevant to the discussion.

The issue seems to center around what bitmap format the subtitles are stored in, and strategies for reducing the file size of each.

[_Al_]

Yes, 40MB for a track is almost nothing.
You are going into a strange territory, while trying to squeeze original and keeping all content, you start to compromise very aggressively.

In subtitle case, it is players that has to be able to read that track. One would rather aim for already established ways of doing things. Take just MKV container, hardware and software players are tuned up to its design. I am not sure how you imagine things to work. For example I am not even sure if using compression "none" for sup subtitles, using WDTV Live player would even work etc. And you talk about new compression. Remember for example when MKV author just set compression for headers as a default and next thing half of people complained that they could not play their MKV content.

[Interpolator]

Yes, 40MB for a track is almost nothing.
You are going into a strange territory, while trying to squeeze original and keeping all content, you start to compromise very aggressively.

Actually, that re-enforces my point and goal. I'm not interested in doing anything non-standard, or dealing with error-prone OCR, just using standard compression techniques.

At issue is what sort of supported compression can be done on each individual image in each .SUP file. I

I wasn't aware of the MKV standard change as I've avoided using that container until x265, preferring mp4. x265 mp4's don't seek properly in VLC unless encapsulated in MKV. This is more likely a VLC flaw (it's weird, although highly versatile amateur software). Either way, MKV discussions aren't really pertinent as that's a macro-container, I'm just interested in what can be done within the .SUP (PGS) standard itself.

What I really need is a detailed spec of the standard and it's surprisingly elusive.

Yes, 40MB on 2GB is nothing. On the other hand a dozen 40MB files on 2GB starts to get material, a 24% increase. It's not critical, I'm just saying reducing .SUP file size but keeping the format standard is a compression strategy worth utilizing.

[_Al_]

I cannot imagine, how would you do all of this this, not establishing new standard. Not trying to be dismissive for anything new, but, ...
For example just from my point of view, from user, ..., you'd render my WDTV player impotent to show you new sup track. That is if that player was able to play HEVC which is not, but anyway you'd need to establish new standard so all new firmware hardware players and software players (under android, windows, linux, apple ...) could read that new sup as well. Because as I said they might expect zlib compression using that sup in MKV file (if they understand sup in the first place), anything else and there is no subtitle track at all . So even if there is a possibility for some improvement in compressibility that is in the book of MKV standards so to speak, and that is a big IF, to improve that compression, how are you going to persuade all firmware and software writers to respect it and tune it up properly. Header compression was MKV standard and it was a fiasco, it took a long time those code writers started to reflect on it but MKVmerge author changed it back again to end this problem for sure I guess.

[Interpolator]

UPDATE: Problem solved - at least as initially tested with VLC...

Taking a 1080p .SUP file, loading into BDSup2Sub, changing the resolution from 1920x1080 to a lower format (there are several standards to choose from), exporting to a new .SUP file (not a new format), and muxing with MeGUI's MKV muxer (I forget which cmd line tool it uses) produces subtitles that appear in the same location, just at lower resolution, as desired.

What is not known is if other players will display the subtitles in the correct location (scaling the image and placement back up to fit the 1080p film). In other words, it could simply be that VLC is just smart like that.

File size comparisons of a sample sub (the actual image sizes vary from subtitle to subtitle and are much smaller than the full film's dimensions, of course):

• 1920x1080... 38,768 kb (original)
• 1280x720.... 22,431 kb
• 720x576..... 12,626 kb
• 720x480..... 12,009 kb

Personally, I'm going with 720p for the main English subtitle track, and 576p for the other languages. Naturally, others may have different preferences and language priorities.

One other note: I did read there's a text standard for Blu-Ray subtitles but it's almost never used. If the source is image, I'm of the opinion that OCR should be avoided unless its meticulously proofread and fixed of errors. I'd rather have an accurate lower-resolution standard subtitle image file which is 12mb than an inaccurate text file of a few hundred kilobytes. Obviously others may disagree.

[EDIT: The film in this example has 6 .SUP tracks totaling 223mb. After conversion this was reduced to 81mb.]

[KarMa]

"Subtitle Edit" is probably the best program I know of for OCR of Bluray subtitles. The error rate for English subtitles are pretty low when I use it. Don't even have to go through the process of matching each character to a picture, just download the English file and you are good. I usually only come across a few wrong characters per video, usually has to do with a mix up between l and i.

[Interpolator]

"Subtitle Edit" is probably the best program I know of for OCR of Bluray subtitles. The error rate for English subtitles are pretty low when I use it.

Thanks, that's good to know. I've always wondered why image-based solutions over 8 or 16bit text characters are favored in the first place.

[_Al_]

changing the resolution from 1920x1080 to a lower format

Isn't it better just to encode VobSub (idx,sub) , same resolution, 1080?
That would give you approximately 15MB file (using your original 38MB sup). So even less than downscaled sup.

"C:\Program Files (x86)\Java\jre1.8.0_31\bin\java.exe" -Xmx256m -jar "C:\tools\BDSup2Sub\BDSup2Sub.jar" "original.sup" "new.sub" /fps:23.976

Interestingly enough, I tested downscaled sup to 720, muxed in 1080p movie and WDTV Live player upscaled it on screen correctly.
But , I would not go this way, that's for sure.

"C:\Program Files (x86)\Java\jre1.8.0_31\bin\java.exe" -Xmx256m -jar "C:\tools\BDSup2Sub\BDSup2Sub.jar" "original.sup" "new.sup" /fps:23.976 /res:720  /filter:lanczos3

[Interpolator]

[QUOTE=_Al_;2400978]

changing the resolution from 1920x1080 to a lower format
Isn't it better just to encode VobSub (idx,sub) , same resolution, 1080?
That would give you approximately 15MB file (using your original 38MB sup). So even less than downscaled sup.

Interestingly enough, I tested downscaled sup to 720, muxed in 1080p movie and WDTV Live player upscaled it on screen correctly.
But , I would not go this way, that's for sure.

Yes, since my last update I discovered .idx and .sub files are more compact; not sure why yet though I think it's simply a reduced color palette, and the resulting lossless bitmaps compress better (presumably using .PNG). I'll look into reducing color palette some more. I like staying within the single-file .SUP format, but the two-file .IDX/.SUB combo isn't a bad option.

Good to know your media app (or is that a physical player?) also scales the lower-res subtitles; I found VLC does. I suspect most will, but that's speculation.

BDSup2Sub is discontinued software, and there are many .SUP files it won't load, throwing an "index out of bounds" error. SupRip has similar problems on the same files.

Subtitle Workshop won't load .SUP files at all.

[_Al_]

WDTV Live is a hardware player, it is quite common in USA, BestBuy was selling those or still is. There is about 6 models out there.

I use BDSup2Sub with my utility that gets Blu-Ray's into MKV, and it uses BDSup2Sub to work with demuxed subtitles. I basically never had any problem for 2 years or so loading SUP subtitle in it. I use eac3to exclusively to demux streams. It is all automatic. I make sure that I do not select closed caption track, but rather regular SUP (that are not boxed in black). I can see those subs on screen through ffplay so I can make sure what I select if there is more versions for same language. This way I know if particular subtitle is a forced subtitle as well.

[Interpolator]

I'm not sure what it is, but many shared mkv's that are extracted using MeGUI's HD Stream extractor (which I think is basically a front end to some app called "eac3to") extracts the .sup files successfully. The original MKV plays the subtitles just fine, but when I go to import into BDSup2Sub version 1.0.2 it spits out a palette ID or index error.

What's really annoying is that the software's bad design allows a palette ID issue to be a fatal error, instead of just using some default value or best guess and continuing, like it obviously should, if the subtitles are valid enough to play properly in VLC and presumably other players. SupRip has similar problems. Sloppy error handling.

I wonder if there are any other tools that will read the sup files.

[Interpolator]

...VLC does have one highly annoying unusual feature. Subtitles are ON by default, and the way to disable them is cryptic and unintuitive. I still use VLC because of its power, but like a number of free apps, its main designer has shown to be rather indifferent to usability and feedback from users.

http://superuser.com/questions/533585/how-to-disable-subtitles-by-default-but-keep-them-available

[_Al_]

It is a free software, those guys are not working for us, think about it, the same goes for all those free apps out there, as long they are not nasty.

[Interpolator]

I felt I should add that BDSup2Sub is a nice app, but buggy, and I seem to have encountered a problem with changing resolution. It seems although when you start with a 23.976 fps Blu-Ray film and opt to convert to either PAL or NTSC resolution (but leave frame rate unchecked), it converts it anyways to 29.97 fps or 25 fps.

Worse, the reverse fix doesn't seem to work - loading and selecting convert frame rate, and entering the wrong frame rate as the source and the desired (23.976) at the bottom.

So, the only resolution reduction that works is 720p, which is acceptable if one exports to the lower-color-depth IDX/.SUB.

Another option mightbe exporting to .XML/PNG and batch-resizing each image, re-importing, and saving to .SUP or .IDX/.SUB. That's a lot more work than I wish to deal with, but it could be scripted.

Unfortunately, development on BDSup2Sub appears to have been discontinued over a year ago. It's still a useful app, and the only one I know of at present that allows this functionality.

As excruciatingly slow as x265 is to encode, it's kind of nice to be able to keep a few alternate language subtitles I'd normally discard vs. eating heavily into the compression x265 affords.

[videobruger]

and the only one I know of at present that allows this functionality.

You can do the same in Subtitle Edit. When you open a PGS file SE prepare for OCR. Instead you can highligth all /Ctrl+A), Right click, and select to export to PGS, IDX/VOBSUB, XML/PNG with re-size and selection of FPS.

[Interpolator]

Really....that's good news. I tried SubtitleEdit but it looked to be an OCR-only app so I abandoned it since I'm not interested in OCR.

They really should redesign that part of the UI. I'll re-install it.

[KarMa]

@videobruger
Did not know that was supported, cool stuff.

[Interpolator]

I did try one of my Blu-Rays using OCR. Time consuming and cumbersome because of the limited dictionary, but it was fairly accurate. It would be impossible for me to be accurate with non-English languages, except for names. Nice to know the feature is there, though, if the need for subtitles because it would be useful for highly compressed very low bitrate a/v.

[videobruger]

me to be accurate with non-English languages

I'm not sure if you're refering to Subtitle Edit but in the OCR window there are 3 dots next to "Language" that allows you to add Tessseract data for any language and 3 dots next to "Dictionary" where you can download appropriate dictionary. Also note you can add unknown words (Like names of characters, places etc) to "User Dictionary"

[Interpolator]

Yes, I did that... it's still a time consuming pain in the ass, and this was for English subtitles!

I could possibly do some other languages that spell names the same as in English with a normal alphabet, but not ones with symbols for letters and words (Arabic, Japanese, Chinese, etc, but I don't include those ones anyways... mostly I'd like to include French, German, Spanish, and Italian, but I also include a few others I see here and there depending on the film... Portugese, Swedish, Dutch, Polish, Czech, Russian and Ukranian).

I don't share rips p2p so to be blunt, I don't care. It doesn't matter. Besides, it's much easier just to reduce a 30mb sup down to a 4mb sub and be done with it and not screw with OCR.

OCR can be an advantage for low-quality/file size rips, obviously and might be worth the time if it's something to be shared with others and there's actually a concern over file size.

These are 3 or 4GB x265/AAC 1080p Blu Ray feature films, mostly critically acclaimed ones, not B-movies or popular but bad Holywood formula films, with a few exceptions. For those, a 600-900mb 960x540 rip would do, and I'd probably rip the dubbed language audio to 12kbps 16kHz mono or just leave it out entirely. But that's just me.

It's a different topic, but I also don't screw with preserving DTS or FLAC audio either. To me that's just silly - I can't hear the difference for anything over about 224kbps AAC, but I usually encode at 448kbps AAC anyways, preserving the 5.1, although I only care about stereo myself because that's all I have or care to have for audio. Surround sound is way over-rated. Quality sound is what matters.

[KarMa]

Because of this thread I recently tested out just compressing .sup files into a 7-Zip file. With LZMA2 and Ultra Compression, I was able to compress 12 subtitle files down from a total of 305MB to 54.4MB. No loss in quality as it is just a lossless .7z archive. But of course the user would have to extract the track they wanted.

[Interpolator]

Because of this thread I recently tested out just compressing .sup files into a 7-Zip file. With LZMA2 and Ultra Compression, I was able to compress 12 subtitle files down from a total of 305MB to 54.4MB. No loss in quality as it is just a lossless .7z archive. But of course the user would have to extract the track they wanted.

Excellent point to bring up. I found similar compression results with winrar - but my purpose was simply to make a backup of the source after converting to lower-but-acceptable quality .IDX/SUB's.

.IDX/SUB's are non-standard in 1080p and 720p resolution as they were designed for 480p/480i DVD's (or 576 for the 25fps folks). Still, I suggest it's acceptable vs. excluding the subs entirely for those "secondary" languages. I've taken to going with the stock .SUP for the English tracks and 720p .IDX/SUB's for the other languages I wish to include.