Using ffmpeg and image-based subtitle tracks

[the_steve_randolph]

I have an MKV that has Russian and French audio tracks and it also has English and German image-based (dvd_subtitle) tracks. I want to hardcode the English subtitle track. I know how to do this with text-based tracks, but, I am getting various errors when I try to hardcode the image-based track. I have been able to export the English track to a SUP with Handbrake. But, I get errors either using the track directly from the MKV or importing the SUP.

Here are some examples that I found online and have tried :

ffmpeg -i in.mkv -map 0 -filter_complex "[0:v][0:s]overlay[v]" -map "[v]" out.mkv

ffmpeg -i in.mkv -map 0 -vf "subtitles=in.mkv:stream_index=0" out.mkv

These commands give me the error message (the subtitle track I want is stream #0:3): "Error initializing output stream 0:3 -- subtitle encoding currently only possible from text to text or bitmap to bitmap"

I am trying to do this in ffmpeg. I realize that some other apps may be able to do this, but, I believe that most/all are based on ffmpeg - so I should be able to do this with an ffmpeg command. Ultimately I want to keep only one audio track and eliminate the two embedded/soft subtitles - which I already know how to do.

Also, I would like to convert the MKV to an MP4, but, I don't think that the command(s) I would use would be any different.

Any ideas?

[ProWo]

Read this:
https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo

[the_steve_randolph]

Read this:
https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo

That is one of the sources I have used and I am not able to get it to work.

[ProWo]

Try

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]"  out.mkv

This should burn in the first subtitle in in.mkv into out.mkv (if image based), without audio (can be muxed afterwards)

[the_steve_randolph]

Try

ffmpeg -i in.mkv -map 0:v -map 0:0 -filter_complex "[0:v][0:0]overlay[v]" out.mkv

This should burn in the first subtitle in in.mkv into out.mkv (if image based), out.mkv without audio (can be muxed afterwards)

Why are you excluding the audio?

[the_steve_randolph]

Try

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]"  out.mkv

That command generates the error: "Filter overlay has an unconnected output"

When I do this command:

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" out.mkv

It generates the error (BTW, stream #0:2 is the second audio track): "Error initializing output stream 0:2 -- Subtitle encoding currently only possible from text to text or bitmap to bitmap"

[pm-s]

Use handbrake

[ProWo]

Could you post a mediainfo report from in.mkv?

[the_steve_randolph]

Use handbrake

As stated above, I am looking for a solution that is a ffmpeg command. Isn't Handbrake based on ffmpeg? If it is, what are you suggesting Handbrake can/will do that I am not able to do with ffmpeg?

[the_steve_randolph]

Could you post a mediainfo report from in.mkv?

General
Unique ID                                : 21430015959520753467612615379059835278 (0x101F455C9EA18935E49B6DDD14D9818E)
Complete name                            : in.mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 1.02 GiB
Duration                                 : 1 h 36 min
Overall bit rate                         : 1 503 kb/s
Movie name                               : 
Encoded date                             : UTC 2015-11-30 06:45:49
Writing application                      : mkvmerge v5.1.0 ('And so it goes') built on Feb  1 2012 11:32:58
Writing library                          : libebml v1.2.2 + libmatroska v1.3.0
Attachments                              : 

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Muxing mode                              : Header stripping
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 36 min
Nominal bit rate                         : 1 200 kb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.116
Title                                    : 
Writing library                          : x264 core 120 r2151 a3f4407
Encoding settings                        : cabac=1 / ref=3 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=1200 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Audio #1
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 1 h 36 min
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : 80 ms
Title                                    : 
Language                                 : Russian
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 1 h 36 min
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : 80 ms
Title                                    : 
Language                                 : French
Default                                  : No
Forced                                   : No

Text #1
ID                                       : 4
Format                                   : VobSub
Muxing mode                              : zlib
Codec ID                                 : S_VOBSUB
Codec ID/Info                            : Picture based subtitle format used on DVDs
Title                                    : 
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text #2
ID                                       : 5
Format                                   : VobSub
Muxing mode                              : zlib
Codec ID                                 : S_VOBSUB
Codec ID/Info                            : Picture based subtitle format used on DVDs
Title                                    : 
Language                                 : German
Default                                  : No
Forced                                   : No

[ProWo]

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -c:s dvd_subtitle out.mkv

[the_steve_randolph]

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]"  - c:s dvd_subtitle out.mkv

I am assuming that you didn't mean to leave that space in "- c". But, when I run that command (without that space), I get this error: "Filter overlay has an unconnected output"

And, again, why are you excluding the audio?

[ProWo]

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -c:s:0 dvd_subtitle out.mkv

[the_steve_randolph]

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -c:s:0 dvd_subtitle out.mkv

That command generates the same error: "Filter overlay has an unconnected output"

[ProWo]

Then try

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -c:s:0 dvd_subtitle out.mkv

[the_steve_randolph]

Then try

ffmpeg -i in.mkv -map 0:v -map 0:s:0 -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -c:s:0 dvd_subtitle out.mkv

This ran and did not generate an error. This is the mediainfo for out.mkv:

Unique ID                                : 100830054107533089207352341002486193295 (0x4BDB297C1BA825198B84A99FD17A9C8F)
Complete name                            : out.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 1.09 GiB
Duration                                 : 1 h 36 min
Overall bit rate                         : 1 619 kb/s
Movie name                               : 
Writing application                      : Lavf58.78.100
Writing library                          : Lavf58.78.100
ErrorDetectionType                       : Per level 1

Video #1
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 36 min
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Title                                    : 
Writing library                          : x264 core 161
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Video #2
ID                                       : 2
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L6.2
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Width                                    : 400 pixels
Height                                   : 422 pixels
Display aspect ratio                     : 0.948
Frame rate mode                          : Constant
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Writing library                          : Lavc58.136.100 libx264
Default                                  : No
Forced                                   : No
Color range                              : Full
Matrix coefficients                      : BT.470 System B/G
FILENAME                                 : random_image.jpg
MIMETYPE                                 : image/jpeg

Video #3
ID                                       : 4
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 36 min
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Writing library                          : x264 core 161
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : No
Forced                                   : No

Text
ID                                       : 3
Format                                   : VobSub
Codec ID                                 : S_VOBSUB
Codec ID/Info                            : Picture based subtitle format used on DVDs
Duration                                 : 1 h 36 min
Title                                    : 
Writing library                          : Lavc58.136.100 dvdsub
Language                                 : English
Default                                  : Yes
Forced                                   : No

What I see when I view out.mkv in VLC:

• the audio tracks are missing (you still have not told me why you want to do that)
• two video tracks were added, but, they have no functionality
• the first (English) subtitle track is still there as a soft/embedded but not hardcoded subtitle
• the second subtitle track was removed

So, still not right

[ProWo]

ffmpeg -i in.mkv -filter_complex "[0:v:0][0:s:0]overlay[vid]" -map "[vid]" -c:s:0 dvd_subtitle -c:s:1 copy -c:a copy out.mkv

[the_steve_randolph]

ffmpeg -i in.mkv -filter_complex "[0:v:0][0:s:0]overlay[vid]" -map "[vid]" -c:s:0 dvd_subtitle -c:s:1 copy -c:a copy out.mkv

This is the mediainfo after that command:

General
Unique ID                                : 66492593406954912071231035078641825036 (0x320601F6CEDC5FA1124AAD02C1D5B50C)
Complete name                            : out.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 559 MiB
Duration                                 : 1 h 36 min
Overall bit rate                         : 808 kb/s
Movie name                               : 
Writing application                      : Lavf58.78.100
Writing library                          : Lavf58.78.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 36 min
Bit rate                                 : 792 kb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.076
Stream size                              : 548 MiB (98%)
Writing library                          : x264 core 161
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Viewing it in VLC, this is what I see:

• the audio tracks have been removed
• there is only one video track
• the English subtitle track is hardcoded into the video
• there are no selectable subtitle tracks

So, that did everything I wanted with the video and subtitles, but, there is no audio.

[ProWo]

So remux the new video with all tracks you need with mkvtoolnix

[the_steve_randolph]

So remux the new video with all tracks you need with mkvtoolnix

As stated in the OP, I am trying to find a solution with ffmpeg. It is hard to learn how to use something if you give up too soon.

[ProWo]

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map 0:a -map 0:s:1 -c:a copy -c:s copy out.mkv

[the_steve_randolph]

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map 0:a -map 0:s:1 -c:a copy -c:s copy out.mkv

Viewing out.mkv in VLC, this is what I see:

• both audio tracks are there
• there is only one video track
• the English subtitle track is hardcoded into the video
• there is a single subtitle track that is German

So, that did not leave just the one audio track and it left a subtitle track.

Looking back at the various commands I have tried, it seems that the -filter_complex" switch does not play nice with "map" processing. This video is large enough that it is taking me about 15 minutes to run each test because of the re-encoding of the video. I am working on it now and will get back with my findings. Feel free to post another try. Thanks for your help.

[ProWo]

To keep only the first audio stream and no subtitle stream use:

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map 0:a:0 -c:a copy out.mkv

To keep only the second audio stream and no subtitle stream use:

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map 0:a:1 -c:a copy out.mkv

The stream numbering of ffmpeg is different from that of mediainfo.
To see the correct stream numbering of ffmpeg, just enter the following:

ffmpeg -i in.mkv

[the_steve_randolph]

WE HAVE SUCCESS!!!!!

Just to recap for future readers of this thread:

The video I have is:

• an MKV
• no "disc" titles
• no chapters
• the first audio track is Russian
• the second audio track is French
• one video track
• the first embedded subtitle track is English (image-based dvd_subtitle)
• the second embedded subtitle track is German (image-based dvd_subtitle)

What I want to do is:

• only use a single ffmpeg command (the version I am using here is: N-101923-gc012f9b265)
• remove the Russian audio track
• hardcode the English subtitle track onto the video
• remove both embedded subtitle tracks

My computer is running updated Windows 10 Pro, Intel Core i9 7940x, 64GB RAM, GEForce GTX1070, and, I am executing the ffmpeg command in a CMD window.


The successful command is:

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map -0:a:0 -map 0:a:1 -sn out.mkv

Converting to MKV lost the "frame width" and "frame height" metadata tags, but, they are not lost when converting to an MP4. In successive efforts I have added "-crf 18" to retain the original quality, assigned a new name to the kept audio track, and, assigned a new name to the "title" metadata tag. The tricky part was making the audio-naming tag of the kept track since I was removing the original audio track #0. I can't make HandBrake do all of that, which is just one of the reasons why I wanted to do all of this in a single ffmpeg command. Here is that command:

ffmpeg -i in.mkv -filter_complex "[0:v][0:s:0]overlay[v]" -map "[v]" -map -0:a:0 -map 0:a:1 -sn -crf 18 -metadata:s:a:0 title=French -metadata:s:a:0 language="" -metadata title="My Title [1974]" out.mp4

A big thanks to ProWo for all of the help and patience, and, a to Selur