Hello everyone.
This is the website(requires JPN VPN): https://plus.nhk.jp/watch/ch/g1
The Video I'm looking to download srt from (requires free registration, but it functions the same as the live stream in the first url): here
While I managed to decrypt the mp4 file from this video, yt-dlp didn't include the subs with the video, even though there is a mention of "stpp" in a line inside the manifest mpd file:I tried --list-subs on the mpd url but all I got is "Language Formats: und mp4. Whatever that means.<Representation codecs="stpp" startWithSAP="1" mimeType="application/mp4" bandwidth="4096" id="4" />
After searching for similar questions in the forum, I found that you can download the subs seperatly via this command:
it worked, the subtitle lines are there, but the xml file is a mess inside and cannot be opened via SubtitleEdit. However, when I view the video using TubeDigger, it instantly recognises a srt file and downloads it for me, and sure enough, it's 100% working and can be opened with SE. And this is exactly why I created this thread, I'd like to know how to get that srt file.yt-dlp --write-subs --external-downloader aria2c --allow-u "mpd" --output sub.xml
If you want to test the mpd file yourself:
My theory is that the srt file is being handled outside the mpd file, via a plugin of sorts. It doesn't show up in the Network Traffic but when you highlight the subs and click inspect element them this is what I see:mpd: https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0021/cenc/fmp4/manifest.mpd?start=2022-03-...03-06-16:45:06
key: 0cd6e5a09b904ea59e1b43dfc5479c2e:080f0ad6d6fe59552 643f09bd05c754d
jbsubtitlelibrary.js is also present in the Network Traffic, no idea if it could be related.
are there other stream services that handle the subs of a stream with a similar approach?
+ Reply to Thread
Results 1 to 18 of 18
-
-
If you can use Tubedigger why do you need an alternative?
This is a pain in the ass for registering, many steps, sometimes translate doesn't show up...
Send login details via private message.
[Attachment 63760 - Click to enlarge] -
Sorry you had to go through this painful process, I should've just provided the credentials earlier. TubeDigger still doesn't work on all videos for some reason, especially during live streamed documentaries that can never be re-watched again, they still have subtitles embedded in them. PM Sent.
Last edited by Mirai; 10th Mar 2022 at 11:36.
-
TubeDigger is converting subtitles to SubRips internally.
Using the MPD link you gave,
Code:yt-dlp --no-warnings --allow-unplayable-formats --downloader aria2c --downloader-args "aria2c:--http-accept-gzip=true" --no-mtime --write-subs --skip-download -k --fixup never --force-generic-extractor "https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0021/cenc/fmp4/manifest.mpd?start=2022-03-06-16:20:06&end=2022-03-06-16:45:06"
Subtitle Edit can open this mp4 without an issue. All you need to do after that is go to Synchronization -> Adjust all times and put the start time of the first line in, click show earlier and then save as an srt. -
Countless of hours researching and I have never came across the fact that an mp4 can be it's own subtitle
! SE is truly a beast!
Thank you so much for the reply, now that the main question has been addressed, I'm still left with another. How do I apply the same ytdlp code on a livestream that has multiple mpd segments? Using the same command will only download about 6 seconds long worth of subs then it stops. -
Sorry, I'm not sure, as I haven't had to deal with that. If in your browser's developer tools network tab, you're seeing multiple manifest.mpd requests with different timestamps in the addresses with six seconds' difference between them, you might be able to have yt-dlp batch download them and then merge them together with Subtitle Edit.
-
I have tried to batch download them with yt-dlp, the problem is that it downloads the first mpd but when it tries to download the second one I get "manifest.f0.mp4 has already been downloaded". The website itself keeps updating the mpd file every 6 seconds so the name is still the same for yt-dlp. Is there a way to have yt-dlp download the mpd segments but also allow copies of the same mpd file instead of ignoring them? For example: (manifest.f0.mp4, manifest.f0.mp4 Copy (1), manifest.f0.mp4 Copy (2).)
EDIT: Here's how the network traffic looks like when watching the stream:
Last edited by Mirai; 13th Mar 2022 at 00:42.
-
-
how about --autonumber with --autonumber-start?
Code:autonumber (numeric): Number that will be increased with each download, starting at --autonumber-start
-
I am also trying to dl a video from NHK+
https://plus.nhk.jp/watch/st/e1_2022033011401?t=5
I found the MPD link, but where can I find the key to dl it ? and how do I use the key in yt-dlp ?
https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0031/cenc/fmp4/manifest.mpd?start=2022-03-...03-30-14:20:06
yt-dlp --no-warnings --allow-unplayable-formats --downloader aria2c --downloader-args "aria2c:--http-accept-gzip=true" --no-mtime --write-subs --skip-download -k --fixup never --force-generic-extractor "https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0021/cenc/fmp4/manifest.mpd?start=2022-03-06-16:20:06&end=2022-03-06-16:45:06"
write-audio / write-video does not work
edit: MPD
Code:<?xml version="1.0" encoding="UTF-8" standalone="no" ?> <MPD minimumUpdatePeriod="PT03S" ns1:schemaLocation="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" minBufferTime="PT03S" mediaPresentationDuration="PT20M" xmlns:ns1="http://www.w3.org/2001/XMLSchema-instance" availabilityStartTime="2022-03-30T14:00:07Z" xmlns:cenc="urn:mpeg:cenc:2013" maxSegmentDuration="PT06S" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:mspr="urn:microsoft:playready" type="static" timeShiftBufferDepth="PT05M"> <BaseURL>https://vod-cdn1-origin1.hsk.nhk.jp/</BaseURL> <BaseURL>https://vod-cdn1-origin3.hsk.nhk.jp/</BaseURL> <BaseURL>https://vod-cdn1-origin2.hsk.nhk.jp/</BaseURL> <Period start="PT0S" id="1"> <AdaptationSet segmentAlignment="true" id="0"> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" /> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED" value="2.0"> <cenc:pssh>AAAAWXBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAADkSEB9hGVQZ/0XSgx/O72GXSmMaDXdpZGV2aW5lX3Rlc3QiEMxC+eyoV0dDtQoAMdYbRmxI49yVmwY=</cenc:pssh> </ContentProtection> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:9A04F079-9840-4286-AB92-E65BE0885F95" value="2.0"> <mspr:pro>dgMAAAEAAQBsAzwAVwBSAE0ASABFAEEARABFAFIAIAB4AG0AbABuAHMAPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBvAG0ALwBEAFIATQAvADIAMAAwADcALwAwADMALwBQAGwAYQB5AFIAZQBhAGQAeQBIAGUAYQBkAGUAcgAiACAAdgBlAHIAcwBpAG8AbgA9ACIANAAuADAALgAwAC4AMAAiAD4APABEAEEAVABBAD4APABQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsARQBZAEwARQBOAD4AMQA2ADwALwBLAEUAWQBMAEUATgA+ADwAQQBMAEcASQBEAD4AQQBFAFMAQwBUAFIAPAAvAEEATABHAEkARAA+ADwALwBQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsASQBEAD4AVgBCAGwAaABIAC8AOABaADAAawBXAEQASAA4ADcAdgBZAFoAZABLAFkAdwA9AD0APAAvAEsASQBEAD4APABMAEEAXwBVAFIATAA+AGgAdAB0AHAAOgAvAC8AcABsAGEAeQByAGUAYQBkAHkALgBkAHIAbQBrAGUAeQBzAGUAcgB2AGUAcgAuAGMAbwBtAC8AcABsAGEAeQByAGUAYQBkAHkALwByAGkAZwBoAHQAcwBtAGEAbgBhAGcAZQByAC4AYQBzAG0AeAA8AC8ATABBAF8AVQBSAEwAPgA8AEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAIAB4AG0AbABuAHMAPQAiACIAPgA8AEMATwBOAFQARQBOAFQASQBEAD4AQwBDADQAMgBGADkARQBDAC0AQQA4ADUANwAtADQANwA0ADMALQBCADUAMABBAC0AMAAwADMAMQBEADYAMQBCADQANgA2AEMAPAAvAEMATwBOAFQARQBOAFQASQBEAD4APAAvAEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAPgA8AEMASABFAEMASwBTAFUATQA+AHMAMQA5AGoAOAAxAEEAawAyAE0AdwA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==</mspr:pro> </ContentProtection> <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" /> <SegmentTemplate timescale="90000" initialization="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/init.mp4?start_time=2022-03-30-14:00:06" media="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/$Time$.m4s"> <SegmentTimeline> <S t="148375476634217" d="540000"/> <S t="148375477080000" d="540000" r="198" /> </SegmentTimeline> </SegmentTemplate> <Representation codecs="mp4a.40.2" startWithSAP="1" mimeType="audio/mp4" bandwidth="76800" id="1" /> </AdaptationSet> <AdaptationSet segmentAlignment="true" id="1"> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" /> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED" value="2.0"> <cenc:pssh>AAAAWXBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAADkSEB9hGVQZ/0XSgx/O72GXSmMaDXdpZGV2aW5lX3Rlc3QiEMxC+eyoV0dDtQoAMdYbRmxI49yVmwY=</cenc:pssh> </ContentProtection> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:9A04F079-9840-4286-AB92-E65BE0885F95" value="2.0"> <mspr:pro>dgMAAAEAAQBsAzwAVwBSAE0ASABFAEEARABFAFIAIAB4AG0AbABuAHMAPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBvAG0ALwBEAFIATQAvADIAMAAwADcALwAwADMALwBQAGwAYQB5AFIAZQBhAGQAeQBIAGUAYQBkAGUAcgAiACAAdgBlAHIAcwBpAG8AbgA9ACIANAAuADAALgAwAC4AMAAiAD4APABEAEEAVABBAD4APABQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsARQBZAEwARQBOAD4AMQA2ADwALwBLAEUAWQBMAEUATgA+ADwAQQBMAEcASQBEAD4AQQBFAFMAQwBUAFIAPAAvAEEATABHAEkARAA+ADwALwBQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsASQBEAD4AVgBCAGwAaABIAC8AOABaADAAawBXAEQASAA4ADcAdgBZAFoAZABLAFkAdwA9AD0APAAvAEsASQBEAD4APABMAEEAXwBVAFIATAA+AGgAdAB0AHAAOgAvAC8AcABsAGEAeQByAGUAYQBkAHkALgBkAHIAbQBrAGUAeQBzAGUAcgB2AGUAcgAuAGMAbwBtAC8AcABsAGEAeQByAGUAYQBkAHkALwByAGkAZwBoAHQAcwBtAGEAbgBhAGcAZQByAC4AYQBzAG0AeAA8AC8ATABBAF8AVQBSAEwAPgA8AEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAIAB4AG0AbABuAHMAPQAiACIAPgA8AEMATwBOAFQARQBOAFQASQBEAD4AQwBDADQAMgBGADkARQBDAC0AQQA4ADUANwAtADQANwA0ADMALQBCADUAMABBAC0AMAAwADMAMQBEADYAMQBCADQANgA2AEMAPAAvAEMATwBOAFQARQBOAFQASQBEAD4APAAvAEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAPgA8AEMASABFAEMASwBTAFUATQA+AHMAMQA5AGoAOAAxAEEAawAyAE0AdwA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==</mspr:pro> </ContentProtection> <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" /> <SegmentTemplate timescale="90000" initialization="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/init.mp4?start_time=2022-03-30-14:00:06" media="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/$Time$.m4s"> <SegmentTimeline> <S t="148375476634217" d="540000"/> <S t="148375477080000" d="540000" r="198" /> </SegmentTimeline> </SegmentTemplate> <Representation codecs="mp4a.40.2" startWithSAP="1" mimeType="audio/mp4" bandwidth="76800" id="2" /> </AdaptationSet> <AdaptationSet segmentAlignment="true" id="2"> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" /> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED" value="2.0"> <cenc:pssh>AAAAWXBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAADkSEB9hGVQZ/0XSgx/O72GXSmMaDXdpZGV2aW5lX3Rlc3QiEMxC+eyoV0dDtQoAMdYbRmxI49yVmwY=</cenc:pssh> </ContentProtection> <ContentProtection cenc:default_KID="1F611954-19FF-45D2-831F-CEEF61974A63" schemeIdUri="urn:uuid:9A04F079-9840-4286-AB92-E65BE0885F95" value="2.0"> <mspr:pro>dgMAAAEAAQBsAzwAVwBSAE0ASABFAEEARABFAFIAIAB4AG0AbABuAHMAPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBvAG0ALwBEAFIATQAvADIAMAAwADcALwAwADMALwBQAGwAYQB5AFIAZQBhAGQAeQBIAGUAYQBkAGUAcgAiACAAdgBlAHIAcwBpAG8AbgA9ACIANAAuADAALgAwAC4AMAAiAD4APABEAEEAVABBAD4APABQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsARQBZAEwARQBOAD4AMQA2ADwALwBLAEUAWQBMAEUATgA+ADwAQQBMAEcASQBEAD4AQQBFAFMAQwBUAFIAPAAvAEEATABHAEkARAA+ADwALwBQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsASQBEAD4AVgBCAGwAaABIAC8AOABaADAAawBXAEQASAA4ADcAdgBZAFoAZABLAFkAdwA9AD0APAAvAEsASQBEAD4APABMAEEAXwBVAFIATAA+AGgAdAB0AHAAOgAvAC8AcABsAGEAeQByAGUAYQBkAHkALgBkAHIAbQBrAGUAeQBzAGUAcgB2AGUAcgAuAGMAbwBtAC8AcABsAGEAeQByAGUAYQBkAHkALwByAGkAZwBoAHQAcwBtAGEAbgBhAGcAZQByAC4AYQBzAG0AeAA8AC8ATABBAF8AVQBSAEwAPgA8AEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAIAB4AG0AbABuAHMAPQAiACIAPgA8AEMATwBOAFQARQBOAFQASQBEAD4AQwBDADQAMgBGADkARQBDAC0AQQA4ADUANwAtADQANwA0ADMALQBCADUAMABBAC0AMAAwADMAMQBEADYAMQBCADQANgA2AEMAPAAvAEMATwBOAFQARQBOAFQASQBEAD4APAAvAEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAPgA8AEMASABFAEMASwBTAFUATQA+AHMAMQA5AGoAOAAxAEEAawAyAE0AdwA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==</mspr:pro> </ContentProtection> <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" /> <SegmentTemplate timescale="90000" initialization="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/init.mp4?start_time=2022-03-30-14:00:06" media="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/$Time$.m4s"> <SegmentTimeline> <S t="148375476632927" d="540000"/> <S t="148375477080000" d="540000" r="198" /> </SegmentTimeline> </SegmentTemplate> <Representation width="960" height="540" codecs="avc1.4d4028" startWithSAP="1" mimeType="video/mp4" bandwidth="1800000" id="0" /> <Representation width="640" height="360" codecs="avc1.4d4028" startWithSAP="1" mimeType="video/mp4" bandwidth="921600" id="100" /> <Representation width="448" height="252" codecs="avc1.4d401e" startWithSAP="1" mimeType="video/mp4" bandwidth="460800" id="200" /> <Representation width="416" height="232" codecs="avc1.42c01e" startWithSAP="1" mimeType="video/mp4" bandwidth="230400" id="300" /> </AdaptationSet> <AdaptationSet segmentAlignment="true" id="3"> <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" /> <SegmentTemplate timescale="90000" initialization="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/init.mp4?start_time=2022-03-30-14:00:06" media="catchup/api/v1/segment/0031/$RepresentationID$/fmp4/cenc/$Time$.m4s"> <SegmentTimeline> <S t="148375476632927" d="540000"/> <S t="148375477080000" d="540000" r="198" /> </SegmentTimeline> </SegmentTemplate> <Representation codecs="stpp" startWithSAP="1" mimeType="application/mp4" bandwidth="4096" id="4" /> </AdaptationSet> </Period> </MPD>
Last edited by nimrahwt; 30th Mar 2022 at 21:50.
-
sent!
I thought you only need the mpd link + the key in it to download itLast edited by nimrahwt; 31st Mar 2022 at 09:37.
-
thanks, I figured it out how to download the video+ audio and decrypted it thanks to this thread
https://forum.videohelp.com/threads/403186-Need-help-downloading-a-tv-show
but how did you get the key? there will be more episodes, and I don't want to always ask you -
yes all info is here: https://forum.videohelp.com/threads/403186-Need-help-downloading-a-tv-show#post2631994
to download put this code for yt-dlp
Code:yt-dlp --allow-u "https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0031/cenc/fmp4/manifest.mpd?start=2022-03-30-14:00:06&end=2022-03-30-14:20:06"
-
yep ^that also works
Code:yt-dlp --allow-u --external-downloader aria2c -f bv,ba "https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0031/cenc/fmp4/manifest.mpd?start=2022-03-31-14:00:06&end=2022-03-31-14:20:06"
Episode 2 is up, guess there is another key?
https://plus.nhk.jp/watch/st/e1_2022033112147
https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0031/cenc/fmp4/manifest.mpd?start=2022-03-...03-31-14:20:06
I dont't see how to get the PlayReady key thereLast edited by nimrahwt; 31st Mar 2022 at 11:01.
-
on this case the key is the same at post#13
for this new link replace to my string above your link on yt-dlp to download
Code:https://vod-cdn1-origin1.hsk.nhk.jp/catchup/0031/cenc/fmp4/manifest.mpd?start=2022-03-31-14:00:06&end=2022-03-31-14:20:06
Last edited by lomero; 1st Apr 2022 at 01:36.
-
small update:
NHK+ changed from MPD to M3U8
instead of aria2c you can just use N_m3u8DL-CLI, its much fasterLast edited by nimrahwt; 19th Jan 2023 at 14:27.
Similar Threads
-
Download free stream, encrypted with widevine
By Bartimeyse in forum Video Streaming DownloadingReplies: 4Last Post: 17th Apr 2021, 05:31 -
Download stream encrypted with EXT-X-HASH
By vanikem in forum Video Streaming DownloadingReplies: 6Last Post: 1st Oct 2019, 22:20 -
Help downloading AES-encrypted HLS video stream
By Woodswolf in forum Video Streaming DownloadingReplies: 26Last Post: 25th May 2019, 14:20 -
Help downloading AES-encrypted HLS video stream
By vidder in forum Video Streaming DownloadingReplies: 3Last Post: 4th Jul 2018, 17:24 -
Help needed on Encrypted RTMP stream...
By invictus in forum Newbie / General discussionsReplies: 2Last Post: 31st Jan 2018, 04:55