VideoHelp Forum




+ Reply to Thread
Results 1 to 13 of 13
  1. I've been building my own HLS/DASH downloader and it's stable enough to put in front of people. Free, open source, and I'm the author - so weigh the benchmarks accordingly and please tell me where it falls over.

    https://github.com/mohaanymo/m314dl

    What it handles:

    - HLS (.m3u8) and DASH (.mpd) - master/media playlists, rendition groups, byte ranges, multi-period manifests with discontinuity markers, fMP4 and TS.
    - Decryption in-process and in-memory, overlapped with the download: CENC (AES-CTR), cbcs (AES-CBC pattern), cens, cbc1. No mp4decrypt or shaka-packager pass afterwards. You pass the key yourself with -key KID:KEY - m314dl does not obtain keys and has nothing to do with getting them.
    - Byte-exact resume. It picks up on the exact byte mid-segment, so an interrupted download doesn't restart or come back with padding errors.
    - Adaptive concurrency, AIMD between 4 and 64 in-flight requests. It backs off on its own when a server starts throwing 503s instead of you guessing a thread count.
    - Live recording with -live-duration and clean Ctrl-C - you get a playable file, not a truncated one.
    - Subtitles: WebVTT, TTML to SRT, stpp-in-fMP4 extraction.
    - Ad-segment skipping by regex, stream selection by regex, custom headers, proxy, cookies.txt.

    Speed on a 5-min 1080p HLS/TS test, same machine and connection: ~0.95s vs ~1.12s for N_m3u8DL-RE and ~1.60s for vsd. Decryption alone benchmarks ~1.4 GB/s (CTR) and ~1.05 GB/s (CBC). My box, my numbers - I'd like to see other people's.

    Single static binary, no runtime deps except ffmpeg on PATH for muxing. Windows, Linux, macOS.

    Code:
    go build -o m314dl .
    Code:
    m314dl -list https://example.com/manifest.mpd
    m314dl -o out.mp4 https://example.com/master.m3u8
    m314dl -key KID:KEY -o out.mp4 https://example.com/manifest.mpd
    m314dl -live-duration 1h -o show.mp4 https://example.com/live.m3u8
    Bug reports, manifests that break it, and resume-corrupted-my-file reports all welcome - GitHub issues or reply here.
    Quote Quote  
  2. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Good work. Another one for the armoury along with N_m3u8DL_RE and dash_mpd_cli.
    It appearance is a little basic but it works and is fast.
    The only slight annoyance is the order it accepts arguments in the command. If this were to be a drop-in replacement it might be a good idea to use a 'don't-care' order for them or follow the daddy-of-them all N_M3u8DL_RE's ordering.
    Noob Starter Pack. Just download any Widevine media! Over 29,000 downloads for V6!.
    https://files.videohelp.com/u/301890/hellyes6.zip
    Quote Quote  
  3. Originally Posted by A_n_g_e_l_a View Post
    Good work. Another one for the armoury along with N_m3u8DL_RE and dash_mpd_cli.
    It appearance is a little basic but it works and is fast.
    The only slight annoyance is the order it accepts arguments in the command. If this were to be a drop-in replacement it might be a good idea to use a 'don't-care' order for them or follow the daddy-of-them all N_M3u8DL_RE's ordering.
    thanks for the feedback, I will consider taking the arguments command problem and fix it in the new versions,

    about the appearance, actually it meant to be this way for this reason "Automation-friendly: plain-line progress on non-TTY (no ANSI garbage in logs), real exit codes, quiet machine-readable output"
    Quote Quote  
  4. Might want to be upfront about the AI side of the project
    Quote Quote  
  5. Originally Posted by weiLahw9 View Post
    Might want to be upfront about the AI side of the project
    I do use AI for cleaning code and documentation
    Quote Quote  
  6. we are drowning in AI slop
    Bypass HMACs, One-time-tokens and Lic.Wrapping: https://github.com/DevLARLEY/WidevineProxy2
    Quote Quote  
  7. nice. curious to test it. thanks for sharing
    Quote Quote  
  8. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Thanks. Neither N_m3u8DL_RE or dash_mpdcli work to download using an mpd from france.tv - the url fails in dns lookup.

    The url (https://cloudingest.ftven.fr/ftv/b/70/b700192c68796/f46ded4d-5d1e-41a8-8825-78d54092f5...m/manifest.mpd) for an encrypted title works with this downloader.

    So thank you for developing this.

    One hiccup it picks up the subtitles but throws an error during conversion. it is a qsm stream with vtt codec.

    Code:
     m314dl -key f46ded4d5d1e41a8882578d54092f512:e43d3149769ccd639ddb884fbbbae4c1 -o 'Astrid_S02E05' https://cloudingest.ftven.fr/ftv/b/70/b70
    0192c68796/f46ded4d-5d1e-41a8-8825-78d54092f512-1784202207_france-domtom_DRM-SEC1.ism/manifest.mpd
    selected: video=2745000.3.3 video 1920x1080 2745kbps avc1.640029 (video=2745000)
    selected: audio_fre=192000.0.0 audio 192kbps mp4a.40.2 [fr] (audio_fre=192000)
    selected: textstream_qsm=1000.1.0 subtitles 1kbps wvtt [qsm] (textstream_qsm=1000)
    100.0% | 3425/3425 segs | 1.07GiB | 1.6MiB/s | ETA ?
    warning: subtitle textstream_qsm=1000.1.0 conversion failed (ffmpeg subtitle extract failed: exit status 234
    [out#0/srt @ 0xb2b85c0] Output file does not contain any stream
    Error opening output file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.srt.
    Error opening output files: Invalid argument); keeping raw file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.rawsub
    done: Astrid_S02E05.mp4
    I used a python downloader of my own, for the subs.

    I want to emphasise to all those who sneer a someone for using AI, this worked when the other more respectable programs failed.
    Noob Starter Pack. Just download any Widevine media! Over 29,000 downloads for V6!.
    https://files.videohelp.com/u/301890/hellyes6.zip
    Quote Quote  
  9. Originally Posted by A_n_g_e_l_a View Post
    Thanks. Neither N_m3u8DL_RE or dash_mpdcli work to download using an mpd from france.tv - the url fails in dns lookup.

    The url (https://cloudingest.ftven.fr/ftv/b/70/b700192c68796/f46ded4d-5d1e-41a8-8825-78d54092f5...m/manifest.mpd) for an encrypted title works with this downloader.

    So thank you for developing this.

    One hiccup it picks up the subtitles but throws an error during conversion. it is a qsm stream with vtt codec.

    Code:
     m314dl -key f46ded4d5d1e41a8882578d54092f512:e43d3149769ccd639ddb884fbbbae4c1 -o 'Astrid_S02E05' https://cloudingest.ftven.fr/ftv/b/70/b70
    0192c68796/f46ded4d-5d1e-41a8-8825-78d54092f512-1784202207_france-domtom_DRM-SEC1.ism/manifest.mpd
    selected: video=2745000.3.3 video 1920x1080 2745kbps avc1.640029 (video=2745000)
    selected: audio_fre=192000.0.0 audio 192kbps mp4a.40.2 [fr] (audio_fre=192000)
    selected: textstream_qsm=1000.1.0 subtitles 1kbps wvtt [qsm] (textstream_qsm=1000)
    100.0% | 3425/3425 segs | 1.07GiB | 1.6MiB/s | ETA ?
    warning: subtitle textstream_qsm=1000.1.0 conversion failed (ffmpeg subtitle extract failed: exit status 234
    [out#0/srt @ 0xb2b85c0] Output file does not contain any stream
    Error opening output file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.srt.
    Error opening output files: Invalid argument); keeping raw file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.rawsub
    done: Astrid_S02E05.mp4
    I used a python downloader of my own, for the subs.

    I want to emphasise to all those who sneer a someone for using AI, this worked when the other more respectable programs failed.
    Thanks angela, could you submit this issue in the repo and I will work on it as fast as I can,
    about AI, I think some people confuse using AI without understanding what it's doing (vibe coding) with using AI while having a clear understanding of what you're doing. I don't vibe coding
    Quote Quote  
  10. Originally Posted by A_n_g_e_l_a View Post
    Thanks. Neither N_m3u8DL_RE or dash_mpdcli work to download using an mpd from france.tv - the url fails in dns lookup.

    The url (https://cloudingest.ftven.fr/ftv/b/70/b700192c68796/f46ded4d-5d1e-41a8-8825-78d54092f5...m/manifest.mpd) for an encrypted title works with this downloader.

    So thank you for developing this.

    One hiccup it picks up the subtitles but throws an error during conversion. it is a qsm stream with vtt codec.

    Code:
     m314dl -key f46ded4d5d1e41a8882578d54092f512:e43d3149769ccd639ddb884fbbbae4c1 -o 'Astrid_S02E05' https://cloudingest.ftven.fr/ftv/b/70/b70
    0192c68796/f46ded4d-5d1e-41a8-8825-78d54092f512-1784202207_france-domtom_DRM-SEC1.ism/manifest.mpd
    selected: video=2745000.3.3 video 1920x1080 2745kbps avc1.640029 (video=2745000)
    selected: audio_fre=192000.0.0 audio 192kbps mp4a.40.2 [fr] (audio_fre=192000)
    selected: textstream_qsm=1000.1.0 subtitles 1kbps wvtt [qsm] (textstream_qsm=1000)
    100.0% | 3425/3425 segs | 1.07GiB | 1.6MiB/s | ETA ?
    warning: subtitle textstream_qsm=1000.1.0 conversion failed (ffmpeg subtitle extract failed: exit status 234
    [out#0/srt @ 0xb2b85c0] Output file does not contain any stream
    Error opening output file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.srt.
    Error opening output files: Invalid argument); keeping raw file Astrid_S02E05.m314dl-subtitles-textstream_qsm_1000.1.0.rawsub
    done: Astrid_S02E05.mp4
    I used a python downloader of my own, for the subs.

    I want to emphasise to all those who sneer a someone for using AI, this worked when the other more respectable programs failed.
    N_m3u8DL-RE 0.5.1-beta works perfectly for this stream. PEBCAK.
    Quote Quote  
  11. Excited to test it.
    Quote Quote  
  12. Originally Posted by danthe View Post
    Excited to test it.
    happy to hear your feedback
    Quote Quote  
  13. Originally Posted by mohaanymo View Post
    happy to hear your feedback
    My personal feedback: your app works fine, but honestly i don't find any great vantages compared to using N_m3u8DL-RE.
    for me they are both the same also the only issue that i find on your app is from download it reports a % of segments like 15/900 when in reality from the folder where the command is executed the downloaded segments are much more like 250/900, so what is reported in the command window of segments number is relative wrong % compared to the real segments already downloaded.
    but it's just a not relevant issue.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!