My journey with AES-128 encrypted HLS

[dronem]

Hello everybody,

I developed some interest in understanding how to deal with AES-128 encrypted HTTP Live Streams (HLS). Basically for fun and the forum was a very good starting point for me, from a practical perspective. So I thought it would be a good first post to share my journey and some insights I gained.

I think they could be useful for others as well, as I had the impression that the question of how to deal with m3u8 playlists comes up regularly. And I think a bird's-eye perspective can be helpful.

But this said, I don't aim to provide a guide without presuppositions. Some kind of technical understanding and RTFM mentality is expected.

1. HL...what?

So, you want to download an AES-128 encrypted HTTP Live Stream. What is the situation you are dealing with?

In simplest terms, two things happened to the video file you are interested in:

1. The video got segmented into a lot of parts.
2. Those parts got encrypted with AES-128.

So to watch the video, a player has to know exactly those two things:

1. What are the parts of the video?
2. What is the key to decrypt the individual video segments?

This is where the playlist/manifest comes into play. Something like "index.m3u8". But the file can be named however...

For my experiment, I've chosen the preview video of that online course:

https://academicenglishnow.com/courses/research-paper-mastery/lessons/week-1-day-4-mai...elling-a-story

2.Getting the manifest

So far we know that the manifest is the key to accessing the video, this is of course based on the information that manifest is holding. But before we take a closer look onto that, let's get a manifest in the first place.

IMHO the easiest way is to use the network tab inside the developer tools of your browser (I hope you defend the web and use non-chrome-based one! e.g. Firefox), reload the page, start the video player on the page, let it play for some time and stop it. Afterwards, filter the traffic for "m3u8". You should see something like that:


[Attachment 77498 - Click to enlarge]

Interesting, it appears like there are three manifests. So let's take a closer look. If we check out the "Week1/Wk-1-Day-4-Maintaing-flow-and-telling-a-coherent-story.m3u8" file, we see the following:

#EXTM3U
#EXT-X-VERSION:3

#EXT-X-STREAM-INF:BANDWIDTH=1047200,RESOLUTION=640x360,CODECS="avc1.4d401f,mp4a.40.2"
0kr337e5_0.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1460800,RESOLUTION=854x480,CODECS="avc1.4d401f,mp4a.40.2"
0kr337e5_1.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=3220800,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2"
0kr337e5_2.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=5711200,RESOLUTION=1920x1080,CODECS="avc1.4d401f,mp4a.40.2"
0kr337e5_3.m3u8

So that manifest only contains links to other manifests for different resolutions. And apparently, I first loaded the manifest for 1280x720 and then the manifest for 1920x1080. (This is also one of the reasons why HLS is used: adaptive streaming.)

So the next step would be taking a look into one of the other manifests, I decided to look into the "0kr337e5_3.m3u8" file.

Which contains the following data:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="https://media.publit.io/hls/eyJpdiI6InA4OEpOTTRYbTN3VUpMdHF0WHkwM0E9PSIsInZhbHVlIjoiSkhHVzRuMUF5alRUdXVwZlJENXB3UT09IiwibWFjIjoiYWY4MjE2ODdhNDc4Y2EzNWJkNDE2MjY1OTIxYTg2N2JiODgxNWM0NzBjNWNlNDhjYTEzMWQ2OThmYTgwY2Y1NiJ9.key",IV=0x73dbb9070d82cf1cf05087ab59c21ef8
#EXTINF:11.211200,
0kr337e5_3_0.ts
#EXTINF:9.609600,
0kr337e5_3_1.ts
#EXTINF:9.609600,
0kr337e5_3_2.ts
[...]
#EXTINF:9.609600,
0kr337e5_3_199.ts
#EXTINF:5.805800,
0kr337e5_3_200.ts
#EXT-X-ENDLIST

Interesting, this is apparently the information that is needed for a player to reconstruct the video. There is a list of all the segments (all the 0kr337e5_3_xxx.ts files).

And there is also this interesting line:

#EXT-X-KEY:METHOD=AES-128,URI="https://media.publit.io/hls/eyJpdiI6InA4OEpOTTRYbTN3VUpMdHF0WHkwM0E9PSIsInZhbHVlIjoiSkhHVzRuMUF5alRUdXVwZlJENXB3UT09IiwibWFjIjoiYWY4MjE2ODdhNDc4Y2EzNWJkNDE2MjY1OTIxYTg2N2JiODgxNWM0NzBjNWNlNDhjYTEzMWQ2OThmYTgwY2Y1NiJ9.key",IV=0x73dbb9070d82cf1cf05087ab59c21ef8

which is stating that the segments are encrypted in AES-128 and also the URI to get the key. (And if needed the IV)

If you want to know more about the entries inside the m3u8, well, Wikipedia.

3. Getting the key

This is the part that can get tricky. Because the key has to be provided to the player and HLS itself doesn't have a concept of how to prevent/decide what is a legitimate request for the key. Therefore, everybody providing an HLS is urged to get creative in restricting access to the key. Based on what I have seen in various posts here, this can be something like certain header information and so on... Actually, I would find it really interesting collecting those.

The first thing I tried was curling the provided URI which didn't work. So I thought if I can play the video in the browser, there has to be a successful request. So I kept looking for it and the network tab got me:

[Attachment 77499 - Click to enlarge]

There was the response I needed a key:

s3Pa8D/Ls0PbVjXrhJzRpw==

(I also got curious because the URI looked like base64 for and indeed it was. Even if the information there wasn't necessary.)

4. Getting the video

So now I just needed something to load, decrypt, and join the segments. Which I did with the often mentioned hlsdl. I realized that I have to provide the key in a HEX format so I converted it:

echo "s3Pa8D/Ls0PbVjXrhJzRpw==" | od -A n -t x1 | sed  's/ //g' | sed -z 's/\n//g'

Which resulted in the following key:

7333506138442f4c73305062566a5872684a7a5270773d3d0a

Which enabled me to download it with hlsdl like that:

hlsdl -k "7333506138442f4c73305062566a5872684a7a5270773d3d0A" "https://media.publit.io/file/AcademicEnglish/ResearchPaperMastery/Week1/0kr337e5_3.m3u8"

I hope you found it useful! I'm looking forward to critique, other solutions, or input that deepens my understanding.

[2nHxWW6GkN1l916N3ayz8HQoi]

Nice short tutorial and pretty nice explanations. You kinda hit an easy video though. I suppose most hls aes 128 videos are like this. However, there are tougher ones that use tricks like this to stop you:
- you can't even find a network request where you can get the plain m3u8 because it is hidden so even if you filter by m3u8 in network requests, you can't find anything
- the hls key obtained from that URL is altered in the code so you can't use the one from the URL directly without knowing how it's modified to get the real key
- there may be additional mechanisms on top of the hls aes 128 encryption that will stop you from downloading it directly even if you get the key and iv
- some fragments may be broken so they need some URL editing
- the site might even have a debugger detector
- key rotation mechanism
- combine all the previous ones with the fact that you can have multiple keys and multiple IVs for different parts of the m3u8

Edit: This comment is meant just to add stuff that you didn't encounter in your example. Again, nice tutorial

[white_snake]

Nice tutorial and thanks Pepe for the addendum.

[mister_ nex]

dronem

Thanks for your informative tutorial, but I have some questions.
Your command to convert to Hex can only be used in Linux.
I wonder if there are similar commands for Windows cmd ?

I used an online converter,the result is two digits shorter.

https://www.duplichecker.com/text-to-hex.php

7333506138442f4c73305062566a5872684a7a5270773d3d0a  your code 

7333506138442f4c73305062566a5872684a7a5270773d3d    online converter

But it is also suitable for hlsdl and everything downloads perfectly.

[Attachment 78096 - Click to enlarge]

P.S
Let me remind you of [ss]vegeta's several old posts on this topic

https://forum.videohelp.com/threads/405890-how-to-download-this-video#post2657896
https://forum.videohelp.com/threads/405471-Can-someone-download-this-(Smithsonian)#post2654461
https://forum.videohelp.com/threads/405890-how-to-download-this-video#post2657887
https://forum.videohelp.com/threads/405216-Help-on-how-to-download-from-THIS-WEBSITE#post2659712

[lomero]

from HEX editor >> 50 digits. and that's enough


[Attachment 78097 - Click to enlarge]

[mister_ nex]

lomero
Did you open the file downloaded from

https://media.publit.io/hls/eyJpdiI6InA4OEpOTTRYbTN3VUpMdHF0WHkwM0E9PSIsInZhbHVlIjoiSkhHVzRuMUF5alRUdXVwZlJENXB3UT09IiwibWFjIjoiYWY4MjE2ODdhNDc4Y2EzNWJkNDE2MjY1OTIxYTg2N2JiODgxNWM0NzBjNWNlNDhjYTEzMWQ2OThmYTgwY2Y1NiJ9.key?eh=true

in a hex editor?

[lomero]

yep.

anyway the point is: 50 digits is enough. but weird, for my knowledge aes key is usually 32 digits
but well for this link work and that's it

[jack_666]

lomero is correct

for my knowledge aes key is usually 32 digits

What the OP did was to do a direct convert of the key s3Pa8D/Ls0PbVjXrhJzRpw== to hex


73 33 ..... 3d3d
s 3 ..... = =


the additional 0a is the LF (Line feed) character which is not part of the key but is introduced by the od.exe convert program.


The correct procedure is to covert the base 64 to hex


b64(s3Pa8D/Ls0PbVjXrhJzRpw==) => b373daf03fcbb343db5635eb849cd1a7 and this is 32 digits

[jack_666]

used this code to convert from b64 to hex in windows cmd

echo s3Pa8D/Ls0PbVjXrhJzRpw== | openssl  base64 -A -d | xxd -ps

b373daf03fcbb343db5635eb849cd1a7

[lomero]

thanks jack, nice & interesting post

[mister_ nex]

yep.

anyway the point is: 50 digits is enough. but weird, for my knowledge aes key is usually 32 digits
but well for this link work and that's it

This converter shows the standard version of the key


[Attachment 78107 - Click to enlarge]


[Attachment 78108 - Click to enlarge]

Nice short tutorial and pretty nice explanations. You kinda hit an easy video though. I suppose most hls aes 128 videos are like this. However, there are tougher ones that use tricks like this to stop you

Indeed, here is one of the complex sites with AES-128 - each file has its own key.

https://www.theoplayer.com/theoplayer-drm-aes-128-encryption

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOWCACHE:1
#EXT-X-KEY:METHOD=AES-128,URI="segment-00000.key"
#EXTINF:4.458667,
segment-00000.ts.enc
#EXT-X-KEY:METHOD=AES-128,URI="segment-00001.key"
#EXTINF:4.010000,
segment-00001.ts.enc
#EXT-X-KEY:METHOD=AES-128,URI="segment-00002.key"
#EXTINF:4.468667,
segment-00002.ts.enc
#EXT-X-KEY:METHOD=AES-128,URI="segment-00003.key"
#EXTINF:3.893000,
segment-00003.ts.enc

jack_666

Thank you for the explanation.Everything works fine

>echo s3Pa8D/Ls0PbVjXrhJzRpw== | openssl  base64 -A -d | xxd -ps
b373daf03fcbb343db5635eb849cd1a7

[Funx]

which version of hlsdl did you use?, can anyone upload it?

[aqzs]

which version of hlsdl did you use?, can anyone upload it?

https://github.com/selsta/hlsdl
https://github.com/canhlinh/hlsdl

Windows build available here : https://github.com/canhlinh/hlsdl/releases/

[metome]

Help me get the key of this link
https://www.vtvcab.vn/channel/thai-binh-1,THAIBINH_SD.html

m3u
https://live-on-v2-akm.akamaized.net/manifest/thai-binh/master.m3u8?drmv2=1&manifestfi...height%3A1-720
https://live-on-v2-akm.akamaized.net/manifest/thai-binh/playlist_360p.m3u8

key url
https://license.sigmadrm.com/license/verify/sigma?keyId=4b2b43f3-c197-474b-bab1-353b56...f-1248ab44f6e5
EM�I����Q�������$i����u��q��|U4�*����@_7*��� 눝°�y�OہA��"
�s�K�����k+M}W�V��#?6��� �$��
��}�zw�X�o�z�=��e��w2�s�GQ�W�҃|�U�@��I��땓� ��L���{�y�b
�M�<��(3���o�۬.l�#P?�����ȗ���y�F���pD<�G*d�R���!4|dA-�{��G�q`7J�
��7��%� �f �A�D�0&u�ٸ8��V�P��]�Ƶ�*��+�%%Ǣ�h�"�J�I7V!=�:ݧ�8O�
>�%�����c*v'� c������:��yT��(�4��X[#�]~u��Hp�me�l`�xqdog}���ލ?�6��q��`�Oб�.8����;��N{#�_`K[JJp���,�ȴԛCͩ&k�*K���#B;�7b˄�ƨӮr�H�E]���ד�'�\�IO��,T�㘸)�3i���ճ/.��0P����oq?�c�/B#(c|N�n��܀��9v����(<�w�)wS*�4?T�XF9��5�q" ���#��, Z�
fvth�K��kK'�t:�&�.0��]�ζE����#dgx��9PH���Б�P&j��Ԏ^}X!��� 5�Gׁ&_�}���|^&��)Ǿ$�3z_$���(?$Iث�C��
�� %\�T��p�6�¯0��ǣ������Q=<Ï�CDMG��1�B��<ٝB_̧Td��j� k_�ꂛ����nB� i� �*�{����Ќ�}�>

[lomero]

paste here the key like EM�I����Q������� it's really useless
and the key link above sigmadrm.com isn't available or expired or forbidden to download
so, download yourself key file and upload here (or some server)

anyway more trouble with vn sites ... but well, share the key here and show us

[metome]

@lomero :here you are

[metome]

body key