Python module(s) for building web crawler?

[sofakng]

Can anybody point me in the right direction of any python modules I could use for web crawling and extracting mpd files?

Firefox and Chrome have Stream Detector plugins but it seems like I might need to build my own to crawl a website, extract mpd files, and then name them correctly, etc.

[Hammer]

You can risk your account/IP to get banned since the sites(if you don't know how to do it) are enough smart to detect it(no mouse movements etc.) and block it right away. It isn't easy and if it's the first time you program in Python I suggest you to check for similar projects on github, understand what the program doing and then "copy pasting" with the aim to make it work for you. There are some libraries(is this what you meant with modules?) that make possible to you to automate a browser(Edge, Chrome, Firefox etc.) but there is a long list of things that you need to know and to experiment to get a working bot. Good luck and let us know how it goes.

[Quint]

Search for "Selenium".

[Karoolus]

Start playing a video and keep an eye on the developer console. Usually the info for the video you're playing is fetched from an API, along with the MPD URL etc.
Automate the API to loop through all available shows/movies.