curl multi-line

28th Feb 2024 02:45 #1
whs912km

View Profile

View Forum Posts

Private Message
Member

Join Date
Nov 2021
i need to run curl command, multiline like this one:

Code:

curl -s "https://link" \ | grep -oP '<img src="\K[^"]*' \ | head -n1

but cmd run each line separately. so, with first line i get all elements from web page, but idk how run right full command
Quote
28th Feb 2024 03:52 #2
october262

View Profile

View Forum Posts

Private Message
Member

Join Date
Feb 2006

Location
United States
Originally Posted by whs912km

i need to run curl command, multiline like this one:

Code:

curl -s "https://link" \ | grep -oP '<img src="\K[^"]*' \ | head -n1

but cmd run each line separately. so, with first line i get all elements from web page, but idk how run right full command

try this - https://stackoverflow.com/questions/32341091/multiline-curl-command
Quote

29th Feb 2024 04:02 #3

Member

thanks for your reply, but same behavior. also with curl only (not cmd/powershell)

my goal is download the file

Code:

https://i1.sndcdn.com/artworks-4KDEzUk5fgjxkLoT-4kINZA-t500x500.jpg

found trough Elements tab:

[Attachment 77338 - Click to enlarge]

here the full command

Code:

curl -s "https://soundcloud.com/defectedrecords/defected-radio-show-most-rated-special-hosted-by-sam-divine-221223" \
| grep -oP '<img src="\K[^"]*' \
| head -n1

unlucky the kind user who gave me this command did not answer any further, and left me in doubt

here my log:

Code:

D:\curl-8.6.0\bin
curl -s "https://soundcloud.com/defectedrecords/defected-radio-show-most-rated-special-hosted-by-sam-divine-221223" \
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="theme-color" content="#333">
<link rel="dns-prefetch" href="//style.sndcdn.com">
<link rel="dns-prefetch" href="//a-v2.sndcdn.com">
<link rel="dns-prefetch" href="//api-v2.soundcloud.com">
<link rel="dns-prefetch" href="//sb.scorecardresearch.com">
<link rel="dns-prefetch" href="//secure.quantserve.com">
<link rel="dns-prefetch" href="//eventlogger.soundcloud.com">
<link rel="dns-prefetch" href="//api.soundcloud.com">
<link rel="dns-prefetch" href="//ssl.google-analytics.com">
<link rel="dns-prefetch" href="//i1.sndcdn.com">

(cut)
......
.......

D:\curl-8.6.0\bin
 | grep -oP '<img src="\K[^"]*' \
| not expected

D:\curl-8.6.0\bin
 | head -n1

Quote

29th Feb 2024 09:15 #4

whs912km

Member

finally i've get right command to rip and print jpg link with:
Code:
curl -s "https://soundcloud.com/defectedrecords/defected-radio-show-most-rated-special-hosted-by-sam-divine-221223" | awk -F'"' '/<img src="https:\/\/i1\.sndcdn\.com\/artworks-.*\.jpg/ {print $2; exit}'
print
Code:
https://i1.sndcdn.com/artworks-4KDEzUk5fgjxkLoT-4kINZA-t500x500.jpg
now i need to automate the full process and download this jpg file. how to ?

Quote

29th Feb 2024 11:02 #5
whs912km

View Profile

View Forum Posts

Private Message
Member

Join Date
Nov 2021
edit command from my first post, get jpg link (removed < before 'image', and put command on a single line)

Code:

curl -s "https://soundcloud.com/defectedrecords/defected-radio-show-most-rated-special-hosted-by-sam-divine-221223" | grep -oP 'img src="\K[^"]*' | head -n1

but still idk how to download ...
Quote

29th Feb 2024 13:12 #6

whs912km

Member

problem solved (thanks to chatGPT). here 4 alternative ways

this command, trough curl, download jpg file with original filename (put link on command above)

Code:

curl -s "https://link" | grep -oP 'img src="\K[^"]*' | tail -n 1 | xargs -n 1 sh -c 'curl -o "$(basename "$0")" "$0"'

or this "artwork.bash" use curl, required link, save with original filename jpg (copy code above, save as artwork.bash and run it from curl folder)

Code:

#!/bin/bash

read -p "Enter the SoundCloud link: " soundcloud_link
curl -s "$soundcloud_link" | grep -oP 'img src="\K[^"]*' | tail -n 1 | xargs -n 1 sh -c 'curl -o "$(basename "$0")" "$0"'

another script use python, with curl, required link, save with original filename jpg

Code:

import subprocess
import os

def extract_image_url(soundcloud_link):
    curl_command = f'curl -s "{soundcloud_link}"'
    curl_output = subprocess.check_output(curl_command, shell=True, text=True)
    
    # Extract the image URL directly in Python
    for line in curl_output.split('\n'):
        if '<meta property="og:image" content="' in line:
            image_url = line.split('<meta property="og:image" content="')[1].split('"')[0]
            return image_url

    raise ValueError("Image URL not found")

def download_image(image_url):
    filename = os.path.basename(image_url)
    subprocess.run(['curl', '-o', filename, image_url])

if __name__ == "__main__":
    soundcloud_link = input("Enter the SoundCloud link: ")
    
    try:
        image_url = extract_image_url(soundcloud_link)
        download_image(image_url)
        print(f"Image downloaded successfully! Saved as: {os.path.basename(image_url)}")
    except Exception as e:
        print(f"Error: {e}")

another command use yt-dlp to download

Code:

import yt_dlp

URL = input('soundcloud link: ')

ydl_opts = {
    'writethumbnail': True,
    'skip_download': True,
}

with yt_dlp.YoutubeDL(ydl_opts) as ydl:
    info = ydl.extract_info(URL, download=False, process=False)
    thumbnails = info.get('thumbnails') or []
    info['thumbnails'] = [t for t in thumbnails if t.get('height') == 500]
    ydl.process_ie_result(info)

Quote

29th Feb 2024 15:26 #7
jack_666

View Profile

View Forum Posts
Member

Join Date
May 2021
use curl -s -o "defected_radio.jpg" to download the image

curl to scrape the soundcloud URL, grep "<img src=.http" to capture the required line, awk "BEGIN{FS=\"\042\"} {print $2}" to parse the line and get the image name and lastly xargs curl -s -o "defected_radio.jpg" to feed the image name to curl and then download the image to current directory.

here is the complete code

Code:

curl -s "https://soundcloud.com/defectedrecords/defected-radio-show-most-rated-special-hosted-by-sam-divine-221223" | grep "<img src=.http" | awk "BEGIN{FS=\"\042\"} {print $2}" | xargs curl -s -o "defected_radio.jpg"
Quote
29th Feb 2024 17:09 #8
whs912km

View Profile

View Forum Posts

Private Message
Member

Join Date
Nov 2021
thank you jack for your advice, but what difference (advantage) from my curl command above ?

Last edited by whs912km; 1st Mar 2024 at 10:56.

Quote
1st Mar 2024 15:20 #9
jack_666

View Profile

View Forum Posts
Member

Join Date
May 2021
what difference (advantage) from my curl command above ?

Advantage ... absolutely none

Difference ... Your code is for linux. The code I provided is for Windows.

I posted because some other person, using windows, may wish to do the the same as you.

I tried to show that piping does not require multiline. Multiline is used when the line of code is very long and you wish to show it confined to a smaller space.

Quote
1st Mar 2024 16:42 #10
whs912km

View Profile

View Forum Posts

Private Message
Member

Join Date
Nov 2021
thanks for your clarification.

however i am win user, i don't use linux ...! and my command above post#6 for curl or also bash file, work great on windows
also my command above (thanks to chatGPT) save file with original filename, without edit manually curl command for -o "defected_radio.jpg"

also bash file ask each time new soundcloud link. this allows to use bash file with different links without edit curl command

Quote
2nd Mar 2024 09:55 #11
jack_666

View Profile

View Forum Posts
Member

Join Date
May 2021
You are absolutely correct. Thanks for the clarification.

Quote

curl multi-line

Thread Tools

Search Thread

Similar Threads

Need curl command to extract m3u8 url

Need curl command to extract m3u8 from live TV website

Curl or Wget command to capture vtt subs?

Divide each line of text into new subtitle line

Multi-line, heavily stylized subtitles (ASS) possible on Blu-ray?