SSH batch file???

[AlecWest]

I'm not even sure if this is possible. But if it is possible, I'd be interested in knowing the procedure.

I have SSH access on two remote servers. Occasionally, I need to move large numbers of files between them. I do this using the "wget" command. Problem? I have to sit in front of a computer to type in each "wget" command line. What I'd like to do (if possible) is create a text file in the directory of my second server where I want to copy files to that includes all the command lines ... like this:

wget http://serverone.com/file1.xxx
wget http://serverone.com/file2.xxx
wget http://serverone.com/file3.xxx
etc., etc., etc.

Then, I'd like to execute that text file as a batch file. Doable? If so, what would the command line look like to execute that batch file?

Regards,
J. Alec West

[Baldrick]

Windows or linux?

[AlecWest]

Windows or linux?

Linux. And BTW, neither server if FXP-compatible. I tried FlashFXP first thing. No go. That leaves me with no other option except entering in and executing "wget" commands, one command line at a time, unless there's a batch option I'm unaware of.

Regards,
J. Alec West

[dLee]

In the Linux world, the command line is called the shell. The shell is called "BASH". Batch files are called shell scripts. You can probably use Perl, too.

You can easily transfer files between servers via SSH utilizing shell scripts.

There are many shell scripts you can find with Google, although most are oldish and somewhat limited. Try "BASH scripting" or "Linux shell scripts" to search for examples.

The Ubuntu forums, wiki's, man and help sites have good info that can easily be adapted to your distro.

Some O'Reilly books that have proven valuable to me are:

Linux Network Administrator's Guide
Linux Server Hacks
Linux in a Nutshell

[disturbed1]

#!/bin/sh
FILES=" \
file1.foo \
file2.foo \
file3.foo \
--- \
file876876.foo";

# Allow for specification of individual file to be downloaded:
if [ -z "$1" ]; then
GETFILE=$FILES
else
GETFILE=$*
fi

for file in \
$GETFILE ;
do
wget http://www.someserver.com/$file ;
done

Probably more complicated then it needs to be, and most likely needs proofread as well

Two other simple ways -

wget http://server.com/file.foo && http://server.com/file2.foo && http://server.com/file3.foo ............

You can also use the mirror and accept list options of wget

wget -m --accept=rar,avi,zip,tgz,tar.gz,tar.bz2,foo,bar,blah http://server.com/

Wget has a few options. While logged into your server with putty, issue one of these commands -
wget --help
man wget


----------edit------------

If you just wanted to create a Linux batch file. Linux batch files are commonly called scripts or shell scripts. Bash is an entire shell - stands for Bourne Again Shell, there are many different shells. The SHELL is NOT called BASH! Bash is only one of many shell types

Once logged into your Linux server -
touch download.sh
(creates an empty file in the current directory called download.sh)
nano download.sh
(Nano is a simple text editor)
Type this inside the nano screen

#!/bin/sh
wget http://serverone.com/file1.xxx
wget http://serverone.com/file2.xxx
wget http://serverone.com/file3.xxx
etc., etc., etc.

Hit ^x (ctrl-x) to exit - look at the prompt at the bottom of the screen

Either -
sh download.sh
or
chmod +x download.sh
./download.sh

[AlecWest]

Thanks to all. This is terra incognita stuff to me. But, it appears very intuitive and very user friendly. I'll be moving a bunch of files in a couple of days and will try a few things then ... and will report on what happens when I do (grin).

Regards,
J. Alec West

[stiltman]

Doesn't it support wild cards?

#!/bin/sh
wget http://serverone.com/*.*

That would seem to be a lot easier

[AlecWest]

Doesn't it support wild cards?

#!/bin/sh
wget http://serverone.com/*.*

That would seem to be a lot easier

It might support wildcards though I didn't try it. In my case, I didn't want to move all files ... just some.

#!/bin/sh
wget http://serverone.com/file1.xxx
wget http://serverone.com/file2.xxx
wget http://serverone.com/file3.xxx
etc., etc., etc.

In trying this method, the batch file worked as a batch file should. However, it gave me FILE NOT FOUND errors. And, I noticed that the reason why is because the batch file added the "%0D" characters to the end of each filename. Fortunately, I had a "stream of consciousness" moment (grin) and said to myself, "Hmmm, I wonder what would happen if I put a semi-colon at the end of each filename - like this:

#!/bin/sh
wget http://serverone.com/file1.xxx;
wget http://serverone.com/file2.xxx;
wget http://serverone.com/file3.xxx;

That worked like a Swiss watch and batch-transfered 10 files (YAYYYY!!!).

Thank you for that VERY useful suggestion ... even though I had to "modify" it somewhat.

Regards,
J. Alec West

[AlecWest]

Just a followup. Using the semi-colon at the end of the command line (after the filename) works fine in the script. However, after each command is executed, I notice a:

"; command not found"

error message. It doesn't affect the transfers at all. But, it does indicate that there's a "proper" way to end a command line. Anybody know what that proper way might be?

Regards,
J. Alec West

[disturbed1]

The ; is not needed.

Did you create a text file, and write the commands in the text file, then execute it?

I just created this text file (shell script) then executed with success

#!/bin/sh
wget www.server.com/file1.xxx
wget www.server.com/file2.xxx
wget www.server.com/file3.xxx

If you want a single command to use, rather than created a shell script and defining every file name -

wget --accept=exe,rar,zip -r www.server.com/

Or -
for file in file1.xxx file2.xxx file3.xxx;do wget www.server.com/$file;done

^^ If $file has spaces or special characters {The Super File.zip} you need to pass www.server.com/"$file" <-- notice the quotes

[AlecWest]

The ; is not needed.

Did you create a text file, and write the commands in the text file, then execute it?

I just created this text file (shell script) then executed with success

#!/bin/sh
wget www.server.com/file1.xxx
wget www.server.com/file2.xxx
wget www.server.com/file3.xxx

I didn't think the semi-colon was needed either. But when I did things the way you said, I got those special characters after each filename and a "file not found" error - every time. I'm not sure what "brand" of Linux my virtual server is set up on. But, if the script as you did it works for you and not for me, that may be the culprit - the "type" of Linux. Or, it could be some internal snafu on the server that I can't fathom (sigh).

In any case, the semi-colon inclusion, despite the error message, works for me ... so, I suppose I'll continue to use it.

Regards,
J. Alec West

[Steve Stepoway]

The character 0D hex is a carriage return. Sounds like the editor used the DOS convention of ending lines with CR/LF, instead of the Unix convention of just a Line Feed.

Steve