Anyone good with basic scripting for FTP/scheduled events?

[Galant]

Hi Guys,

I've recently taken over a project/client, part of which involves the automated daily transfer of a few files via FTP. Previously this had been done using Filezilla, however the client has been having problems with the automation and I've decided to ditch it and upon the recommendation of Kalniel use WinSCP.

I don't have a lot of experience on the scripting side of things, so although I think I've got the gist of things, I'm getting a little hung up on the nitty-gritty of it.

Since scripts already exist which had been used with Filezilla, I'm wondering if these will work. They're relatively simple, small scripts compared to others I've seen as examples, and so that's confusing me a bit.

There are two existing scripts:

1 - ftptransfer.txt
Login credentials, remote dir setup, multiple put and quit -

<username>
<password>
CD Outbox
ascii
prompt n
mput *.*
quit

2 - transfer.bat
Local Dir setp, Runs FTP based on other script creds, and archives the local files -

cd C:\Sales
ftp -s:C:\Sales\FTP_transfer\ftptransfer <ip address>
move *.* C:\Sales\Archive

Do you think those can be used for the WinSCP script or should I start fresh?


I found this example on the WinSCP site and modified it, does this look correct, do you think it's all necessary?

[Reflection.Assembly]::LoadFrom("WinSCPnet.dll") | Out-Null


$sessionOptions = New-Object WinSCP.SessionOptions
$sessionOptions.Protocol = [WinSCP.Protocol]::ftp
$sessionOptions.HostName = "<IP address>"
$sessionOptions.UserName = "<username>"
$sessionOptions.Password = "<password>"
$sessionOptions.SshHostKeyFingerprint = "ssh-rsa 1024 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"

$session = New-Object WinSCP.Session

$session.Open($sessionOptions)

$localPath = "C:\Sales\"
$remotePath = "/Gibraltar/Outbox"
$backupPath = "C:\Sales\Archive"

$transferResult = $session.PutFiles($localPath, $remotePath)

# Iterate over every transfer
foreach ($transfer in $transferResult.Transfers)
{
# Success or error?
if ($transfer.Error -eq $Null)
{
Write-Host ("Upload of {0} succeeded, moving to backup" -f
$transfer.FileName)
# Upload succeeded, move source file to backup
Move-Item $transfer.FileName $backupPath
}
else
{
Write-Host ("Upload of {0} failed: {1}" -f
$transfer.FileName, $transfer.Error.Message)
}
}
}
finally
{
# Disconnect, clean up
$session.Dispose()
}

exit 0
}
catch [Exception]
{
Write-Host $_.Exception.Message
exit 1
}


Thanks,

G.

[bytejunkie]

i think you're mistaken. script 1 and 2 arent really touching filezilla. they're just using windows inbuilt ftp client which you can use from a cmd prompt by typing ftp or even ftp ip.address

you just need to keep it basic and try and solve the issue you've got, rather than introducing another one.

[Singh400]

Did you at least try and solve the problems with FileZilla first?

[Splash]

As an example of what I use to download a backup of a hosted site using WinSCP: I have a backup.txt file which I keep somewhere locally which has something like the following contents

option batch on
option confirm off
open account@ftp.site.com
get /home/site/backups/backup.tar "D:\Data\backups\"
exit

I then have a scheduled task that runs through the Windows task scheduler that runs

"c:\Program Files (x86)\WinSCP\WinSCP.com" /console /script=c:\backup.txt

Obviously you'd need to tweak your backup.txt file to do what you wanted, and also save the connection details in WinSCP (including password/keys/whatever) - this may be a no-no for security reasons (this is a basic site, and only I have access to the backup server so I'm prepared to accept the risk) in which case you can specify them in the backup.txt as you specified in your original post. Does that help?

[bytejunkie]

As an example of what I use to download a backup of a hosted site using WinSCP:

if you think carefully about it kev, you've also got access to a quite clever (IIRC) Perl script that does some ftping over some pretty lousy connections.
might be useful in this situation if as i recall its got some error checking in. im not sure how you'd error check with the winSCP method.

[Splash]

yep, but again it all depends on requirements. I do regular test restores (to my test server as a data refresh) so I'm reasonably happy with that risk. It's not commercial data, let's put it that way...

Also I hate Perl. (and that perl script isn't quite as reliable as you might hope - we still see corrupt uploads)



Plus the most important part: Galant doesn't have access to that script

[bytejunkie]

perl is only as bad as the code monkey writing it. or rather the non coder writing it since a serious coder would never use it. ha.

Galant might not, but he does know someone who could read it and post some examples based on the lines that matter? maybe.

corrupt uploads must be about the quality of the connections no? it doesn't sound like it error checks though. i thought it did but maybe im just imagining an improvement that someone suggested once.

[Galant]

Hi guys,

Sorry for the delay in responding. Thanks for the input.

Byte - Good catch! My predecessor didn't document his procedures, and since Filezilla was sitting there and we use it for certain manual downloads my brain was stuck thinking about Filezilla. This being the case, it eliminates Filezilla from my troubleshooting concerns. That though, still brings me back to the initial problem, that the same scripting, when I run it myself, works. However, when I leave it to run by itself, it doesn't. That's even the case when I run the batch file by manually running the scheduled event. I cannot figure out why this would be the case. Any ideas?

2 - Splash - That's the sort of thing I was looking for, thanks. I didn't think I'd need all the extra scripting. I'm not too concerned about the security since what I'm already running is what was set up by the parent company and they're happy with it. I suppose, rather than troubleshooting the basic problem, I can attempt to just get the automation up and running by using WinSCP.

I've already set up the connection using WinSCP and saved it. It works fine. All I need to do is to get the scripting in place so that I can set it to run as a scheduled event. Also, I have have some scripting in there to get WinSCP to also move the local files in question once they've been sent, to a different folder
- C:\Sales\Archive

Using your example:

option batch on
option confirm off
open account@ftp.site.com
mput C:\Sales\*.* "<drive>\Gibraltar\Outbox\"
mv C:\Sales\*.* "C:\Sales\Archive"
exit

Would that be correct?


Thanks.

[bytejunkie]

so is it running under an account? is that account still valid? password right?
have you looked in event viewer about the time it failed?

[Splash]

your code should just work, yep. Bear in mind that there's no error handling there, so if there's any reason that you can't upload the files in c:\sales (perhaps a file locked for write) then there's no mechanism to retry that after a specific time: it'll just be moved to the archive directory.

Dependent on the nature of your setup this might just work, or it might not be an acceptable risk: only you can really decide on that I'm afraid. Bytejunkie makes a good point about troubleshooting the previous job though. Are you able to post the script that was previously used (perhaps substituting some dummy paths for anything you're not comfortable posting) as well as the method that it was invoked (I guess a scheduled task)?

[Galant]

Hi guys,

Sorry it's been a while, but I've been playing with a few things as well as having had other priorities.

Especial thanks to you Splash because your little comment about the lack of error handling got me thinking about possible avenues of failure and I found the problem with the original setup. Somehow, in the scheduled event settings, the Windows log-on credentials got deleted. Bit of an obvious step but nevertheless I missed it. Not sure why, without the credentials it managed to run the batch and move the files but couldn't run the ftp, I'll have to look it up, still, the process ran successfully last night.

So thanks for that.

Thanks too for WinSCP as I now have it on hand and might well implement it in the future.

One last question if you can help, guys, any recommendations on some error handling code I can include, if not to try the process again, at least to produce some sort of notification that it failed, and and maybe why?

Maybe create a .txt file or something?

Thanks again!