How to copy large files over extremely slow and utilised networks (Without BITS)

Tools required

7za.exe – Standalone 7z command-line tool
RoboCopy.exe – Robust copying tool from MS Resource Kit
PSExec.exe – Remote process execution tool from Sysinternals

Steps

1. Compress and span data that you wish to transmit. Compressing obviously reduces the size of data that needs to be transferred and spanning data over a number of small files means that the checkpoint for each copy is reached easier and fewer bits are retransmitted.

7za.exe a -v102400 [ARCHIVENAME].7z “[PATHTOCOMPRESS]”

2. RoboCopy spanned archives to target host The /IPG interpacket gap setting of a 100 seems to reduce the amount of recopies. RoboCopy is a good tool for copying files in general but it still couldn’t transfer the large files that where required without breaking it into 100KB chunks.

RoboCopy.exe “\[TARGETHOST]C$Temp” *.7z.* /zb /w:3 /ipg:100

3. RoboCopy.exe 7za.exe to target host. This standalone is required to extract spanned archive.

RoboCopy .exe “\[TARGETHOST]C$Windows” 7za.exe /zb /w:3 /ipg:100

4. Execute 7za.exe remotely with PSExec.exe.

Psexec.exe \ [TARGETHOST] -d 7za.exe e C:Temp[ARCHIVENAME].7z.001 -aoa -o”PATHTODECOMPRESS”

Automated in a batch

[sourcecode language=”plain”]
@Echo Off
del *.7z.*
7za a -v102400 %2.7z "%3"
robocopy . "\%1c$Temp" *.7z.* /zb /w:3 /ipg:100
robocopy . "\%1c$Windows" 7za.exe /zb /w:3 /ipg:100
psexec \%1 -d 7za e C:Temp%2.7z.001 -aoa -o"%4"
</span></em>
[/sourcecode]

Usage: CopyData.bat HOSTNAME ARCHIVENAME SOURCEFOLDER DESTINATIONFOLDER