September 29, 2025October 30, 2025

Caution: Heavy Load. Large File Copies With BITS

Ok. Check this out… Have you ever had to copy large files, or a large amount of files from machine A to machine B? No? Do you even work in IT?

For this entry, we are leveraging BITS (Background Intelligent Transfer Service) to copy terrabytes worth of data between off-domain boxes. Why are these off domain devices? That’s classified. I could tell you, but then I’d have to kill you.

The full script can be found here, but stay a while! This is a two parter, and you don’t want to miss part 1 because its not on GitHub.

Part 1 | Secured Credentials

In order to transfer between off-domain devices, we need the creds of an authenticated user on the box. If you’re lazy and doing an SMB copy, you’re typically prompted with UAC to access the other machine. But, we’re using automation here, so work harder and think smarter!

We’re going to encapsulate these creds. Don’t ask me how this works, I’m always paranoid about encrypting creds with PowerShell, so my hand dandy friend, ChatGPT, helped me with this part.

First you’ll run this:

# Path to store the AES key (protect this file!)
$keyPath = "C:\temp\aesKey.bin"

# Create a new 256-bit AES key
$aesKey = New-Object byte[] 32
[Security.Cryptography.RNGCryptoServiceProvider]::Create().GetBytes($aesKey)

# Save the key as binary
Set-Content -Path $keyPath -Value $aesKey -Encoding Byte

P.S. Change the $keyPath variable to something of your choosing.

Then we’ll run this:

# Enter username and password interactively
$user = Read-Host "Enter username (DOMAIN\User)"
$securePwd = Read-Host "Enter password" -AsSecureString

# Encrypt password using the AES key
$key = Get-Content "C:\temp\aesKey.bin" -Encoding Byte
$encryptedPwd = $securePwd | ConvertFrom-SecureString -Key $key

# Store in JSON file
@{
    User = $user
    EncryptedPassword = $encryptedPwd
} | ConvertTo-Json | Set-Content "C:\temp\portableCred.json"

P.S. Match the path in Get-Content to your AES key, and also modify the path in the ending Set-Content cmdlet to a path of your choosing.

Cool. Now you should have the keys to the castle. Protect it with your life!

Now, the gloves come off. No more AI assistance. Let’s crank this script out by hand; thanks to caffeine and loud music.

Part 2 | Copy Mania

Define some parameters. Define a function. Blah. Blah. Blah.

param(
    [parameter(Mandatory=$true)]
        [string]$Server,
    [parameter(Mandatory=$true)]
        [string]$Repo,
    [parameter(Mandatory=$true)]
        [string]$Log,
    [parameter(Mandatory=$true)]
        [string]$AESKey,
    [parameter(Mandatory=$true)]
        [string]$JSONKey

)

function Write-Log {
    param(
        [string]$Message,
        [string]$Level = "Info",
        [string]$LogFile = "$LogPath\PPCopy-$(Get-Date -format yyyy-MM-dd).log"
    )
    $Timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    $LogEntry = "$Timestamp [$Level] $Message"
    Add-Content -Path $LogFile -Value $LogEntry -Force

}

Now, let’s get those creds from Part 1 and pump them in.

$key = Get-Content $AESKey -Encoding Byte
$j = Get-Content $JSONKey | ConvertFrom-Json
$securePwd = $j.EncryptedPassword | ConvertTo-SecureString -Key $key
$cred = New-Object System.Management.Automation.PSCredential ($j.User, $securePwd)

Keep up! Now we’re mapping your server from the -Server parameter as a drive.

New-PSDrive  -Name "CopyDrive" -PSProvider FileSystem -Root "$($Server)" -Credential $cred

Next up! Recursively thumb through your -Server and -Repo locations and grab the full file path to individual files within.

$Storagebox = Get-ChildItem -Path $Server -Directory | Select-Object Name, FullName       
$Repository = Get-ChildItem -Path $Repo -Recurse | Sort-Object -Property LastWriteTime | Select-Object Name, FullName, LastWriteTime

Followed up by a big chunk of code to:

Compare the latest files (I have this set to “yesterday” aka one day ago, so change this accordingly)
Create a directory based off of the file names. (example based on my use-case: C:\temp\Ryans.backup.zip becomes a folder named “Ryans”) Configure this line accordingly as well. change the trailing .Split(“.”)[0] to the delimiter and position of your choosing.

Write-Log -Message "Initializing | Finding Files in Repository"
foreach($file in $Repository){
    if($file.LastWriteTime -ge (Get-date).AddDays(-1)){
        Write-Log -Message "Found $($file.Name)"
        $BKFile = $file.Name.Split(".")[0]
        if($Storagebox -eq $null){
            if(Test-Path "$($Server)\$BKFile"){
                Write-Log -Message "$($Server)\$BKFile exists"
            }
            Else {
                Write-Log -Message "Creating directory: $($Server)\$BKFile"
                New-Item "$($Server)\$BKFile" -ItemType Directory -Force
                $Storagebox = Get-ChildItem -Path $Server -Directory | Select-Object Name, FullName   
            }
        }

Before we start moving files, let’s clean up some the existing files. Storage is cheap, but do you really need a copy of a copy of a copy of a 4 month old backup?

        foreach($share in $Storagebox){
            Write-Log -Message "Acting on $($share.Name)"
           $BKShare = $share.Name.Split("\")[0]
            $LegacyFiles = Get-ChildItem -Path $share.FullName
                foreach($oldfile in $LegacyFiles){                    
                    if($oldfile.LastWriteTime -lt $(Get-Date).AddDays(-2)){
                        Write-Log -Message "Removing last copy's files from $($BKShare)"
                        $LegacyFiles | Remove-Item
                    } 
                    Else {
                        Write-Log "No files found to clean up in $($BKShare)"
                    }
                }

Now for the fun part! BITS! Here’s the breakdown: let’s match the full filename to the folder in our -Server specification now that we’ve created the directory, start copying via BITS, and keep tabs on the file copy.

            if($BKFile -match $BKShare){
                Write-Log -Message "Acting on $BKFile"                
                $Job = Start-BitsTransfer -Source $file.FullName -Destination $Share.FullName -Asynchronous
                while (($Job.JobState -eq "Transferring") -or ($Job.JobState -eq "Connecting")) {
                    Start-Sleep -Seconds 15
                    Write-Log -Message "BITS Job $($Job.JobId) Status: $($Job.JobState) Progress: $([math]::Round($Job.BytesTransferred / 1GB, 2)) GB / $([math]::Round($Job.BytesTotal / 1GB, 2)) GB"
                }
                Switch($Job.JobState) {
                    "Transferred"   { Write-Log -Message "BITS Job $($Job.JobId) is complete."; Complete-BitsTransfer -BitsJob $Job }
                    "Error"         { Write-Log -Message "BITS Job $($Job.JobId) errored out with: $($Job.Description)" }
                    "Suspended"     { Write-Log -Message "BITS Job $($Job.JobId) is suspended. Resuming..."; Resume-BitsTransfer -BitsJob $Job -Asynchronous}
                    "Cancelled"     { Write-Log -Message "BITS Job $($Job.JobId) was cancelled." }
                    "TransientError"{ Write-Log -Message "BITS Job $($Job.JobId) hit a transient error. Retrying..."; Resume-BitsTransfer -BitsJob $Job -Asynchronous  }
                    default         { Write-Log -Message "BITS Job $($Job.JobId) ended with unexpected state: $($Job.JobState)" }
                }
                } 
            }
        } 
    Else {
        Write-Log -Message "No recent files found on $($Repo)"
    }

That’s a lot of code to digest! Let’s break it down further since this is the meat-and-potatoes of the script.

We initiate a BITS transfer with the Start-BITSTransfer cmdlet and specify the source and destination of the file transfer. I’m using the -asynchronous flag in order to keep this copy in the background.

The while loop is in play to keep tabs on the copy job. We check in on the copy job every 15 seconds (modify your Start-Sleep command appropriately). Then we monitor and log the transfer states of the BITS transfer job with our switch statement.

Finally, we wrap it up by removing the drive we created a few steps ago, and finalize our log.

Write-Log -Message "Removing Mapped Drive"
Remove-PSDrive -Name "CopyDrive" -PSProvider FileSystem -Force
Write-Log -Message "Done!"

Phew! That’s all, folks!

Give this a shot for all of your large copy needs, and remember:

This project is provided “as is” without any warranty of any kind, express or implied. Use it at your own risk. The authors and contributors are not responsible for any damage, data loss, or other issues that may arise from using this software. You are solely responsible for any actions taken based on this code.