大文件复制期间的进展(复制项和写入进展?)

小开

Not that I'm aware of. I wouldn't recommend using copy-item for this anyway. I don't think it has been designed to be robust like robocopy.exe to support retry which you would want for extremely large file copies over the network.

小开

最佳答案

I haven't heard about progress with Copy-Item. If you don't want to use any external tool, you can experiment with streams. The size of buffer varies, you may try different values (from 2kb to 64kb).

function Copy-File {
param( [string]$from, [string]$to)
$ffile = [io.file]::OpenRead($from)
$tofile = [io.file]::OpenWrite($to)
Write-Progress -Activity "Copying file" -status "$from -> $to" -PercentComplete 0
try {
[byte[]]$buff = new-object byte[] 4096
[long]$total = [int]$count = 0
do {
$count = $ffile.Read($buff, 0, $buff.Length)
$tofile.Write($buff, 0, $count)
$total += $count
if ($total % 1mb -eq 0) {
Write-Progress -Activity "Copying file" -status "$from -> $to" `
-PercentComplete ([long]($total * 100 / $ffile.Length))
}
} while ($count -gt 0)
}
finally {
$ffile.Dispose()
$tofile.Dispose()
Write-Progress -Activity "Copying file" -Status "Ready" -Completed
}
}

小开

Alternativly this option uses the native windows progress bar...

$FOF_CREATEPROGRESSDLG = "&H0&"


$objShell = New-Object -ComObject "Shell.Application"


$objFolder = $objShell.NameSpace($DestLocation)


$objFolder.CopyHere($srcFile, $FOF_CREATEPROGRESSDLG)

小开

I amended the code from stej (which was great, just what i needed!) to use larger buffer, [long] for larger files and used System.Diagnostics.Stopwatch class to track elapsed time and estimate time remaining.

Also added reporting of transfer rate during transfer and outputting overall elapsed time and overall transfer rate.

Using 4MB (4096*1024 bytes) buffer to get better than Win7 native throughput copying from NAS to USB stick on laptop over wifi.

On To-Do list:

add error handling (catch)
handle get-childitem file list as input
nested progress bars when copying multiple files (file x of y, % if total data copied etc)
input parameter for buffer size

Feel free to use/improve :-)

function Copy-File {
param( [string]$from, [string]$to)
$ffile = [io.file]::OpenRead($from)
$tofile = [io.file]::OpenWrite($to)
Write-Progress `
-Activity "Copying file" `
-status ($from.Split("\")|select -last 1) `
-PercentComplete 0
try {
$sw = [System.Diagnostics.Stopwatch]::StartNew();
[byte[]]$buff = new-object byte[] (4096*1024)
[long]$total = [long]$count = 0
do {
$count = $ffile.Read($buff, 0, $buff.Length)
$tofile.Write($buff, 0, $count)
$total += $count
[int]$pctcomp = ([int]($total/$ffile.Length* 100));
[int]$secselapsed = [int]($sw.elapsedmilliseconds.ToString())/1000;
if ( $secselapsed -ne 0 ) {
[single]$xferrate = (($total/$secselapsed)/1mb);
} else {
[single]$xferrate = 0.0
}
if ($total % 1mb -eq 0) {
if($pctcomp -gt 0)`
{[int]$secsleft = ((($secselapsed/$pctcomp)* 100)-$secselapsed);
} else {
[int]$secsleft = 0};
Write-Progress `
-Activity ($pctcomp.ToString() + "% Copying file @ " + "{0:n2}" -f $xferrate + " MB/s")`
-status ($from.Split("\")|select -last 1) `
-PercentComplete $pctcomp `
-SecondsRemaining $secsleft;
}
} while ($count -gt 0)
$sw.Stop();
$sw.Reset();
}
finally {
write-host (($from.Split("\")|select -last 1) + `
" copied in " + $secselapsed + " seconds at " + `
"{0:n2}" -f [int](($ffile.length/$secselapsed)/1mb) + " MB/s.");
$ffile.Close();
$tofile.Close();
}
}

小开

It seems like a much better solution to just use BitsTransfer, it seems to come OOTB on most Windows machines with PowerShell 2.0 or greater.

Import-Module BitsTransfer
Start-BitsTransfer -Source $Source -Destination $Destination -Description "Backup" -DisplayName "Backup"

小开

cmd /c copy /z src dest

not pure PowerShell, but executable in PowerShell and it displays progress in percents

小开

Trevor Sullivan has a write-up on how to add a command called Copy-ItemWithProgress to PowerShell on Robocopy.

小开

This recursive function copies files and directories recursively from source path to destination path If file already exists on destination path, it copies them only with newer files.

Function Copy-FilesBitsTransfer(
[Parameter(Mandatory=$true)][String]$sourcePath,
[Parameter(Mandatory=$true)][String]$destinationPath,
[Parameter(Mandatory=$false)][bool]$createRootDirectory = $true)
{
$item = Get-Item $sourcePath
$itemName = Split-Path $sourcePath -leaf
if (!$item.PSIsContainer){ #Item Is a file


$clientFileTime = Get-Item $sourcePath | select LastWriteTime -ExpandProperty LastWriteTime


if (!(Test-Path -Path $destinationPath\$itemName)){
Start-BitsTransfer -Source $sourcePath -Destination $destinationPath -Description "$sourcePath >> $destinationPath" -DisplayName "Copy Template file" -Confirm:$false
if (!$?){
return $false
}
}
else{
$serverFileTime = Get-Item $destinationPath\$itemName | select LastWriteTime -ExpandProperty LastWriteTime


if ($serverFileTime -lt $clientFileTime)
{
Start-BitsTransfer -Source $sourcePath -Destination $destinationPath -Description "$sourcePath >> $destinationPath" -DisplayName "Copy Template file" -Confirm:$false
if (!$?){
return $false
}
}
}
}
else{ #Item Is a directory
if ($createRootDirectory){
$destinationPath = "$destinationPath\$itemName"
if (!(Test-Path -Path $destinationPath -PathType Container)){
if (Test-Path -Path $destinationPath -PathType Leaf){ #In case item is a file, delete it.
Remove-Item -Path $destinationPath
}


New-Item -ItemType Directory $destinationPath | Out-Null
if (!$?){
return $false
}


}
}
Foreach ($fileOrDirectory in (Get-Item -Path "$sourcePath\*"))
{
$status = Copy-FilesBitsTransfer $fileOrDirectory $destinationPath $true
if (!$status){
return $false
}
}
}


return $true
}

小开

Sean Kearney from the Hey, Scripting Guy! Blog has a solution I found works pretty nicely.

Function Copy-WithProgress
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true,
ValueFromPipelineByPropertyName=$true,
Position=0)]
$Source,
[Parameter(Mandatory=$true,
ValueFromPipelineByPropertyName=$true,
Position=0)]
$Destination
)


$Source=$Source.tolower()
$Filelist=Get-Childitem "$Source" –Recurse
$Total=$Filelist.count
$Position=0


foreach ($File in $Filelist)
{
$Filename=$File.Fullname.tolower().replace($Source,'')
$DestinationFile=($Destination+$Filename)
Write-Progress -Activity "Copying data from '$source' to '$Destination'" -Status "Copying File $Filename" -PercentComplete (($Position/$total)*100)
Copy-Item $File.FullName -Destination $DestinationFile
$Position++
}
}

Then to use it:

Copy-WithProgress -Source $src -Destination $dest

小开

Hate to be the one to bump an old subject, but I found this post extremely useful. After running performance tests on the snippets by stej and it's refinement by Graham Gold, plus the BITS suggestion by Nacht, I have decuded that:

I really liked Graham's command with time estimations and speed readings.
I also really liked the significant speed increase of using BITS as my transfer method.

Faced with the decision between the two... I found that Start-BitsTransfer supported Asynchronous mode. So here is the result of my merging the two.

function Copy-File {
# ref: https://stackoverflow.com/a/55527732/3626361
param([string]$From, [string]$To)


try {
$job = Start-BitsTransfer -Source $From -Destination $To `
-Description "Moving: $From => $To" `
-DisplayName "Backup" -Asynchronous


# Start stopwatch
$sw = [System.Diagnostics.Stopwatch]::StartNew()
Write-Progress -Activity "Connecting..."


while ($job.JobState.ToString() -ne "Transferred") {
switch ($job.JobState.ToString()) {
"Connecting" {
break
}
"Transferring" {
$pctcomp = ($job.BytesTransferred / $job.BytesTotal) * 100
$elapsed = ($sw.elapsedmilliseconds.ToString()) / 1000


if ($elapsed -eq 0) {
$xferrate = 0.0
}
else {
$xferrate = (($job.BytesTransferred / $elapsed) / 1mb);
}


if ($job.BytesTransferred % 1mb -eq 0) {
if ($pctcomp -gt 0) {
$secsleft = ((($elapsed / $pctcomp) * 100) - $elapsed)
}
else {
$secsleft = 0
}


Write-Progress -Activity ("Copying file '" + ($From.Split("\") | Select-Object -last 1) + "' @ " + "{0:n2}" -f $xferrate + "MB/s") `
-PercentComplete $pctcomp `
-SecondsRemaining $secsleft
}
break
}
"Transferred" {
break
}
Default {
throw $job.JobState.ToString() + " unexpected BITS state."
}
}
}


$sw.Stop()
$sw.Reset()
}
finally {
Complete-BitsTransfer -BitsJob $job
Write-Progress -Activity "Completed" -Completed
}
}

小开

i found none of the examples above met my needs, i wanted to copy a directory with sub directories, the problem is my source directory had too many files so i quickly hit the BITS file limit (i had > 1500 file) also the total directory size was quite large.

i found a function using robocopy that was a good starting point at https://keithga.wordpress.com/2014/06/23/copy-itemwithprogress/, however i found it wasn't quite robust enough, it didn't handle trailing slashes, spaces gracefully and did not stop the copy when the script was halted.

Here is my refined version:

function Copy-ItemWithProgress
{
<#
.SYNOPSIS
RoboCopy with PowerShell progress.


.DESCRIPTION
Performs file copy with RoboCopy. Output from RoboCopy is captured,
parsed, and returned as Powershell native status and progress.


.PARAMETER Source
Directory to copy files from, this should not contain trailing slashes


.PARAMETER Destination
DIrectory to copy files to, this should not contain trailing slahes


.PARAMETER FilesToCopy
A wildcard expresion of which files to copy, defaults to *.*


.PARAMETER RobocopyArgs
List of arguments passed directly to Robocopy.
Must not conflict with defaults: /ndl /TEE /Bytes /NC /nfl /Log


.PARAMETER ProgressID
When specified (>=0) will use this identifier for the progress bar


.PARAMETER ParentProgressID
When specified (>= 0) will use this identifier as the parent ID for progress bars
so that they appear nested which allows for usage in more complex scripts.


.OUTPUTS
Returns an object with the status of final copy.
REMINDER: Any error level below 8 can be considered a success by RoboCopy.


.EXAMPLE
C:\PS> .\Copy-ItemWithProgress c:\Src d:\Dest


Copy the contents of the c:\Src directory to a directory d:\Dest
Without the /e or /mir switch, only files from the root of c:\src are copied.


.EXAMPLE
C:\PS> .\Copy-ItemWithProgress '"c:\Src Files"' d:\Dest /mir /xf *.log -Verbose


Copy the contents of the 'c:\Name with Space' directory to a directory d:\Dest
/mir and /XF parameters are passed to robocopy, and script is run verbose


.LINK
https://keithga.wordpress.com/2014/06/23/copy-itemwithprogress


.NOTES
By Keith S. Garner (KeithGa@KeithGa.com) - 6/23/2014
With inspiration by Trevor Sullivan @pcgeek86
Tweaked by Justin Marshall - 02/20/2020


#>


[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Source,
[Parameter(Mandatory=$true)]
[string]$Destination,
[Parameter(Mandatory=$false)]
[string]$FilesToCopy="*.*",
[Parameter(Mandatory = $true,ValueFromRemainingArguments=$true)]
[string[]] $RobocopyArgs,
[int]$ParentProgressID=-1,
[int]$ProgressID=-1
)


#handle spaces and trailing slashes
$SourceDir = '"{0}"' -f ($Source -replace "\\+$","")
$TargetDir = '"{0}"' -f ($Destination -replace "\\+$","")




$ScanLog  = [IO.Path]::GetTempFileName()
$RoboLog  = [IO.Path]::GetTempFileName()
$ScanArgs = @($SourceDir,$TargetDir,$FilesToCopy) + $RobocopyArgs + "/ndl /TEE /bytes /Log:$ScanLog /nfl /L".Split(" ")
$RoboArgs = @($SourceDir,$TargetDir,$FilesToCopy) + $RobocopyArgs + "/ndl /TEE /bytes /Log:$RoboLog /NC".Split(" ")


# Launch Robocopy Processes
write-verbose ("Robocopy Scan:`n" + ($ScanArgs -join " "))
write-verbose ("Robocopy Full:`n" + ($RoboArgs -join " "))
$ScanRun = start-process robocopy -PassThru -WindowStyle Hidden -ArgumentList $ScanArgs
try
{
$RoboRun = start-process robocopy -PassThru -WindowStyle Hidden -ArgumentList $RoboArgs
try
{
# Parse Robocopy "Scan" pass
$ScanRun.WaitForExit()
$LogData = get-content $ScanLog
if ($ScanRun.ExitCode -ge 8)
{
$LogData|out-string|Write-Error
throw "Robocopy $($ScanRun.ExitCode)"
}
$FileSize = [regex]::Match($LogData[-4],".+:\s+(\d+)\s+(\d+)").Groups[2].Value
write-verbose ("Robocopy Bytes: $FileSize `n" +($LogData -join "`n"))
#determine progress parameters
$ProgressParms=@{}
if ($ParentProgressID -ge 0) {
$ProgressParms['ParentID']=$ParentProgressID
}
if ($ProgressID -ge 0) {
$ProgressParms['ID']=$ProgressID
} else {
$ProgressParms['ID']=$RoboRun.Id
}
# Monitor Full RoboCopy
while (!$RoboRun.HasExited)
{
$LogData = get-content $RoboLog
$Files = $LogData -match "^\s*(\d+)\s+(\S+)"
if ($null -ne $Files )
{
$copied = ($Files[0..($Files.Length-2)] | ForEach-Object {$_.Split("`t")[-2]} | Measure-Object -sum).Sum
if ($LogData[-1] -match "(100|\d?\d\.\d)\%")
{
write-progress Copy -ParentID $ProgressParms['ID'] -percentComplete $LogData[-1].Trim("% `t") $LogData[-1]
$Copied += $Files[-1].Split("`t")[-2] /100 * ($LogData[-1].Trim("% `t"))
}
else
{
write-progress Copy -ParentID $ProgressParms['ID'] -Complete
}
write-progress ROBOCOPY  -PercentComplete ($Copied/$FileSize*100) $Files[-1].Split("`t")[-1] @ProgressParms
}
}
} finally {
if (!$RoboRun.HasExited) {Write-Warning "Terminating copy process with ID $($RoboRun.Id)..."; $RoboRun.Kill() ; }
$RoboRun.WaitForExit()
# Parse full RoboCopy pass results, and cleanup
(get-content $RoboLog)[-11..-2] | out-string | Write-Verbose
remove-item $RoboLog
write-output ([PSCustomObject]@{ ExitCode = $RoboRun.ExitCode })


}
} finally {
if (!$ScanRun.HasExited) {Write-Warning "Terminating scan process with ID $($ScanRun.Id)..."; $ScanRun.Kill() }
$ScanRun.WaitForExit()


remove-item $ScanLog
}
}