[ACCEPTED]-File.Copy vs. Manual FileStream.Write For Copying File-performance

File.Copy was build around CopyFile Win32 function 5 and this function takes lot of attention 4 from MS crew (remember this Vista-related 3 threads about slow copy performance).

Several 2 clues to improve performance of your method:

1. Like many said earlier remove Flush method from your cycle. You do not need it at all.
2. Increasing buffer may help, but only on file-to-file operations, for network shares, or ftp servers this will slow down instead. 60 * 1024 is ideal for network shares, at least before vista. for ftp 32k will be enough in most cases.
3. Help os by providing your caching strategy (in your case sequential reading and writing), use FileStream constructor override with FileOptions parameter (SequentalScan).
4. You can speed up copying by using asynchronous pattern (especially useful for network-to-file cases), but do not use threads for this, instead use overlapped io (BeginRead, EndRead, BeginWrite, EndWrite in .net), and do not forget set Asynchronous option in FileStream constructor (see FileOptions)

Example 1 of asynchronous copy pattern:

int Readed = 0;
IAsyncResult ReadResult;
IAsyncResult WriteResult;

ReadResult = sourceStream.BeginRead(ActiveBuffer, 0, ActiveBuffer.Length, null, null);
do
{
    Readed = sourceStream.EndRead(ReadResult);

    WriteResult = destStream.BeginWrite(ActiveBuffer, 0, Readed, null, null);
    WriteBuffer = ActiveBuffer;

    if (Readed > 0)
    {
      ReadResult = sourceStream.BeginRead(BackBuffer, 0, BackBuffer.Length, null, null);
      BackBuffer = Interlocked.Exchange(ref ActiveBuffer, BackBuffer);
    }

    destStream.EndWrite(WriteResult);
  }
  while (Readed > 0);

Three changes will dramatically improve 2 performance:

1. Increase your buffer size, try 1MB (well -just experiment)
2. After you open your fileStream, call fileStream.SetLength(inStream.Length) to allocate the entire block on disk up front (only works if inStream is seekable)
3. Remove fileStream.Flush() - it is redundant and probably has the single biggest impact on performance as it will block until the flush is complete. The stream will be flushed anyway on dispose.

This seemed about 3-4 times 1 faster in the experiments I tried:

   public static void Copy(System.IO.Stream inStream, string outputFilePath)
    {
        int bufferSize = 1024 * 1024;

        using (FileStream fileStream = new FileStream(outputFilePath, FileMode.OpenOrCreate, FileAccess.Write))
        {
            fileStream.SetLength(inStream.Length);
            int bytesRead = -1;
            byte[] bytes = new byte[bufferSize];

            while ((bytesRead = inStream.Read(bytes, 0, bufferSize)) > 0)
            {
                fileStream.Write(bytes, 0, bytesRead);
            }
       }
    }

Dusting off reflector we can see that File.Copy 2 actually calls the Win32 API:

if (!Win32Native.CopyFile(fullPathInternal, dst, !overwrite))

Which resolves 1 to

[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
internal static extern bool CopyFile(string src, string dst, bool failIfExists);

And here is the documentation for CopyFile

You'll never going to able to beat the operating 16 system at doing something so fundemental 15 with your own code, not even if you crafted 14 it carefully in assembler.

If you need make 13 sure that your operations occur with the 12 best performance AND you want to mix and 11 match various sources then you will need 10 to create a type that describes the resource 9 locations. You then create an API that 8 has functions such as Copy that takes two such 7 types and having examined the descriptions 6 of both chooses the best performing copy 5 mechanism. E.g., having determined that 4 both locations are windows file locations 3 you it would choose File.Copy OR if the 2 source is windows file but the destination 1 is to be HTTP POST it uses a WebRequest.

Try to remove the Flush call, and move it 3 to be outside the loop.

Sometimes the OS 2 knows best when to flush the IO.. It allows 1 it to better use its internal buffers.

Here's a similar answer

How do I copy the contents of one stream to another?

Your main problem 2 is the call to Flush(), that will bind your 1 performance to the speed of the I/O.

Mark Russinovich would be the authority on this.

He wrote 6 on his blog an entry Inside Vista SP1 File Copy Improvements which sums up the Windows 5 state of the art through Vista SP1.

My semi-educated 4 guess would be that File.Copy would be most 3 robust over the greatest number of situations. Of 2 course, that doesn't mean in some specific 1 corner case, your own code might beat it...

One thing that stands out is that you are 13 reading a chunk, writing that chunk, reading 12 another chunk and so on.

Streaming operations 11 are great candidates for multithreading. My 10 guess is that File.Copy implements multithreading.

Try 9 reading in one thread and writing in another 8 thread. You will need to coordinate the 7 threads so that the write thread doesn't 6 start writing away a buffer until the read 5 thread is done filling it up. You can solve 4 this by having two buffers, one that is 3 being read while the other is being written, and 2 a flag that says which buffer is currently 1 being used for which purpose.