In a very small experiment that I've performed in #61676 I've observed up to x2 perf gains that were rather specific to ext4 File System (which is the default on Linux?).
As pointed by @tmds in #61676 (comment) large part of the gains was simply undoing the cost of ftruncate.
For File.Copy($srcPath, $destPath, overwrite: true), ftruncate is currently always performed before copying the file. It's not obvious, as it's a side effect of using FileMode.Create here:
|
using SafeFileHandle dst = SafeFileHandle.Open(destFullPath, overwrite ? FileMode.Create : FileMode.CreateNew, |
the actual sys-call is performed here:
|
if ((mode == FileMode.Create || mode == FileMode.Truncate) && !DisableFileLocking) |
|
{ |
|
// Truncate the file now if the file mode requires it. This ensures that the file only will be truncated |
|
// if opened successfully. |
|
if (Interop.Sys.FTruncate(this, 0) < 0) |
It would be great to dig in more into that and see if we could make File.Copy($srcPath, $destPath, overwrite: true) faster by delaying the truncation to happen after file copying ends. I've done some quick experiment and I've observed up to 5x gain for large files on ext4. Another aspect of the experiment would be to measure the gains of file preallocation for cases when new files are being created.
The person who is willing to work on this issue should measure the performance using both ext4 and btrfs file systems and ensure that whatever changes we made don't regress any of the file systems.
In a very small experiment that I've performed in #61676 I've observed up to x2 perf gains that were rather specific to ext4 File System (which is the default on Linux?).
As pointed by @tmds in #61676 (comment) large part of the gains was simply undoing the cost of
ftruncate.For
File.Copy($srcPath, $destPath, overwrite: true),ftruncateis currently always performed before copying the file. It's not obvious, as it's a side effect of usingFileMode.Createhere:runtime/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Unix.cs
Line 24 in c091dc3
the actual sys-call is performed here:
runtime/src/libraries/System.Private.CoreLib/src/Microsoft/Win32/SafeHandles/SafeFileHandle.Unix.cs
Lines 410 to 414 in c091dc3
It would be great to dig in more into that and see if we could make
File.Copy($srcPath, $destPath, overwrite: true)faster by delaying the truncation to happen after file copying ends. I've done some quick experiment and I've observed up to 5x gain for large files on ext4. Another aspect of the experiment would be to measure the gains of file preallocation for cases when new files are being created.The person who is willing to work on this issue should measure the performance using both
ext4andbtrfsfile systems and ensure that whatever changes we made don't regress any of the file systems.