mirror of
https://github.com/nestriness/cdc-file-transfer.git
synced 2026-05-01 17:03:07 +03:00
[cdc_rsync] Improve throughput for local copies (#74)
On Windows, fclose() seems to be very expensive for large files, where closing a 1 GB file takes up to 5 seconds. This CL calls fclose() in background threads. This tremendously improves local syncs, e.g. copying a 4.5 GB, 300 files data set takes only 7 seconds instead of 30 seconds. Also increases the buffer size for copying from 16K to 128K (better throughput for local copies), and adds a timestamp to debug and verbose console logs (useful when comparing client and server logs).
This commit is contained in:
@@ -623,7 +623,7 @@ absl::Status CdcRsyncClient::SendMissingFiles() {
|
||||
|
||||
ParallelFileOpener file_opener(&files_, missing_file_indices_);
|
||||
|
||||
constexpr size_t kBufferSize = 16000;
|
||||
constexpr size_t kBufferSize = 128 * 1024;
|
||||
for (uint32_t server_index = 0; server_index < missing_file_indices_.size();
|
||||
++server_index) {
|
||||
uint32_t client_index = missing_file_indices_[server_index];
|
||||
|
||||
Reference in New Issue
Block a user