github-mirrors/netris-cdc-file-transfer
1
0
Fork 0
mirror of https://github.com/nestriness/cdc-file-transfer.git synced 2026-01-30 14:35:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

[cdc_rsync] Improve throughput for local copies (#74)

Browse Source 
On Windows, fclose() seems to be very expensive for large files, where
closing a 1 GB file takes up to 5 seconds. This CL calls fclose() in
background threads. This tremendously improves local syncs, e.g.
copying a 4.5 GB, 300 files data set takes only 7 seconds instead of
30 seconds.

Also increases the buffer size for copying from 16K to 128K (better
throughput for local copies), and adds a timestamp to debug and
verbose console logs (useful when comparing client and server logs).
This commit is contained in:
Lutz Justen
2023-01-31 16:33:03 +01:00
committed by GitHub
parent 1200b34316
commit 5a909bb443
9 changed files with 275 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
integration_tests/cdc_rsync/output_test.py
View File

@@ -115,7 +115,7 @@ class OutputTest(test_base.CdcRsyncTest):
# server-side output
self._assert_regex(
r'DEBUG server_socket\.cc\([0-9]+\): Receive\(\): EOF\(\) detected',
r'DEBUG \d+\.\d{3} server_socket\.cc\([0-9]+\): Receive\(\): EOF\(\) detected',
output)
# TODO: Add a check here, as currently the output is misleading
@@ -139,7 +139,7 @@ class OutputTest(test_base.CdcRsyncTest):
# server-side output
self._assert_regex(
r'VERBOSE message_pump\.cc\([0-9]+\): ThreadDoReceivePacket\(\): Received packet of size',
r'VERBOSE \d+\.\d{3} message_pump\.cc\([0-9]+\): ThreadDoReceivePacket\(\): Received packet of size',
output)
def test_quiet(self):