github-mirrors/netris-cdc-file-transfer
1
0
Fork 0
mirror of https://github.com/nestriness/cdc-file-transfer.git synced 2026-01-30 14:25:36 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
21a1b37787a3a8b467052c8ff98bc2df8116afcb
netris-cdc-file-transfer/common/file_watcher_win_test.cc
chrschng 76bbdb01bb Merge dynamic manifest updates to Github (#7) 
This change introduces dynamic manifest updates to asset streaming.

Asset streaming describes the directory to be streamed in a manifest, which is a proto definition of all content metadata. This information is sufficient to answer `stat` and `readdir` calls in the FUSE layer without additional round-trips to the workstation.

When a directory is streamed for the first time, the corresponding manifest is created in two steps:
1. The directory is traversed recursively and the inode information of all contained files and directories is written to the manifest.
2. The content of all identified files is processed to generate each file's chunk list. This list is part of the definition of a file in the manifest.
  * The chunk boundaries are identified using our implementation of the FastCDC algorithm.
  * The hash of each chunk is calculated using the BLAKE3 hash function.
  * The length and hash of each chunk is appended to the file's chunk list.

Prior to this change, when the user mounted a workstation directory on a client, the asset streaming server pushed an intermediate manifest to the gamelet as soon as step 1 was completed. At this point, the FUSE client started serving the virtual file system and was ready to answer `stat` and `readdir` calls. In case the FUSE client received any call that required file contents, such as `read`, it would block the caller until the server completed step 2 above and pushed the final manifest to the client. This works well for large directories (> 100GB) with a reasonable number of files (< 100k). But when dealing with millions of tiny files, creating the full manifest can take several minutes.

With this change, we introduce dynamic manifest updates. When the FUSE layer receives an `open` or `readdir` request for a file or directory that is incomplete, it sends an RPC to the workstation about what information is missing from the manifest. The workstation identifies the corresponding file chunker or directory scanner tasks and moves them to the front of the queue. As soon as the task is completed, the workstation pushes an updated intermediate manifest to the client which now includes the information to serve the FUSE request. The queued FUSE request is resumed and returns the result to the caller.

While this does not reduce the required time to build the final manifest, it splits up the work into smaller tasks. This allows us to interrupt the current work and prioritize those tasks which are required to handle an incoming request from the client. While this still takes a round-trip to the workstation plus the processing time for the task, an updated manifest is received within a few seconds, which is much better than blocking for several minutes. 

This latency is only visible when serving data while the manifest is still being created. The situation improves as the manifest creation on the workstation progresses. As soon as the final manifest is pushed, all metadata can be served directly without having to wait for pending tasks.
2022-11-16 11:20:32 +01:00

613 lines
22 KiB
C++
Raw Blame History

// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "common/file_watcher_win.h"
#define WIN32_LEAN_AND_MEAN
#include <chrono>
#include <string>
#include <thread>
#include "absl/strings/match.h"
#include "common/buffer.h"
#include "common/log.h"
#include "common/path.h"
#include "common/platform.h"
#include "common/status_test_macros.h"
#include "common/util.h"
#include "gtest/gtest.h"
namespace cdc_ft {
const char* ActionToString(const FileWatcherWin::FileAction& fa) {
switch (fa) {
case FileWatcherWin::FileAction::kAdded:
return "ADDED";
case FileWatcherWin::FileAction::kModified:
return "MODIFIED";
case FileWatcherWin::FileAction::kDeleted:
return "DELETED";
default:
return "UNKNOWN";
}
}
std::ostream& operator<<(std::ostream& os,
const FileWatcherWin::FileMap& files) {
for (const auto& [path, fi] : files) {
os << "path=" << path << ", action=" << ActionToString(fi.action)
<< ", is_dir=" << fi.is_dir << ", mtime=" << fi.mtime
<< ", size=" << fi.size << std::endl;
}
return os;
}
namespace {
constexpr char kWatcherTestDir[] = "watcher_test_dir";
constexpr char kWatcherWatchedDir[] = "watcher_watched_dir";
constexpr char kFirstData[] = {10, 20, 30, 40, 50, 60, 70, 80, 90};
constexpr char kSecondData[] = {100, 101, 102, 103, 104, 105, 106, 107, 108,
100, 101, 102, 103, 104, 105, 106, 107, 108};
constexpr size_t kFirstDataSize = sizeof(kFirstData);
constexpr size_t kSecondDataSize = sizeof(kSecondData);
constexpr char kFirstFile[] = "first_test_file.txt";
constexpr char kSecondFile[] = "second_test_file.txt";
constexpr char kFirstDir[] = "first_test_dir";
constexpr char kSecondDir[] = "second_test_dir";
constexpr bool kFile = false;
constexpr bool kDir = true;
constexpr absl::Duration kWaitTimeout = absl::Seconds(5);
constexpr unsigned int kFWTimeout = 10;
using FileMap = FileWatcherWin::FileMap;
using FileAction = FileWatcherWin::FileAction;
using FileInfo = FileWatcherWin::FileInfo;
class FileWatcherParameterizedTest : public ::testing::TestWithParam<bool> {
public:
FileWatcherParameterizedTest() : watcher_(watcher_dir_path_) {
Log::Initialize(std::make_unique<ConsoleLog>(LogLevel::kInfo));
}
~FileWatcherParameterizedTest() { Log::Shutdown(); }
void SetUp() override {
legacyReadDirectoryChanges_ = GetParam();
if (legacyReadDirectoryChanges_)
watcher_.EnforceLegacyReadDirectoryChangesForTesting();
EXPECT_OK(path::RemoveDirRec(watcher_dir_path_));
EXPECT_OK(path::CreateDirRec(watcher_dir_path_));
}
void TearDown() override { EXPECT_OK(path::RemoveDirRec(watcher_dir_path_)); }
// True if ReadDirectoryChangesW should be enforced (instead of *ExW).
bool legacyReadDirectoryChanges_ = false;
protected:
void OnFilesChanged() {
absl::MutexLock lock(&files_changed_mutex_);
files_changed_ = true;
}
void OnDirRecreated() {
absl::MutexLock lock(&files_changed_mutex_);
dir_recreated_ = true;
}
bool WaitForChange(uint32_t min_event_count = 0) {
absl::MutexLock lock(&files_changed_mutex_);
bool changed = false;
do {
auto cond = [this]() { return files_changed_; };
changed = files_changed_mutex_.AwaitWithTimeout(absl::Condition(&cond),
kWaitTimeout);
files_changed_ = false;
} while (changed && watcher_.GetEventCountForTesting() < min_event_count);
return changed;
}
bool WaitForDirRecreated(uint32_t min_event_count = 0) {
absl::MutexLock lock(&files_changed_mutex_);
bool changed = false;
do {
auto cond = [this]() { return dir_recreated_; };
changed = files_changed_mutex_.AwaitWithTimeout(absl::Condition(&cond),
kWaitTimeout);
dir_recreated_ = false;
} while (changed &&
watcher_.GetDirRecreateEventCountForTesting() < min_event_count);
return changed;
}
FileMap GetChangedFiles(size_t number_of_files) {
FileMap modified_files;
// Wait for events, until they are processed.
while (modified_files.size() < number_of_files) {
EXPECT_TRUE(WaitForChange());
for (const auto& [path, info] : watcher_.GetModifiedFiles())
modified_files.insert_or_assign(path, info);
}
return modified_files;
}
void ExpectFile(const FileMap& modified_files, const std::string& path) {
EXPECT_TRUE(modified_files.find(path) != modified_files.end())
<< path << " is missing from " << std::endl
<< modified_files;
}
void ExpectFileInfo(const FileMap& modified_files, const std::string& path,
FileAction action, bool is_dir, uint64_t size) {
auto iter = modified_files.find(path);
EXPECT_TRUE(iter != modified_files.end())
<< path << " is missing from " << std::endl
<< modified_files;
if (iter != modified_files.end()) {
EXPECT_EQ(iter->second.action, action);
// is_dir and size are not available for the legacy ReadDirectoryChangesW,
// but that data isn't needed, anyway.
if (action != FileAction::kDeleted || !legacyReadDirectoryChanges_) {
EXPECT_EQ(iter->second.is_dir, is_dir);
EXPECT_EQ(iter->second.size, size);
}
// Don't bother checking mtime here, it's checked elsewhere.
}
}
const std::string test_dir_path_ =
path::Join(path::GetTempDir(), kWatcherTestDir);
const std::string watcher_dir_path_ =
path::Join(test_dir_path_, kWatcherWatchedDir);
const std::string first_file_path_ =
path::Join(watcher_dir_path_, kFirstFile);
const std::string second_file_path_ =
path::Join(watcher_dir_path_, kSecondFile);
const std::string first_dir_path_ = path::Join(watcher_dir_path_, kFirstDir);
const std::string second_dir_path_ =
path::Join(watcher_dir_path_, kSecondDir);
FileWatcherWin watcher_;
bool files_changed_ ABSL_GUARDED_BY(files_changed_mutex_) = false;
bool dir_recreated_ ABSL_GUARDED_BY(files_changed_mutex_) = false;
absl::Mutex files_changed_mutex_;
};
TEST_P(FileWatcherParameterizedTest, DirDoesNotExist) {
FileWatcherWin watcher("non-existing folder");
if (legacyReadDirectoryChanges_)
watcher_.EnforceLegacyReadDirectoryChangesForTesting();
EXPECT_NOT_OK(watcher.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_FALSE(watcher.IsWatching());
absl::Status status = watcher.GetStatus();
EXPECT_NOT_OK(status);
EXPECT_TRUE(absl::IsFailedPrecondition(status));
EXPECT_TRUE(absl::StrContains(status.message(), "Could not start watching"));
}
TEST_P(FileWatcherParameterizedTest, CreateFile) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, CreateFileDelayed) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
std::this_thread::sleep_for(std::chrono::milliseconds(20));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, CreateTwoFiles) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(path::WriteFile(second_file_path_, kFirstData, kFirstDataSize));
FileMap modified_files = GetChangedFiles(2u);
EXPECT_EQ(modified_files.size(), 2u);
ExpectFile(modified_files, kFirstFile);
ExpectFile(modified_files, kSecondFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, CreateDir) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::CreateDir(first_dir_path_));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstDir);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RenameDir) {
EXPECT_OK(path::CreateDir(first_dir_path_));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::RenameFile(first_dir_path_, second_dir_path_));
FileMap modified_files = GetChangedFiles(2u);
EXPECT_EQ(modified_files.size(), 2u);
ExpectFileInfo(modified_files, kFirstDir, FileAction::kDeleted, kDir, 0);
ExpectFileInfo(modified_files, kSecondDir, FileAction::kAdded, kDir, 0);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RenameFile) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::RenameFile(first_file_path_, second_file_path_));
FileMap modified_files = GetChangedFiles(2u);
EXPECT_EQ(modified_files.size(), 2u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kDeleted, kFile,
kFirstDataSize);
ExpectFileInfo(modified_files, kSecondFile, FileAction::kAdded, kFile,
kFirstDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RemoveDir) {
EXPECT_OK(path::CreateDir(first_dir_path_));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::RemoveDirRec(first_dir_path_));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstDir);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RemoveFile) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::RemoveFile(first_file_path_));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ChangeFile) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kSecondData, kSecondDataSize));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, DirHierarchy) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
std::vector<std::string> files = {"1.txt", "2.txt", "3.txt", "4.txt",
"5.txt", "6.txt", "7.txt", "8.txt"};
EXPECT_OK(path::CreateDir(first_dir_path_));
EXPECT_OK(path::CreateDir(second_dir_path_));
for (const std::string& file : files) {
for (const auto& path :
{first_dir_path_, second_dir_path_, watcher_dir_path_})
EXPECT_OK(
path::WriteFile(path::Join(path, file), kFirstData, kFirstDataSize));
}
FileMap modified_files = GetChangedFiles(files.size() * 3 + 2);
ASSERT_EQ(modified_files.size(), files.size() * 3 + 2);
for (const std::string& file : files) {
ExpectFile(modified_files, path::Join(kFirstDir, file));
ExpectFile(modified_files, path::Join(kSecondDir, file));
ExpectFile(modified_files, file);
}
ExpectFile(modified_files, kFirstDir);
ExpectFile(modified_files, kSecondDir);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, NoReadDirChanges) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(watcher_.StopWatching());
FileMap modified_files = GetChangedFiles(0u);
EXPECT_EQ(modified_files.size(), 0u);
}
TEST_P(FileWatcherParameterizedTest, RestartWatchingNoChanges) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(watcher_.StopWatching());
// Restart with reading.
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(watcher_.StopWatching());
// Restart without reading.
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RestartWatchingWithChanges) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StopWatching());
// first_test_dir should not be in the modification set.
EXPECT_OK(path::CreateDir(first_dir_path_));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(second_file_path_, kSecondData, kSecondDataSize));
FileMap modified_files = GetChangedFiles(2u);
EXPECT_EQ(modified_files.size(), 2u);
ExpectFile(modified_files, kFirstFile);
ExpectFile(modified_files, kSecondFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ReadFileNoNotification) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
Buffer data;
EXPECT_OK(path::ReadFile(first_file_path_, &data));
FileMap modified_files = GetChangedFiles(0u);
EXPECT_EQ(modified_files.size(), 0u);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, SearchFilesNoNotification) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
unsigned int counter = 0;
auto handler = [&counter](const std::string& /*dir*/,
const std::string& /*filename*/,
int64_t /*modified_time*/, uint64_t /*size*/,
bool /*is_directory*/) {
++counter;
return absl::OkStatus();
};
EXPECT_OK(path::SearchFiles(first_file_path_, true, handler));
EXPECT_EQ(counter, 1u);
FileMap modified_files = GetChangedFiles(0u);
EXPECT_EQ(modified_files.size(), 0u);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionAdd) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/2)); // 1x add, 1x modify
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 1u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kAdded, kFile,
kFirstDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionModify) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kSecondData, kSecondDataSize));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/2)); // 2x modify
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 1u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kModified, kFile,
kSecondDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionAddModify) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(path::WriteFile(first_file_path_, kSecondData, kSecondDataSize));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/4)); // 1x add, 3x modify
// Add + modify should not result in FileAction::kModified.
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 1u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kAdded, kFile,
kSecondDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionDelete) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::RemoveFile(first_file_path_));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/1)); // 1x remove
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 1u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kDeleted, kFile,
kFirstDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionAddModifyRemove) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(path::WriteFile(first_file_path_, kSecondData, kSecondDataSize));
EXPECT_OK(path::RemoveFile(first_file_path_));
EXPECT_TRUE(
WaitForChange(/*min_event_count=*/5)); // 1x add, 3x modify, 1x remove
// The watcher should collapse add-modify-remove sequences and report no
// changes.
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 0u);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ActionModifyRemove) {
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kSecondData, kSecondDataSize));
EXPECT_OK(path::RemoveFile(first_file_path_));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/3)); // 2x modify, 1x remove
FileMap modified_files = watcher_.GetModifiedFiles();
EXPECT_EQ(modified_files.size(), 1u);
ExpectFileInfo(modified_files, kFirstFile, FileAction::kDeleted, kFile,
kSecondDataSize);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, ModifiedTime) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); }));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_TRUE(WaitForChange(/*min_event_count=*/2)); // 2x modify
FileMap modified_files = watcher_.GetModifiedFiles();
ASSERT_EQ(modified_files.size(), 1u);
time_t mtime;
EXPECT_OK(path::GetFileTime(first_file_path_, &mtime));
const FileInfo& info = modified_files.begin()->second;
EXPECT_EQ(info.mtime, mtime);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, DeleteWatchedDir) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); },
[this]() { OnDirRecreated(); }, kFWTimeout));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_TRUE(WaitForChange(2u)); // 2x modify
EXPECT_OK(path::RemoveDirRec(watcher_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(1u));
EXPECT_TRUE(watcher_.GetModifiedFiles().empty());
EXPECT_NOT_OK(watcher_.GetStatus());
// The error status should not be overwritten.
EXPECT_NOT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RecreateWatchedDir) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); },
[this]() { OnDirRecreated(); }, kFWTimeout));
EXPECT_OK(path::RemoveDirRec(watcher_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(1u));
EXPECT_OK(path::CreateDirRec(watcher_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(2u));
EXPECT_TRUE(watcher_.GetModifiedFiles().empty());
EXPECT_OK(watcher_.GetStatus());
// Creation of a new file should be detected.
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RecreateUpperDir) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); },
[this]() { OnDirRecreated(); }, kFWTimeout));
// Initially, there should be no dir_recreated_events.
EXPECT_EQ(watcher_.GetDirRecreateEventCountForTesting(), 0u);
EXPECT_OK(path::RemoveDirRec(test_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(1u));
// Only 1 event should exist (for directory removal).
EXPECT_OK(path::CreateDirRec(test_dir_path_));
EXPECT_OK(path::CreateDirRec(watcher_dir_path_));
// As the upper level directory is created separately, no new event should be
// available.
EXPECT_TRUE(WaitForDirRecreated(2u));
// Only 1 additional event should be registered for creation of the directory.
EXPECT_EQ(watcher_.GetDirRecreateEventCountForTesting(), 2u);
EXPECT_TRUE(watcher_.GetModifiedFiles().empty());
EXPECT_OK(watcher_.GetStatus());
// Creation of a new file should be detected.
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
FileMap modified_files = GetChangedFiles(1u);
EXPECT_EQ(modified_files.size(), 1u);
ExpectFile(modified_files, kFirstFile);
// No new events should be registered for the watched directory.
EXPECT_EQ(watcher_.GetDirRecreateEventCountForTesting(), 2u);
EXPECT_OK(watcher_.StopWatching());
}
TEST_P(FileWatcherParameterizedTest, RecreateWatchedDirNoOldChanges) {
EXPECT_OK(watcher_.StartWatching([this]() { OnFilesChanged(); },
[this]() { OnDirRecreated(); }, kFWTimeout));
EXPECT_OK(path::WriteFile(first_file_path_, kFirstData, kFirstDataSize));
EXPECT_OK(path::RemoveDirRec(watcher_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(1u));
EXPECT_OK(path::CreateDirRec(watcher_dir_path_));
EXPECT_TRUE(WaitForDirRecreated(2u));
EXPECT_TRUE(watcher_.GetModifiedFiles().empty());
EXPECT_OK(watcher_.GetStatus());
EXPECT_OK(watcher_.StopWatching());
}
struct TestName {
std::string operator()(const testing::TestParamInfo<bool>& legacy) const {
return legacy.param ? "ReadDirectoryChangesW" : "ReadDirectoryChangesExW";
}
};
// Run without and with legacy ReadDirectoryChangesW function.
INSTANTIATE_TEST_CASE_P(FileWatcherTest, FileWatcherParameterizedTest,
::testing::Values(false, true), TestName());
} // namespace
} // namespace cdc_ft
Reference in New Issue View Git Blame Copy Permalink