github-mirrors/netris-cdc-file-transfer
1
0
Fork 0
mirror of https://github.com/nestriness/cdc-file-transfer.git synced 2026-01-30 10:35:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

Rename asset_stream_manager to cdc_stream (#27)

Browse Source 
Fixes #13
This commit is contained in:
Lutz Justen
2022-12-01 16:14:56 +01:00
committed by GitHub
parent f0539226a2
commit 01a60e2490
58 changed files with 197 additions and 213 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/build_and_test.yml vendored
View File

@@ -45,19 +45,19 @@ jobs:
- name: Build - name: Build
run: | run: |
bazel build --config=windows //cdc_rsync //asset_stream_manager //tests_common //tests_asset_streaming_30 //tests_cdc_rsync bazel build --config=windows //cdc_rsync //cdc_stream //tests_common //tests_cdc_stream //tests_cdc_rsync
- name: Test - name: Test
run: | run: |
bazel-bin\tests_common\tests_common.exe bazel-bin\tests_common\tests_common.exe
bazel-bin\tests_asset_streaming_30\tests_asset_streaming_30.exe bazel-bin\tests_cdc_stream\tests_cdc_stream.exe
bazel-bin\tests_cdc_rsync\tests_cdc_rsync.exe bazel-bin\tests_cdc_rsync\tests_cdc_rsync.exe
bazel test --config=windows --test_output=errors --local_test_jobs=1 ` bazel test --config=windows --test_output=errors --local_test_jobs=1 `
//asset_stream_manager/... `
//cdc_fuse_fs/... ` //cdc_fuse_fs/... `
//cdc_rsync/... ` //cdc_rsync/... `
//cdc_rsync/base/... ` //cdc_rsync/base/... `
//cdc_rsync_server/... ` //cdc_rsync_server/... `
//cdc_stream/... `
//common/... ` //common/... `
//data_store/... ` //data_store/... `
//fastcdc/... ` //fastcdc/... `

11
.github/workflows/create_release.yml vendored
View File

@@ -53,19 +53,19 @@ jobs:
- name: Build - name: Build
run: | run: |
bazel build --config=windows --compilation_mode=opt --copt=/GL ` bazel build --config=windows --compilation_mode=opt --copt=/GL `
//cdc_rsync //asset_stream_manager //tests_common //tests_asset_streaming_30 //tests_cdc_rsync //cdc_rsync //cdc_stream //tests_common //tests_cdc_stream //tests_cdc_rsync
- name: Test - name: Test
run: | run: |
bazel-bin\tests_common\tests_common.exe bazel-bin\tests_common\tests_common.exe
bazel-bin\tests_asset_streaming_30\tests_asset_streaming_30.exe bazel-bin\tests_cdc_stream\tests_cdc_stream.exe
bazel-bin\tests_cdc_rsync\tests_cdc_rsync.exe bazel-bin\tests_cdc_rsync\tests_cdc_rsync.exe
bazel test --config=windows --compilation_mode=opt --copt=/GL --test_output=errors --local_test_jobs=1 ` bazel test --config=windows --compilation_mode=opt --copt=/GL --test_output=errors --local_test_jobs=1 `
//asset_stream_manager/... `
//cdc_fuse_fs/... ` //cdc_fuse_fs/... `
//cdc_rsync/... ` //cdc_rsync/... `
//cdc_rsync/base/... ` //cdc_rsync/base/... `
//cdc_rsync_server/... ` //cdc_rsync_server/... `
//cdc_stream/... `
//common/... ` //common/... `
//data_store/... ` //data_store/... `
//fastcdc/... ` //fastcdc/... `
@@ -77,7 +77,7 @@ jobs:
run: | run: |
mkdir artifacts mkdir artifacts
cp bazel-bin/cdc_rsync/cdc_rsync.exe artifacts cp bazel-bin/cdc_rsync/cdc_rsync.exe artifacts
cp bazel-bin/asset_stream_manager/asset_stream_manager.exe artifacts cp bazel-bin/cdc_stream/cdc_stream.exe artifacts
cp LICENSE artifacts cp LICENSE artifacts
cp README.md artifacts cp README.md artifacts
@@ -91,9 +91,6 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: [Build-And-Test-Windows, Build-And-Test-Linux] needs: [Build-And-Test-Windows, Build-And-Test-Linux]
steps: steps:
- name: Test
run: echo '${{ toJSON(github) }}'
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3

25
README.md
View File

@@ -143,10 +143,10 @@ The two tools can be built and used independently.
``` ```
* Build Windows components * Build Windows components
``` ```
bazel build --config windows --compilation_mode=opt --copt=/GL //asset_stream_manager bazel build --config windows --compilation_mode=opt --copt=/GL //cdc_stream
``` ```
* Copy the Linux build output files `cdc_fuse_fs` and `libfuse.so` from * Copy the Linux build output files `cdc_fuse_fs` and `libfuse.so` from
`bazel-bin/cdc_fuse_fs` on the Linux system to `bazel-bin\asset_stream_manager` `bazel-bin/cdc_fuse_fs` on the Linux system to `bazel-bin\cdc_stream`
on the Windows machine. on the Windows machine.
## Usage ## Usage
@@ -211,13 +211,13 @@ cdc_rsync C:\path\to\assets\* user@linux.device.com:~/assets -vr
### CDC Stream ### CDC Stream
`cdc_stream` consists of a background service called `asset_stream_manager`, `cdc_stream` consists of a background service, which has to be started in
which has to be started in advance with advance with
``` ```
asset_stream_manager cdc_stream start-service
``` ```
The service logs to `%APPDATA%\cdc-file-transfer\logs` by default. Try The service logs to `%APPDATA%\cdc-file-transfer\logs` by default. Try
`asset_stream_manager --helpfull` to get a list of available flags. `cdc_stream --help` to get a list of available flags.
To stream the Windows directory `C:\path\to\assets` to `~/assets` on the Linux To stream the Windows directory `C:\path\to\assets` to `~/assets` on the Linux
device, run device, run
@@ -235,17 +235,14 @@ cdc_stream stop user@linux.device.com:~/assets
## Troubleshooting ## Troubleshooting
`cdc_rsync` always logs to the console. By default, the `asset_stream_manager` `cdc_rsync` always logs to the console. By default, the `cdc_stream` service
service logs to a timestamped file in `%APPDATA%\cdc-file-transfer\logs`. It can logs to a timestamped file in `%APPDATA%\cdc-file-transfer\logs`. It can be
be switched to log to console by starting it with `--log_to_stdout`: switched to log to console by starting it with `--log-to-stdout`:
``` ```
asset_stream_manager --log_to_stdout cdc_stream start-service --log_to_stdout
``` ```
Both `cdc_rsync` and `asset_stream_manager` support command line flags to control log Both `cdc_rsync` and `cdc_stream` support command line flags to control log
verbosity. Passing `-vvv` prints debug logs, `-vvvv` prints verbose logs. The verbosity. Passing `-vvv` prints debug logs, `-vvvv` prints verbose logs. The
debug logs contain all SSH and SCP commands that are attempted to run, which is debug logs contain all SSH and SCP commands that are attempted to run, which is
very useful for troubleshooting. very useful for troubleshooting.
`cdc_stream` is just a thin client for the asset streaming service. Nothing ever
goes wrong with it <sup>[citation needed]</sup>.

80
all_files.vcxitems
View File

@@ -15,26 +15,26 @@
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<ClCompile Include="$(MSBuildThisFileDirectory)absl_helper\jedec_size_flag.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)absl_helper\jedec_size_flag.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\asset_stream_config.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\asset_stream_config.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\asset_stream_server.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\asset_stream_server.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\background_service_impl.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\background_service_impl.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\base_command.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\base_command.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\cdc_fuse_manager.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\cdc_fuse_manager.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\grpc_asset_stream_server.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\grpc_asset_stream_server.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\local_assets_stream_manager_client.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\local_assets_stream_manager_client.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\local_assets_stream_manager_service_impl.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\local_assets_stream_manager_service_impl.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\main.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\main.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\metrics_recorder.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\metrics_recorder.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\metrics_recorder_test.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\metrics_recorder_test.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\multi_session.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\multi_session.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\multi_session_test.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\multi_session_test.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\session.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\session.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\session_management_server.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\session_management_server.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\session_manager.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\session_manager.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\start_command.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\start_command.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\start_service_command.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\start_service_command.cc" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\stop_command.cc" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\stop_command.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)asset_stream_manager\testing_asset_stream_server.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_stream\testing_asset_stream_server.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_stream_client.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_stream_client.cc" />
<ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_test.cc" /> <ClCompile Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_test.cc" />
@@ -145,24 +145,24 @@
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<ClInclude Include="$(MSBuildThisFileDirectory)absl_helper\jedec_size_flag.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)absl_helper\jedec_size_flag.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\asset_stream_config.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\asset_stream_config.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\asset_stream_server.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\asset_stream_server.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\background_service_impl.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\background_service_impl.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\base_command.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\base_command.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\cdc_fuse_manager.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\cdc_fuse_manager.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\grpc_asset_stream_server.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\grpc_asset_stream_server.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\local_assets_stream_manager_client.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\local_assets_stream_manager_client.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\local_assets_stream_manager_service_impl.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\local_assets_stream_manager_service_impl.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\metrics_recorder.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\metrics_recorder.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\multi_session.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\multi_session.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\session.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\session.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\session_config.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\session_config.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\session_management_server.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\session_management_server.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\session_manager.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\session_manager.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\start_command.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\start_command.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\start_service_command.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\start_service_command.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\stop_command.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\stop_command.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)asset_stream_manager\testing_asset_stream_server.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_stream\testing_asset_stream_server.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_stream_client.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\asset_stream_client.h" />
<ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\cdc_fuse_fs.h" /> <ClInclude Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\cdc_fuse_fs.h" />
@@ -244,7 +244,7 @@
<None Include="$(MSBuildThisFileDirectory).bazelrc" /> <None Include="$(MSBuildThisFileDirectory).bazelrc" />
<None Include="$(MSBuildThisFileDirectory).gitignore" /> <None Include="$(MSBuildThisFileDirectory).gitignore" />
<None Include="$(MSBuildThisFileDirectory)absl_helper\BUILD" /> <None Include="$(MSBuildThisFileDirectory)absl_helper\BUILD" />
<None Include="$(MSBuildThisFileDirectory)asset_stream_manager\BUILD" /> <None Include="$(MSBuildThisFileDirectory)cdc_stream\BUILD" />
<None Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\BUILD" /> <None Include="$(MSBuildThisFileDirectory)cdc_fuse_fs\BUILD" />
<None Include="$(MSBuildThisFileDirectory)cdc_indexer\BUILD" /> <None Include="$(MSBuildThisFileDirectory)cdc_indexer\BUILD" />
<None Include="$(MSBuildThisFileDirectory)cdc_indexer\README.md" /> <None Include="$(MSBuildThisFileDirectory)cdc_indexer\README.md" />
@@ -268,7 +268,7 @@
<None Include="$(MSBuildThisFileDirectory)proto\manifest.proto" /> <None Include="$(MSBuildThisFileDirectory)proto\manifest.proto" />
<None Include="$(MSBuildThisFileDirectory)README.md" /> <None Include="$(MSBuildThisFileDirectory)README.md" />
<None Include="$(MSBuildThisFileDirectory)rm_bazel_out_dir.bat" /> <None Include="$(MSBuildThisFileDirectory)rm_bazel_out_dir.bat" />
<None Include="$(MSBuildThisFileDirectory)tests_asset_streaming_30\BUILD" /> <None Include="$(MSBuildThisFileDirectory)tests_cdc_stream\BUILD" />
<None Include="$(MSBuildThisFileDirectory)tests_common\BUILD" /> <None Include="$(MSBuildThisFileDirectory)tests_common\BUILD" />
<None Include="$(MSBuildThisFileDirectory)tests_cdc_rsync\BUILD" /> <None Include="$(MSBuildThisFileDirectory)tests_cdc_rsync\BUILD" />
<None Include="$(MSBuildThisFileDirectory)tools\BUILD" /> <None Include="$(MSBuildThisFileDirectory)tools\BUILD" />

0
asset_stream_manager/.gitignore → cdc_stream/.gitignore vendored
View File

2
asset_stream_manager/BUILD → cdc_stream/BUILD
View File

@@ -5,7 +5,7 @@ package(default_visibility = [
]) ])
cc_binary( cc_binary(
name = "asset_stream_manager", name = "cdc_stream",
srcs = ["main.cc"], srcs = ["main.cc"],
data = [":roots_pem"], data = [":roots_pem"],
deps = [ deps = [

4
asset_stream_manager/asset_stream_config.cc → cdc_stream/asset_stream_config.cc
View File

@@ -12,14 +12,14 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/asset_stream_config.h" #include "cdc_stream/asset_stream_config.h"
#include <sstream> #include <sstream>
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"
#include "absl/strings/str_join.h" #include "absl/strings/str_join.h"
#include "absl_helper/jedec_size_flag.h" #include "absl_helper/jedec_size_flag.h"
#include "asset_stream_manager/base_command.h" #include "cdc_stream/base_command.h"
#include "common/buffer.h" #include "common/buffer.h"
#include "common/path.h" #include "common/path.h"
#include "common/status_macros.h" #include "common/status_macros.h"

8
asset_stream_manager/asset_stream_config.h → cdc_stream/asset_stream_config.h
View File

@@ -14,15 +14,15 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_ASSET_STREAM_CONFIG_H_ #ifndef CDC_STREAM_ASSET_STREAM_CONFIG_H_
#define ASSET_STREAM_MANAGER_ASSET_STREAM_CONFIG_H_ #define CDC_STREAM_SET_STREAM_CONFIG_H_
#include <map> #include <map>
#include <set> #include <set>
#include <string> #include <string>
#include "absl/status/status.h" #include "absl/status/status.h"
#include "asset_stream_manager/session_config.h" #include "cdc_stream/session_config.h"
#include "session.h" #include "session.h"
namespace lyra { namespace lyra {
@@ -108,4 +108,4 @@ class AssetStreamConfig {
}; // namespace cdc_ft }; // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_ASSET_STREAM_CONFIG_H_ #endif // CDC_STREAM_SET_STREAM_CONFIG_H_

6
asset_stream_manager/asset_stream_server.cc → cdc_stream/asset_stream_server.cc
View File

@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/asset_stream_server.h" #include "cdc_stream/asset_stream_server.h"
#include "asset_stream_manager/grpc_asset_stream_server.h" #include "cdc_stream/grpc_asset_stream_server.h"
#include "asset_stream_manager/testing_asset_stream_server.h" #include "cdc_stream/testing_asset_stream_server.h"
namespace cdc_ft { namespace cdc_ft {

7
asset_stream_manager/asset_stream_server.h → cdc_stream/asset_stream_server.h
View File

@@ -14,8 +14,9 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_ASSET_STREAM_SERVER_H_ #ifndef CDC_STREAM_ASSET_STREAM_SERVER_H_
#define ASSET_STREAM_MANAGER_ASSET_STREAM_SERVER_H_ #define CDC_STREAM_ASSET_STREAM_SERVER_H_
#include <memory> #include <memory>
#include <string> #include <string>
@@ -93,4 +94,4 @@ class AssetStreamServer {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_ASSET_STREAM_SERVER_H_ #endif // CDC_STREAM_ASSET_STREAM_SERVER_H_

2
asset_stream_manager/background_service_impl.cc → cdc_stream/background_service_impl.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/background_service_impl.h" #include "cdc_stream/background_service_impl.h"
#include "common/grpc_status.h" #include "common/grpc_status.h"
#include "common/log.h" #include "common/log.h"

15
asset_stream_manager/background_service_impl.h → cdc_stream/background_service_impl.h
View File

@@ -14,23 +14,18 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_BACKGROUND_SERVICE_IMPL_H_ #ifndef CDC_STREAM_BACKGROUND_SERVICE_IMPL_H_
#define ASSET_STREAM_MANAGER_BACKGROUND_SERVICE_IMPL_H_ #define CDC_STREAM_BACKGROUND_SERVICE_IMPL_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "asset_stream_manager/background_service_impl.h" #include "cdc_stream/background_service_impl.h"
#include "asset_stream_manager/session_management_server.h" #include "cdc_stream/session_management_server.h"
#include "grpcpp/grpcpp.h" #include "grpcpp/grpcpp.h"
#include "proto/background_service.grpc.pb.h" #include "proto/background_service.grpc.pb.h"
namespace cdc_ft { namespace cdc_ft {
// Implements a service to manage a background process as a server. // Implements a service to manage a background process as a server.
// The corresponding client is implemented by ProcessManager. The background
// process in this case is asset_stream_manager. ProcessManager starts the
// process on demand (e.g. when `ggp instance mount --local-dir` is invoked) and
// manages its lifetime: It calls GetPid() initially, HealthCheck() periodically
// to monitor the process, and Exit() on shutdown.
// This service is owned by SessionManagementServer. // This service is owned by SessionManagementServer.
class BackgroundServiceImpl final class BackgroundServiceImpl final
: public backgroundservice::BackgroundService::Service { : public backgroundservice::BackgroundService::Service {
@@ -65,4 +60,4 @@ class BackgroundServiceImpl final
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_BACKGROUND_SERVICE_IMPL_H_ #endif // CDC_STREAM_BACKGROUND_SERVICE_IMPL_H_

2
asset_stream_manager/base_command.cc → cdc_stream/base_command.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/base_command.h" #include "cdc_stream/base_command.h"
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"
#include "absl_helper/jedec_size_flag.h" #include "absl_helper/jedec_size_flag.h"

6
asset_stream_manager/base_command.h → cdc_stream/base_command.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_BASE_COMMAND_H_ #ifndef CDC_STREAM_BASE_COMMAND_H_
#define ASSET_STREAM_MANAGER_BASE_COMMAND_H_ #define CDC_STREAM_BASE_COMMAND_H_
#include <string> #include <string>
@@ -89,4 +89,4 @@ class BaseCommand {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_BASE_COMMAND_H_ #endif // CDC_STREAM_BASE_COMMAND_H_

2
asset_stream_manager/cdc_fuse_manager.cc → cdc_stream/cdc_fuse_manager.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/cdc_fuse_manager.h" #include "cdc_stream/cdc_fuse_manager.h"
#include "absl/strings/match.h" #include "absl/strings/match.h"
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"

6
asset_stream_manager/cdc_fuse_manager.h → cdc_stream/cdc_fuse_manager.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_CDC_FUSE_MANAGER_H_ #ifndef CDC_STREAM_CDC_FUSE_MANAGER_H_
#define ASSET_STREAM_MANAGER_CDC_FUSE_MANAGER_H_ #define CDC_STREAM_CDC_FUSE_MANAGER_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "common/remote_util.h" #include "common/remote_util.h"
@@ -96,4 +96,4 @@ class CdcFuseManager {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_CDC_FUSE_MANAGER_H_ #endif // CDC_STREAM_CDC_FUSE_MANAGER_H_

8
asset_stream_manager/asset_stream_manager.vcxproj → cdc_stream/cdc_stream.vcxproj
View File

@@ -42,12 +42,12 @@
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup> </ImportGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<OutDir>$(SolutionDir)bazel-out\x64_windows-dbg\bin\asset_stream_manager\</OutDir> <OutDir>$(SolutionDir)bazel-out\x64_windows-dbg\bin\cdc_stream\</OutDir>
<NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions> <NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions>
<AdditionalOptions>/std:c++17</AdditionalOptions> <AdditionalOptions>/std:c++17</AdditionalOptions>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<OutDir>$(SolutionDir)bazel-out\x64_windows-opt\bin\asset_stream_manager\</OutDir> <OutDir>$(SolutionDir)bazel-out\x64_windows-opt\bin\asset_stcdc_streamream_manager\</OutDir>
<NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions> <NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions>
<AdditionalOptions>/std:c++17</AdditionalOptions> <AdditionalOptions>/std:c++17</AdditionalOptions>
</PropertyGroup> </PropertyGroup>
@@ -66,8 +66,8 @@
</ItemDefinitionGroup> </ItemDefinitionGroup>
<!-- Bazel setup --> <!-- Bazel setup -->
<PropertyGroup> <PropertyGroup>
<BazelTargets>//asset_stream_manager</BazelTargets> <BazelTargets>//cdc_stream</BazelTargets>
<BazelOutputFile>asset_stream_manager.exe</BazelOutputFile> <BazelOutputFile>cdc_stream.exe</BazelOutputFile>
<BazelIncludePaths>..\;..\third_party\absl;..\bazel-cdc-file-transfer\external\com_github_jsoncpp\include;..\bazel-cdc-file-transfer\external\com_github_blake3\c;..\third_party\googletest\googletest\include;..\bazel-cdc-file-transfer\external\com_google_protobuf\src;..\bazel-cdc-file-transfer\external\com_github_grpc_grpc\include;..\bazel-out\x64_windows-dbg\bin;..\bazel-cdc-file-transfer\external\com_github_lyra\include;$(VC_IncludePath);$(WindowsSDK_IncludePath)</BazelIncludePaths> <BazelIncludePaths>..\;..\third_party\absl;..\bazel-cdc-file-transfer\external\com_github_jsoncpp\include;..\bazel-cdc-file-transfer\external\com_github_blake3\c;..\third_party\googletest\googletest\include;..\bazel-cdc-file-transfer\external\com_google_protobuf\src;..\bazel-cdc-file-transfer\external\com_github_grpc_grpc\include;..\bazel-out\x64_windows-dbg\bin;..\bazel-cdc-file-transfer\external\com_github_lyra\include;$(VC_IncludePath);$(WindowsSDK_IncludePath)</BazelIncludePaths>
</PropertyGroup> </PropertyGroup>
<Import Project="..\NMakeBazelProject.targets" /> <Import Project="..\NMakeBazelProject.targets" />

0
asset_stream_manager/asset_stream_manager.vcxproj.filters → cdc_stream/cdc_stream.vcxproj.filters
View File

2
asset_stream_manager/grpc_asset_stream_server.cc → cdc_stream/grpc_asset_stream_server.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/grpc_asset_stream_server.h" #include "cdc_stream/grpc_asset_stream_server.h"
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"
#include "absl/time/time.h" #include "absl/time/time.h"

8
asset_stream_manager/grpc_asset_stream_server.h → cdc_stream/grpc_asset_stream_server.h
View File

@@ -14,13 +14,13 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_GRPC_ASSET_STREAM_SERVER_H_ #ifndef CDC_STREAM_GRPC_ASSET_STREAM_SERVER_H_
#define ASSET_STREAM_MANAGER_GRPC_ASSET_STREAM_SERVER_H_ #define CDC_STREAM_GRPC_ASSET_STREAM_SERVER_H_
#include <memory> #include <memory>
#include <string> #include <string>
#include "asset_stream_manager/asset_stream_server.h" #include "cdc_stream/asset_stream_server.h"
#include "common/thread_safe_map.h" #include "common/thread_safe_map.h"
namespace grpc { namespace grpc {
@@ -67,4 +67,4 @@ class GrpcAssetStreamServer : public AssetStreamServer {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_GRPC_ASSET_STREAM_SERVER_H_ #endif // CDC_STREAM_GRPC_ASSET_STREAM_SERVER_H_

2
asset_stream_manager/local_assets_stream_manager_client.cc → cdc_stream/local_assets_stream_manager_client.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/local_assets_stream_manager_client.h" #include "cdc_stream/local_assets_stream_manager_client.h"
#include <vector> #include <vector>

6
asset_stream_manager/local_assets_stream_manager_client.h → cdc_stream/local_assets_stream_manager_client.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_ #ifndef CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_
#define ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_ #define CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_
#include <memory> #include <memory>
@@ -74,4 +74,4 @@ class LocalAssetsStreamManagerClient {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_ #endif // CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_CLIENT_H_

6
asset_stream_manager/local_assets_stream_manager_service_impl.cc → cdc_stream/local_assets_stream_manager_service_impl.cc
View File

@@ -12,15 +12,15 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/local_assets_stream_manager_service_impl.h" #include "cdc_stream/local_assets_stream_manager_service_impl.h"
#include <iomanip> #include <iomanip>
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"
#include "absl/strings/str_replace.h" #include "absl/strings/str_replace.h"
#include "absl/strings/str_split.h" #include "absl/strings/str_split.h"
#include "asset_stream_manager/multi_session.h" #include "cdc_stream/multi_session.h"
#include "asset_stream_manager/session_manager.h" #include "cdc_stream/session_manager.h"
#include "common/grpc_status.h" #include "common/grpc_status.h"
#include "common/log.h" #include "common/log.h"
#include "common/path.h" #include "common/path.h"

10
asset_stream_manager/local_assets_stream_manager_service_impl.h → cdc_stream/local_assets_stream_manager_service_impl.h
View File

@@ -14,13 +14,13 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_ #ifndef CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_
#define ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_ #define CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "absl/status/statusor.h" #include "absl/status/statusor.h"
#include "asset_stream_manager/session.h" #include "cdc_stream/session.h"
#include "asset_stream_manager/session_config.h" #include "cdc_stream/session_config.h"
#include "metrics/metrics.h" #include "metrics/metrics.h"
#include "proto/local_assets_stream_manager.grpc.pb.h" #include "proto/local_assets_stream_manager.grpc.pb.h"
@@ -107,4 +107,4 @@ class LocalAssetsStreamManagerServiceImpl final
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_ #endif // CDC_STREAM_LOCAL_ASSETS_STREAM_MANAGER_SERVICE_IMPL_H_

6
asset_stream_manager/main.cc → cdc_stream/main.cc
View File

@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/start_command.h" #include "cdc_stream/start_command.h"
#include "asset_stream_manager/start_service_command.h" #include "cdc_stream/start_service_command.h"
#include "asset_stream_manager/stop_command.h" #include "cdc_stream/stop_command.h"
#include "lyra/lyra.hpp" #include "lyra/lyra.hpp"
int main(int argc, char* argv[]) { int main(int argc, char* argv[]) {

2
asset_stream_manager/metrics_recorder.cc → cdc_stream/metrics_recorder.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/metrics_recorder.h" #include "cdc_stream/metrics_recorder.h"
#include "common/log.h" #include "common/log.h"

6
asset_stream_manager/metrics_recorder.h → cdc_stream/metrics_recorder.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_METRICS_RECORDER_H_ #ifndef CDC_STREAM_METRICS_RECORDER_H_
#define ASSET_STREAM_MANAGER_METRICS_RECORDER_H_ #define CDC_STREAM_METRICS_RECORDER_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "common/util.h" #include "common/util.h"
@@ -74,4 +74,4 @@ class SessionMetricsRecorder : public MetricsRecorder {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_METRICS_RECORDER_H_ #endif // CDC_STREAM_METRICS_RECORDER_H_

2
asset_stream_manager/metrics_recorder_test.cc → cdc_stream/metrics_recorder_test.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/metrics_recorder.h" #include "cdc_stream/metrics_recorder.h"
#include "common/status_test_macros.h" #include "common/status_test_macros.h"
#include "gtest/gtest.h" #include "gtest/gtest.h"

4
asset_stream_manager/multi_session.cc → cdc_stream/multi_session.cc
View File

@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/multi_session.h" #include "cdc_stream/multi_session.h"
#include "asset_stream_manager/session.h" #include "cdc_stream/session.h"
#include "common/file_watcher_win.h" #include "common/file_watcher_win.h"
#include "common/log.h" #include "common/log.h"
#include "common/path.h" #include "common/path.h"

12
asset_stream_manager/multi_session.h → cdc_stream/multi_session.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_MULTI_SESSION_H_ #ifndef CDC_STREAM_MULTI_SESSION_H_
#define ASSET_STREAM_MANAGER_MULTI_SESSION_H_ #define CDC_STREAM_MULTI_SESSION_H_
#include <memory> #include <memory>
#include <string> #include <string>
@@ -24,9 +24,9 @@
#include "absl/status/status.h" #include "absl/status/status.h"
#include "absl/status/statusor.h" #include "absl/status/statusor.h"
#include "asset_stream_manager/asset_stream_server.h" #include "cdc_stream/asset_stream_server.h"
#include "asset_stream_manager/metrics_recorder.h" #include "cdc_stream/metrics_recorder.h"
#include "asset_stream_manager/session_config.h" #include "cdc_stream/session_config.h"
#include "common/stopwatch.h" #include "common/stopwatch.h"
#include "data_store/data_store_writer.h" #include "data_store/data_store_writer.h"
#include "manifest/file_chunk_map.h" #include "manifest/file_chunk_map.h"
@@ -264,4 +264,4 @@ class MultiSession {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_MULTI_SESSION_H_ #endif // CDC_STREAM_MULTI_SESSION_H_

4
asset_stream_manager/multi_session_test.cc → cdc_stream/multi_session_test.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/multi_session.h" #include "cdc_stream/multi_session.h"
#include <chrono> #include <chrono>
#include <string> #include <string>
@@ -20,7 +20,7 @@
#include <vector> #include <vector>
#include "absl/strings/match.h" #include "absl/strings/match.h"
#include "asset_stream_manager/testing_asset_stream_server.h" #include "cdc_stream/testing_asset_stream_server.h"
#include "common/path.h" #include "common/path.h"
#include "common/platform.h" #include "common/platform.h"
#include "common/process.h" #include "common/process.h"

4
asset_stream_manager/session.cc → cdc_stream/session.cc
View File

@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/session.h" #include "cdc_stream/session.h"
#include "asset_stream_manager/cdc_fuse_manager.h" #include "cdc_stream/cdc_fuse_manager.h"
#include "common/log.h" #include "common/log.h"
#include "common/port_manager.h" #include "common/port_manager.h"
#include "common/status.h" #include "common/status.h"

10
asset_stream_manager/session.h → cdc_stream/session.h
View File

@@ -14,16 +14,16 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_SESSION_H_ #ifndef CDC_STREAM_SESSION_H_
#define ASSET_STREAM_MANAGER_SESSION_H_ #define CDC_STREAM_SESSION_H_
#include <memory> #include <memory>
#include <string> #include <string>
#include <unordered_map> #include <unordered_map>
#include "absl/status/status.h" #include "absl/status/status.h"
#include "asset_stream_manager/metrics_recorder.h" #include "cdc_stream/metrics_recorder.h"
#include "asset_stream_manager/session_config.h" #include "cdc_stream/session_config.h"
#include "common/remote_util.h" #include "common/remote_util.h"
namespace cdc_ft { namespace cdc_ft {
@@ -100,4 +100,4 @@ class Session {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_SESSION_H_ #endif // CDC_STREAM_SESSION_H_

6
asset_stream_manager/session_config.h → cdc_stream/session_config.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_SESSION_CONFIG_H_ #ifndef CDC_STREAM_SESSION_CONFIG_H_
#define ASSET_STREAM_MANAGER_SESSION_CONFIG_H_ #define CDC_STREAM_SESSION_CONFIG_H_
#include <cstdint> #include <cstdint>
@@ -60,4 +60,4 @@ struct SessionConfig {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_SESSION_CONFIG_H_ #endif // CDC_STREAM_SESSION_CONFIG_H_

11
asset_stream_manager/session_management_server.cc → cdc_stream/session_management_server.cc
View File

@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/session_management_server.h" #include "cdc_stream/session_management_server.h"
#include "absl/strings/str_format.h" #include "absl/strings/str_format.h"
#include "asset_stream_manager/background_service_impl.h" #include "cdc_stream/background_service_impl.h"
#include "asset_stream_manager/local_assets_stream_manager_service_impl.h" #include "cdc_stream/local_assets_stream_manager_service_impl.h"
#include "asset_stream_manager/session_manager.h" #include "cdc_stream/session_manager.h"
#include "common/log.h" #include "common/log.h"
#include "common/status.h" #include "common/status.h"
#include "common/status_macros.h" #include "common/status_macros.h"
@@ -56,8 +56,7 @@ absl::Status SessionManagementServer::Start(int port) {
} }
if (!server_) { if (!server_) {
return MakeStatus( return MakeStatus(
"Failed to start session management server. Check asset_stream_manager " "Failed to start session management server. Check cdc_stream logs.");
"logs.");
} }
LOG_INFO("Session management server listening on '%s'", server_address); LOG_INFO("Session management server listening on '%s'", server_address);
return absl::OkStatus(); return absl::OkStatus();

6
asset_stream_manager/session_management_server.h → cdc_stream/session_management_server.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_SESSION_MANAGEMENT_SERVER_H_ #ifndef CDC_STREAM_SESSION_MANAGEMENT_SERVER_H_
#define ASSET_STREAM_MANAGER_SESSION_MANAGEMENT_SERVER_H_ #define CDC_STREAM_SESSION_MANAGEMENT_SERVER_H_
#include <memory> #include <memory>
@@ -62,4 +62,4 @@ class SessionManagementServer {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_SESSION_MANAGEMENT_SERVER_H_ #endif // CDC_STREAM_SESSION_MANAGEMENT_SERVER_H_

4
asset_stream_manager/session_manager.cc → cdc_stream/session_manager.cc
View File

@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/session_manager.h" #include "cdc_stream/session_manager.h"
#include "absl/strings/str_split.h" #include "absl/strings/str_split.h"
#include "asset_stream_manager/multi_session.h" #include "cdc_stream/multi_session.h"
#include "common/log.h" #include "common/log.h"
#include "common/process.h" #include "common/process.h"
#include "common/status.h" #include "common/status.h"

8
asset_stream_manager/session_manager.h → cdc_stream/session_manager.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_SESSION_MANAGER_H_ #ifndef CDC_STREAM_SESSION_MANAGER_H_
#define ASSET_STREAM_MANAGER_SESSION_MANAGER_H_ #define CDC_STREAM_SESSION_MANAGER_H_
#include <memory> #include <memory>
#include <unordered_map> #include <unordered_map>
@@ -23,7 +23,7 @@
#include "absl/status/status.h" #include "absl/status/status.h"
#include "absl/status/statusor.h" #include "absl/status/statusor.h"
#include "absl/synchronization/mutex.h" #include "absl/synchronization/mutex.h"
#include "asset_stream_manager/session_config.h" #include "cdc_stream/session_config.h"
#include "metrics/metrics.h" #include "metrics/metrics.h"
namespace cdc_ft { namespace cdc_ft {
@@ -100,4 +100,4 @@ class SessionManager {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_SESSION_MANAGER_H_ #endif // CDC_STREAM_SESSION_MANAGER_H_

6
asset_stream_manager/start_command.cc → cdc_stream/start_command.cc
View File

@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/start_command.h" #include "cdc_stream/start_command.h"
#include <memory> #include <memory>
#include "asset_stream_manager/local_assets_stream_manager_client.h" #include "cdc_stream/local_assets_stream_manager_client.h"
#include "asset_stream_manager/session_management_server.h" #include "cdc_stream/session_management_server.h"
#include "common/log.h" #include "common/log.h"
#include "common/path.h" #include "common/path.h"
#include "common/remote_util.h" #include "common/remote_util.h"

8
asset_stream_manager/start_command.h → cdc_stream/start_command.h
View File

@@ -14,11 +14,11 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_START_COMMAND_H_ #ifndef CDC_STREAM_START_COMMAND_H_
#define ASSET_STREAM_MANAGER_START_COMMAND_H_ #define CDC_STREAM_START_COMMAND_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "asset_stream_manager/base_command.h" #include "cdc_stream/base_command.h"
namespace cdc_ft { namespace cdc_ft {
@@ -45,4 +45,4 @@ class StartCommand : public BaseCommand {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_START_COMMAND_H_ #endif // CDC_STREAM_START_COMMAND_H_

12
asset_stream_manager/start_service_command.cc → cdc_stream/start_service_command.cc
View File

@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/start_service_command.h" #include "cdc_stream/start_service_command.h"
#include "asset_stream_manager/background_service_impl.h" #include "cdc_stream/background_service_impl.h"
#include "asset_stream_manager/local_assets_stream_manager_service_impl.h" #include "cdc_stream/local_assets_stream_manager_service_impl.h"
#include "asset_stream_manager/session_management_server.h" #include "cdc_stream/session_management_server.h"
#include "asset_stream_manager/session_manager.h" #include "cdc_stream/session_manager.h"
#include "common/clock.h" #include "common/clock.h"
#include "common/grpc_status.h" #include "common/grpc_status.h"
#include "common/log.h" #include "common/log.h"
@@ -95,7 +95,7 @@ absl::Status StartServiceCommand::Run() {
if (!status.ok()) { if (!status.ok()) {
LOG_ERROR("%s", status.ToString()); LOG_ERROR("%s", status.ToString());
} else { } else {
LOG_INFO("Asset stream manager shut down successfully."); LOG_INFO("Streaming service shut down successfully.");
} }
return status; return status;

10
asset_stream_manager/start_service_command.h → cdc_stream/start_service_command.h
View File

@@ -14,15 +14,15 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_START_SERVICE_COMMAND_H_ #ifndef CDC_STREAM_START_SERVICE_COMMAND_H_
#define ASSET_STREAM_MANAGER_START_SERVICE_COMMAND_H_ #define CDC_STREAM_START_SERVICE_COMMAND_H_
#include <memory> #include <memory>
#include "absl/status/status.h" #include "absl/status/status.h"
#include "absl/status/statusor.h" #include "absl/status/statusor.h"
#include "asset_stream_manager/asset_stream_config.h" #include "cdc_stream/asset_stream_config.h"
#include "asset_stream_manager/base_command.h" #include "cdc_stream/base_command.h"
namespace cdc_ft { namespace cdc_ft {
@@ -51,4 +51,4 @@ class StartServiceCommand : public BaseCommand {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_START_SERVICE_COMMAND_H_ #endif // CDC_STREAM_START_SERVICE_COMMAND_H_

6
asset_stream_manager/stop_command.cc → cdc_stream/stop_command.cc
View File

@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/stop_command.h" #include "cdc_stream/stop_command.h"
#include <memory> #include <memory>
#include "asset_stream_manager/local_assets_stream_manager_client.h" #include "cdc_stream/local_assets_stream_manager_client.h"
#include "asset_stream_manager/session_management_server.h" #include "cdc_stream/session_management_server.h"
#include "common/log.h" #include "common/log.h"
#include "common/path.h" #include "common/path.h"
#include "common/status_macros.h" #include "common/status_macros.h"

8
asset_stream_manager/stop_command.h → cdc_stream/stop_command.h
View File

@@ -14,11 +14,11 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_STOP_COMMAND_H_ #ifndef CDC_STREAM_STOP_COMMAND_H_
#define ASSET_STREAM_MANAGER_STOP_COMMAND_H_ #define CDC_STREAM_STOP_COMMAND_H_
#include "absl/status/status.h" #include "absl/status/status.h"
#include "asset_stream_manager/base_command.h" #include "cdc_stream/base_command.h"
namespace cdc_ft { namespace cdc_ft {
@@ -41,4 +41,4 @@ class StopCommand : public BaseCommand {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_STOP_COMMAND_H_ #endif // CDC_STREAM_STOP_COMMAND_H_

0
asset_stream_manager/testdata/multi_session/non_empty/a.txt → cdc_stream/testdata/multi_session/non_empty/a.txt vendored
View File

0
asset_stream_manager/testdata/multi_session/non_empty/subdir/b.txt → cdc_stream/testdata/multi_session/non_empty/subdir/b.txt vendored
View File

0
asset_stream_manager/testdata/multi_session/non_empty/subdir/c.txt → cdc_stream/testdata/multi_session/non_empty/subdir/c.txt vendored
View File

0
asset_stream_manager/testdata/multi_session/non_empty/subdir/d.txt → cdc_stream/testdata/multi_session/non_empty/subdir/d.txt vendored
View File

0
asset_stream_manager/testdata/root.txt → cdc_stream/testdata/root.txt vendored
View File

2
asset_stream_manager/testing_asset_stream_server.cc → cdc_stream/testing_asset_stream_server.cc
View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
#include "asset_stream_manager/testing_asset_stream_server.h" #include "cdc_stream/testing_asset_stream_server.h"
#include "data_store/data_store_reader.h" #include "data_store/data_store_reader.h"
#include "manifest/file_chunk_map.h" #include "manifest/file_chunk_map.h"

8
asset_stream_manager/testing_asset_stream_server.h → cdc_stream/testing_asset_stream_server.h
View File

@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
*/ */
#ifndef ASSET_STREAM_MANAGER_TESTING_ASSET_STREAM_SERVER_H_ #ifndef CDC_STREAM_TESTING_ASSET_STREAM_SERVER_H_
#define ASSET_STREAM_MANAGER_TESTING_ASSET_STREAM_SERVER_H_ #define CDC_STREAM_TESTING_ASSET_STREAM_SERVER_H_
#include <memory> #include <memory>
#include <string> #include <string>
@@ -23,7 +23,7 @@
#include "absl/base/thread_annotations.h" #include "absl/base/thread_annotations.h"
#include "absl/status/status.h" #include "absl/status/status.h"
#include "absl/synchronization/mutex.h" #include "absl/synchronization/mutex.h"
#include "asset_stream_manager/grpc_asset_stream_server.h" #include "cdc_stream/grpc_asset_stream_server.h"
#include "manifest/manifest_proto_defs.h" #include "manifest/manifest_proto_defs.h"
namespace cdc_ft { namespace cdc_ft {
@@ -57,4 +57,4 @@ class TestingAssetStreamServer : public AssetStreamServer {
} // namespace cdc_ft } // namespace cdc_ft
#endif // ASSET_STREAM_MANAGER_TESTING_ASSET_STREAM_SERVER_H_ #endif // CDC_STREAM_TESTING_ASSET_STREAM_SERVER_H_

6
file_transfer.sln
View File

@@ -8,15 +8,15 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_rsync", "cdc_rsync\cdc_
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_rsync_server", "cdc_rsync_server\cdc_rsync_server.vcxproj", "{4ECE65E0-D950-4B96-8AD5-0313261B8C8D}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_rsync_server", "cdc_rsync_server\cdc_rsync_server.vcxproj", "{4ECE65E0-D950-4B96-8AD5-0313261B8C8D}"
EndProject EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Asset Streaming 3.0", "Asset Streaming 3.0", "{E4C4704F-C5C8-4A1D-B7E7-4D201DD29585}" Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "CDC Stream", "CDC Stream", "{E4C4704F-C5C8-4A1D-B7E7-4D201DD29585}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_fuse_fs", "cdc_fuse_fs\cdc_fuse_fs.vcxproj", "{A537310C-0571-43D5-B7FE-C867F702294F}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_fuse_fs", "cdc_fuse_fs\cdc_fuse_fs.vcxproj", "{A537310C-0571-43D5-B7FE-C867F702294F}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "asset_stream_manager", "asset_stream_manager\asset_stream_manager.vcxproj", "{84D81562-D66C-4A60-9F48-2696D7D81D26}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cdc_stream", "cdc_stream\cdc_stream.vcxproj", "{84D81562-D66C-4A60-9F48-2696D7D81D26}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "all_files", "all_files.vcxitems", "{F542AF2D-5A17-4F55-BE40-B1A2A6182811}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "all_files", "all_files.vcxitems", "{F542AF2D-5A17-4F55-BE40-B1A2A6182811}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tests_asset_streaming_30", "tests_asset_streaming_30\tests_asset_streaming_30.vcxproj", "{2B36A6AA-FD11-4F77-A79B-29A3B185E701}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tests_cdc_stream", "tests_cdc_stream\tests_cdc_stream.vcxproj", "{2B36A6AA-FD11-4F77-A79B-29A3B185E701}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tests_cdc_rsync", "tests_cdc_rsync\tests_cdc_rsync.vcxproj", "{DBFE017C-31F0-47CA-9649-06F3C3CF0E41}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tests_cdc_rsync", "tests_cdc_rsync\tests_cdc_rsync.vcxproj", "{DBFE017C-31F0-47CA-9649-06F3C3CF0E41}"
EndProject EndProject

2
manifest/manifest_builder.h
View File

@@ -34,8 +34,6 @@ namespace cdc_ft {
// finalized with a call to Flush(). When the CdcParamsProto given during // finalized with a call to Flush(). When the CdcParamsProto given during
// construction specifies an average chunk size, then the manifest will be split // construction specifies an average chunk size, then the manifest will be split
// into balanced chunks of at most this size. // into balanced chunks of at most this size.
//
// See (internal).
class ManifestBuilder { class ManifestBuilder {
public: public:
// Default permission bits for new directories and files, respectively. // Default permission bits for new directories and files, respectively.

7
proto/manifest.proto
View File

@@ -12,12 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
// This proto defines the manifest format used for Asset Streaming 3.0 and // This proto defines the manifest format used for CDC Stream.
// package diffing.
//
// References:
// * (internal).0
// * (internal)
syntax = "proto3"; syntax = "proto3";

0
tests_asset_streaming_30/.gitignore → tests_cdc_stream/.gitignore vendored
View File

10
tests_asset_streaming_30/BUILD → tests_cdc_stream/BUILD
View File

@@ -12,25 +12,25 @@ package(default_visibility = ["//visibility:public"])
# To run this target with Bazel directly, just use "bazel run" instead of # To run this target with Bazel directly, just use "bazel run" instead of
# "bazel test". # "bazel test".
cc_binary( cc_binary(
name = "tests_asset_streaming_30", name = "tests_cdc_stream",
testonly = True, testonly = True,
srcs = [ srcs = [
"//asset_stream_manager:all_test_sources",
"//cdc_fuse_fs:all_test_sources", "//cdc_fuse_fs:all_test_sources",
"//cdc_stream:all_test_sources",
"//data_store:all_test_sources", "//data_store:all_test_sources",
"//manifest:all_test_sources", "//manifest:all_test_sources",
"//metrics:all_test_sources", "//metrics:all_test_sources",
], ],
data = [ data = [
"//asset_stream_manager:all_test_data", "//cdc_stream:all_test_data",
"//manifest:all_test_data", "//manifest:all_test_data",
], ],
deps = [ deps = [
"//asset_stream_manager:metrics_recorder",
"//asset_stream_manager:multi_session",
"//cdc_fuse_fs:asset", "//cdc_fuse_fs:asset",
"//cdc_fuse_fs:cdc_fuse_fs_lib_mocked", "//cdc_fuse_fs:cdc_fuse_fs_lib_mocked",
"//cdc_fuse_fs:mock_config_stream_client", "//cdc_fuse_fs:mock_config_stream_client",
"//cdc_stream:metrics_recorder",
"//cdc_stream:multi_session",
"//common:test_main", "//common:test_main",
"//common:testing_clock", "//common:testing_clock",
"//data_store:data_provider", "//data_store:data_provider",

10
tests_asset_streaming_30/tests_asset_streaming_30.vcxproj → tests_cdc_stream/tests_cdc_stream.vcxproj
View File

@@ -14,17 +14,17 @@
<VCProjectVersion>15.0</VCProjectVersion> <VCProjectVersion>15.0</VCProjectVersion>
<ProjectGuid>{2B36A6AA-FD11-4F77-A79B-29A3B185E701}</ProjectGuid> <ProjectGuid>{2B36A6AA-FD11-4F77-A79B-29A3B185E701}</ProjectGuid>
<Keyword>Win32Proj</Keyword> <Keyword>Win32Proj</Keyword>
<RootNamespace>tests_asset_streaming_30</RootNamespace> <RootNamespace>tests_cdc_stream</RootNamespace>
<WindowsTargetPlatformVersion Condition="$(VisualStudioVersion) == 15">$([Microsoft.Build.Utilities.ToolLocationHelper]::GetLatestSDKTargetPlatformVersion('Windows', '10.0'))</WindowsTargetPlatformVersion> <WindowsTargetPlatformVersion Condition="$(VisualStudioVersion) == 15">$([Microsoft.Build.Utilities.ToolLocationHelper]::GetLatestSDKTargetPlatformVersion('Windows', '10.0'))</WindowsTargetPlatformVersion>
<WindowsTargetPlatformVersion Condition="$(VisualStudioVersion) == 16">10.0</WindowsTargetPlatformVersion> <WindowsTargetPlatformVersion Condition="$(VisualStudioVersion) == 16">10.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<OutDir>$(SolutionDir)bazel-out\x64_windows-opt\bin\tests_asset_streaming_30\</OutDir> <OutDir>$(SolutionDir)bazel-out\x64_windows-opt\bin\tests_cdc_stream\</OutDir>
<NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions> <NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions>
<AdditionalOptions>/std:c++17</AdditionalOptions> <AdditionalOptions>/std:c++17</AdditionalOptions>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<OutDir>$(SolutionDir)bazel-out\x64_windows-dbg\bin\tests_asset_streaming_30\</OutDir> <OutDir>$(SolutionDir)bazel-out\x64_windows-dbg\bin\tests_cdc_stream\</OutDir>
<NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions> <NMakePreprocessorDefinitions>UNICODE</NMakePreprocessorDefinitions>
<AdditionalOptions>/std:c++17</AdditionalOptions> <AdditionalOptions>/std:c++17</AdditionalOptions>
</PropertyGroup> </PropertyGroup>
@@ -59,8 +59,8 @@
</ItemDefinitionGroup> </ItemDefinitionGroup>
<!-- Bazel setup --> <!-- Bazel setup -->
<PropertyGroup> <PropertyGroup>
<BazelTargets>//tests_asset_streaming_30</BazelTargets> <BazelTargets>//tests_cdc_stream</BazelTargets>
<BazelOutputFile>tests_asset_streaming_30.exe</BazelOutputFile> <BazelOutputFile>tests_cdc_stream.exe</BazelOutputFile>
<BazelIncludePaths>..\;..\third_party\absl;..\bazel-cdc-file-transfer\external\com_github_jsoncpp\include;..\bazel-cdc-file-transfer\external\com_github_blake3\c;..\third_party\googletest\googletest\include;..\bazel-cdc-file-transfer\external\com_google_protobuf\src;..\bazel-cdc-file-transfer\external\com_github_grpc_grpc\include;..\bazel-out\x64_windows-dbg\bin;$(VC_IncludePath);$(WindowsSDK_IncludePath)</BazelIncludePaths> <BazelIncludePaths>..\;..\third_party\absl;..\bazel-cdc-file-transfer\external\com_github_jsoncpp\include;..\bazel-cdc-file-transfer\external\com_github_blake3\c;..\third_party\googletest\googletest\include;..\bazel-cdc-file-transfer\external\com_google_protobuf\src;..\bazel-cdc-file-transfer\external\com_github_grpc_grpc\include;..\bazel-out\x64_windows-dbg\bin;$(VC_IncludePath);$(WindowsSDK_IncludePath)</BazelIncludePaths>
</PropertyGroup> </PropertyGroup>
<Import Project="..\NMakeBazelProject.targets" /> <Import Project="..\NMakeBazelProject.targets" />

6
third_party/lyra/BUILD.bazel vendored
View File

@@ -2,12 +2,14 @@
# This project provides Linux compatible Dirent interface for Microsoft Windows. # This project provides Linux compatible Dirent interface for Microsoft Windows.
# #
load("@rules_cc//cc:defs.bzl", "cc_library")
package(default_visibility = ["//visibility:public"])
licenses(["notice"]) # BSL-1.0 license licenses(["notice"]) # BSL-1.0 license
exports_files(["LICENSE"]) exports_files(["LICENSE"])
package(default_visibility = ["//visibility:public"])
cc_library( cc_library(
name = "lyra", name = "lyra",
hdrs = [ hdrs = [