feat(core): Implement Steam library sync with metadata extraction and image processing (#278)

## Description
<!-- Briefly describe the purpose and scope of your changes -->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added AWS queue infrastructure and SQS handler for processing Steam
game libraries and images.
- Introduced event-driven handling for new credentials and game
additions, including image uploads to S3.
- Added client functions to fetch Steam user libraries, friends lists,
app info, and related images.
- Added new database columns and schema updates to track game
acquisition, playtime, and family sharing.
  - Added utility function for chunking arrays.
- Added new event notifications for library queue processing and game
creation.
  - Added new lookup functions for categories and teams by slug.
- Introduced a new Team API with endpoints to list and fetch teams by
slug.
  - Added a new Steam library page displaying game images.

- **Enhancements**
  - Improved game creation with event notifications and upsert logic.
  - Enhanced category and team retrieval with new lookup functions.
  - Renamed and refined image categories for clearer classification.
  - Expanded dependencies for image processing and AWS SDK integration.
- Improved image processing utilities with caching, ranking, and
metadata extraction.
  - Refined Steam client utilities for concurrency and error handling.

- **Bug Fixes**
- Fixed event publishing timing and removed deprecated credential
retrieval methods.

- **Chores**
- Updated infrastructure configurations with increased timeouts, memory,
and resource linking.
- Added new dependencies for image processing, caching, and AWS SDK
clients.
  - Refined internal code structure and imports for clarity.
  - Removed Steam provider and related UI components from the frontend.
- Disabled authentication providers and Steam-related routes in the
frontend.
  - Updated API fetch handler to accept environment bindings.

- **Refactor**
- Simplified query result handling and renamed functions for better
clarity.
- Removed outdated event handler in favor of consolidated event
subscriber.
- Consolidated and simplified database relationships and permission
queries.

- **Tests**
  - No explicit test changes included in this release.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Wanjohi
2025-05-17 00:51:18 +03:00
committed by GitHub
parent cc2065299d
commit e1a903a7c9
82 changed files with 7819 additions and 1002 deletions

View File

@@ -3,6 +3,7 @@ import { auth } from "./auth";
import { domain } from "./dns";
import { cluster } from "./cluster";
import { postgres } from "./postgres";
import { LibraryQueue } from "./steam";
import { secret, steamEncryptionKey } from "./secret";
export const apiService = new sst.aws.Service("Api", {
@@ -13,6 +14,7 @@ export const apiService = new sst.aws.Service("Api", {
bus,
auth,
postgres,
LibraryQueue,
steamEncryptionKey,
secret.PolarSecret,
secret.PolarWebhookSecret,

View File

@@ -1,19 +1,22 @@
import { vpc } from "./vpc";
import { storage } from "./storage";
// import { email } from "./email";
import { allSecrets } from "./secret";
import { postgres } from "./postgres";
import { steamEncryptionKey } from "./secret";
export const bus = new sst.aws.Bus("Bus");
bus.subscribe("Event", {
vpc,
handler: "./packages/functions/src/event/event.handler",
handler: "packages/functions/src/events/index.handler",
link: [
// email,
postgres,
...allSecrets
storage,
steamEncryptionKey
],
timeout: "5 minutes",
timeout: "10 minutes",
memory: "3002 MB",// For faster processing of large(r) images
permissions: [
{
actions: ["ses:SendEmail"],

View File

@@ -1,7 +1,19 @@
new sst.x.DevCommand("Steam", {
dev: {
command: "bun dev",
directory: "packages/steam",
autostart: true,
},
import { vpc } from "./vpc";
import { postgres } from "./postgres";
import { steamEncryptionKey } from "./secret";
export const LibraryQueue = new sst.aws.Queue("LibraryQueue", {
fifo: true,
visibilityTimeout: "10 minutes",
});
LibraryQueue.subscribe({
vpc,
timeout: "10 minutes",
memory: "3002 MB",
handler: "packages/functions/src/queues/library.handler",
link: [
postgres,
steamEncryptionKey
],
});