whimsical-c4lic0/OF-DL
1
0
Fork 0
forked from sim0n00ps/OF-DL
Code Pull Requests Activity
OF-DL/AGENTS.md

13 KiB
Raw Blame History

AGENTS.md

Note: Keep AGENTS.md updated as project structure, key services, or workflows change.

This repo is OF DL (also known as OF-DL), a C# app suite (console + Avalonia desktop GUI) that downloads media from a user's OnlyFans account(s). This document is for AI agents helping developers modify the project. It focuses on architecture, data flow, and the most important change points.

Quick Flow

  1. Program.Main is the app entrypoint: default launch is GUI; --cli switches to CLI mode.
  2. StartupService.CheckVersionAsync checks the latest release tag (OFDLV*) from git.ofdl.tools when not in DEBUG.
  3. StartupService.ValidateEnvironmentAsync validates OS, FFmpeg, rules.json, and Widevine device files.
  4. AuthService loads auth.json or opens a browser login (PuppeteerSharp) and persists auth data.
  5. ApiService signs every API request with dynamic rules and the current auth.
  6. DownloadOrchestrationService selects creators, prepares folders/DBs, and calls DownloadService per media type.
  7. DownloadService downloads media, handles DRM, and records metadata in SQLite.
  8. OF DL.Gui starts with config validation, then auth validation/browser login, then loads users/lists and provides multi-select download controls.

Project Layout

  • OF DL.Cli/CLI/ contains Spectre.Console UI helpers and progress reporting (CLI-only).
  • OF DL.Gui/ contains the Avalonia desktop UI (App, MainWindow, AboutWindow, FaqWindow, MVVM view models, and GUI event handlers).
  • OF DL.Gui/Helpers/ contains GUI-specific utility helpers (for example, Docker-aware web-link behavior shared across windows).
  • OF DL.Core/Services/ contains application services (API, auth, download, config, DB, startup, logging, filenames).
  • OF DL.Core/Models/ holds configuration, auth, API request/response models, downloads/startup results, DTOs, entities, and mapping helpers.
  • OF DL.Core/Widevine/ implements Widevine CDM handling and key derivation.
  • OF DL.Core/Helpers/, OF DL.Core/Utils/, OF DL.Core/Crypto/, OF DL.Core/Enumerations/ contain shared core logic. OF DL.Core/Helpers/EnvironmentHelper.cs centralizes environment checks (Docker and Windows).
  • docs/ and mkdocs.yml define the documentation site.
  • site/ is generated MkDocs output and should not be edited by hand.
  • docker/ contains container entrypoint and supervisor configuration.

Key Services

  • ApiService (OF DL.Core/Services/ApiService.cs) builds signed headers, performs HTTP requests, and maps DTOs to entities. It also handles DRM-related calls like MPD/PSSH extraction and license requests.
  • AuthService (OF DL.Core/Services/AuthService.cs) loads auth.json or performs browser-based login with PuppeteerSharp, then persists auth. It also normalizes cookies.
  • ConfigService (OF DL.Core/Services/ConfigService.cs) loads config.conf (HOCON), migrates legacy config.json, and updates global settings (logging, text sanitization).
  • DownloadService (OF DL.Core/Services/DownloadService.cs) downloads all media (images, video, audio) and handles DRM video decryption and FFmpeg execution.
  • DownloadOrchestrationService (OF DL.Core/Services/DownloadOrchestrationService.cs) coordinates user selection, subscription lists, per-user folder prep, and per-media-type download execution.
  • DBService (OF DL.Core/Services/DBService.cs) manages SQLite metadata DBs for downloaded media and a users.db index.
  • StartupService (OF DL.Core/Services/StartupService.cs) validates FFmpeg, rules.json, Widevine device files, and performs release version checks.
  • LoggingService (OF DL.Core/Services/LoggingService.cs) writes logs to logs/OFDL.txt and updates log level based on config.
  • FileNameService (OF DL.Core/Services/FileNameService.cs) formats filenames using the custom format rules from config.

Models

  • DTOs live under OF DL.Core/Models/Dtos/ and mirror API response JSON.
  • Entities live under OF DL.Core/Models/Entities/ and represent the internal domain used by download logic.
  • Mappers in OF DL.Core/Models/Mappers/ convert DTOs into entities to isolate API changes from downstream logic.
  • Non-DTO/Entity models are grouped by concern under OF DL.Core/Models/OfdlApi/, Auth/, Config/, Downloads/, and Startup/.
  • Classes in OF DL.Core/Models/OfdlApi/ mirror request and response JOSN OF DL APIs (custom and gitea)
  • Classes in OF DL.Core/Models/Config/ are used for reading and storing application configuration
  • Classes in OF DL.Core/Models/Downloads/ contain counts and application state for downloads

Configuration

  • Primary config file is config.conf (HOCON). ConfigService migrates legacy config.json if found and creates a default config.conf if missing.
  • Config lives in OF DL.Core/Models/Config/Config.cs and is populated by ConfigService.LoadConfigFromFileAsync.
  • ConfigService.UpdateConfig is the central place where runtime config changes are applied (logging level and text sanitization).
  • CLI flag --non-interactive forces non-interactive mode; ConfigService.IsCliNonInteractive and Config.NonInteractiveMode both gate prompts.
  • FFmpeg path is read from config.conf, auth.json, or auto-detected from PATH/current directory.

Runtime Files (relative to the working directory)

  • config.conf, auth.json, and rules.json are loaded from the current working directory.
  • cdm/ (Widevine device files), chrome-data/ (Puppeteer profile), and logs/ are created under the working directory.
  • users.db is stored at the working directory root.

Execution and Testing

  • .NET SDK: 10.x (net10.0 for all projects).
  • Build from the repo root:
dotnet build OF DL.sln
  • Run from source (GUI mode, default):
dotnet run --project "OF DL/OF DL.Cli.csproj"
  • Run CLI mode:
dotnet run --project "OF DL/OF DL.Cli.csproj" -- --cli
  • If you want a local rules.json fallback, run from OF DL.Cli/ or copy OF DL.Cli/rules.json into your working directory.
  • Run tests:
dotnet test "OF DL.Tests/OF DL.Tests.csproj"
  • Optional coverage (coverlet collector):
dotnet test "OF DL.Tests/OF DL.Tests.csproj" --collect:"XPlat Code Coverage"

Authentication Flow

  • Auth data is stored in auth.json using the Auth model in OF DL.Core/Models/Auth/Auth.cs.
  • AuthService.LoadFromBrowserAsync launches Chrome via PuppeteerSharp, waits for login, then extracts auth_id and sess cookies, bcTokenSha from localStorage (used as X_BC), and USER_AGENT from the browser.
  • AuthService.ValidateCookieString rewrites the cookie string so it contains only auth_id and sess and ensures a trailing ;.
  • AuthService uses chrome-data/ as its user data directory; Logout deletes chrome-data and auth.json.

Environment variables used by auth:

  • OFDL_DOCKER=true toggles Docker-specific instructions and browser flags.
  • OFDL_PUPPETEER_EXECUTABLE_PATH overrides the Chromium path for PuppeteerSharp.

Dynamic Rules and Signature Headers

  • All OnlyFans API requests use dynamic headers from ApiService.GetDynamicHeaders.
  • Dynamic rules are fetched from https://git.ofdl.tools/sim0n00ps/dynamic-rules/raw/branch/main/rules.json with fallback to local rules.json in the current working directory. The repo ships OF DL/rules.json as the default rules file.
  • Cache durations: 15 minutes for remote rules, 5 minutes for local rules.
  • DynamicRules shape is defined in OF DL.Core/Models/OfdlApi/DynamicRules.cs and includes app-token, static_param, prefix, suffix, checksum_constant, and checksum_indexes.

Signature algorithm in GetDynamicHeaders:

  • input = "{static_param}\n{timestamp_ms}\n{path+query}\n{user_id}"
  • hash = SHA1(input) lower-case hex string
  • checksum = sum(hashString[index] char values) + checksum_constant
  • sign = "{prefix}:{hash}:{checksum_hex}:{suffix}"

Headers included in signed requests:

  • app-token, sign, time, user-id, user-agent, x-bc, cookie.

Widevine CDM and DRM Decryption

  • Runtime Widevine device files are expected at cdm/devices/chrome_1610/device_client_id_blob and cdm/devices/chrome_1610/device_private_key (relative to the working directory). Paths are defined in OF DL.Core/Widevine/Constants.cs and validated in StartupService.

DRM flow is primarily in DownloadService.GetDecryptionInfo and ApiService DRM helpers:

  • ApiService.GetDRMMPDPSSH downloads the MPD manifest and extracts the cenc:pssh value.
  • ApiService.GetDRMMPDLastModified uses CloudFront signed cookies and returns MPD Last-Modified.
  • DownloadService.GetDecryptionInfo builds DRM headers (via GetDynamicHeaders) and hits the license endpoint.

Two decryption paths exist:

  • If CDM device files exist, ApiService.GetDecryptionKeyCDM uses Widevine/CDMApi.
  • If missing, ApiService.GetDecryptionKeyOFDL calls https://ofdl.tools/WV as a fallback.

DownloadService.DownloadDrmMedia runs FFmpeg with -cenc_decryption_key, CloudFront cookies, and auth cookies/user-agent. Output is written to {filename}_source.mp4, then moved and recorded in SQLite.

Download Paths, Data, and Logs

  • Default download root is __user_data__/sites/OnlyFans/{username} when DownloadPath is blank. This is computed in DownloadOrchestrationService.ResolveDownloadPath.
  • Each creator folder gets a Metadata/user_data.db (SQLite) managed by DBService.
  • A global users.db in the working directory tracks subscribed creators and user IDs.
  • Logs are written to logs/OFDL.txt (rolling daily); FFmpeg report files are also written under logs/ when debug logging is enabled.

Docs (MkDocs)

  • Docs source lives under docs/ and configuration is in mkdocs.yml.
  • Build the site with mkdocs build --clean (outputs to site/).
  • Preview locally with mkdocs serve.

CI/CD (Gitea Workflows)

  • /.gitea/workflows/publish-docs.yml builds and deploys docs on tag pushes matching OFDLV* and on manual dispatch.
  • /.gitea/workflows/publish-docker.yml builds multi-arch Docker images on OFDLV* tags and pushes to the Gitea registry.
  • /.gitea/workflows/publish-release.yml publishes Windows and Linux builds on OFDLV* tags and creates a draft release.

Docker Image

  • Built via /.gitea/workflows/publish-docker.yml on tag pushes OFDLV*.
  • Image tags: git.ofdl.tools/sim0n00ps/of-dl:latest and git.ofdl.tools/sim0n00ps/of-dl:{version}.
  • Build args include VERSION (tag name with OFDLV stripped).
  • Platforms: linux/amd64 and linux/arm64.
  • Runtime uses /config as the working directory and /data for downloads; docker/entrypoint.sh seeds /config/config.conf and /config/rules.json from /default-config.

Release Checklist

  1. Update docs under docs/ and verify locally with mkdocs build --clean or mkdocs serve.
  2. Tag the release as OFDLV{version} and push the tag.
  3. Verify the draft release artifact and publish the release in Gitea.

Coding Style (from .editorconfig)

  • Indentation: 4 spaces by default, 2 spaces for XML/YAML/project files. No tabs.
  • Line endings: LF for *.sh, CRLF for *.cmd/*.bat.
  • C# braces on new lines (csharp_new_line_before_open_brace = all).
  • Prefer predefined types (int, string) and avoid var except when type is apparent.
  • using directives go outside the namespace and System namespaces are sorted first.
  • Private/internal fields use _camelCase; private/internal static fields use s_ prefix.
  • const fields should be PascalCase.
  • Prefer braces for control blocks and expression-bodied members (silent preferences).

Where to Look First

  • OF DL/Program.cs for the execution path and menu flow.
  • OF DL.Gui/ViewModels/MainWindowViewModel.cs for GUI startup flow (config -> auth -> users/lists -> selection).
  • OF DL.Gui/Views/MainWindow.axaml for GUI layout and interaction points.
  • OF DL.Gui/Views/AboutWindow.axaml and OF DL.Gui/Views/FaqWindow.axaml for Help menu windows.
  • OF DL.Core/Services/ApiService.cs for OF API calls and header signing.
  • OF DL.Core/Services/DownloadService.cs for downloads and DRM handling.
  • OF DL.Core/Services/DownloadOrchestrationService.cs for creator selection and flow control.
  • OF DL.Core/Widevine/ for CDM key generation and license parsing.
  • OF DL.Core/Models/Config/Config.cs and OF DL.Core/Services/ConfigService.cs for config shape and parsing.
  • OF DL.Core/Services/AuthService.cs for user-facing authentication behavior and browser login flow.
  • OF DL.Core/Helpers/EnvironmentHelper.cs for shared Docker/Windows runtime checks.
  • docs/ for public documentation; update docs whenever user-facing behavior or configuration changes.

Documentation updates for common changes:

  • Config option added/removed/changed in Config or config.conf: update docs/config/all-configuration-options.md ( full spec), docs/config/configuration.md (organized list), and docs/config/custom-filename-formats.md if filename tokens or formats are affected.
  • Authentication flow changes (browser login, legacy methods, required fields): update docs/config/auth.md.
  • CLI/menu flow or download workflow changes: update docs/running-the-program.md.
  • Docker runtime or container flags/paths change: update docs/installation/docker.md.