Go to file
Team Goon f7d82cd207 v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs
Major Features:
-  Complete TPDB scraper implementation with real API calls
-  Auto-fetch on cache miss: search commands now automatically import from TPDB when not found locally
-  Comprehensive documentation (5 markdown files for Bookstack)
-  Import commands for performers, studios, and scenes
-  Fixed JSON type mismatches (aliases array, studio numeric IDs)

Changes:
1. TPDB Scraper (internal/scraper/tpdb/):
   - types.go: Full API response structures with correct types
     - PerformerResponse.Aliases: string → []string (TPDB returns array)
     - StudioResponse.ID: string → int (TPDB returns numeric IDs)
     - SiteInfo.ID: string → int (scenes reference studios by number)
   - mapper.go: Maps TPDB responses to internal models
     - Converts aliases array to comma-separated string
     - Converts numeric studio IDs to strings using strconv.Itoa()
   - scraper.go: Real HTTP client with Bearer token auth
     - SearchPerformers, SearchStudios, SearchScenes implemented
     - GetPerformerByID, GetStudioByID, GetSceneByID implemented

2. CLI Auto-Fetch (cmd/goondex/main.go):
   - performer-search: Auto-fetches from TPDB if local DB empty
   - studio-search: Auto-fetches from TPDB if local DB empty
   - scene-search: Auto-fetches basic metadata (no relationships)
   - Graceful handling of missing TPDB_API_KEY
   - Import → search again to get local IDs

3. Documentation (docs/):
   - INDEX.md: Documentation overview and navigation
   - ARCHITECTURE.md: System design, data flow, component diagrams
   - DATABASE_SCHEMA.md: Complete schema with relationships and indexes
   - CLI_REFERENCE.md: All commands with examples
   - TPDB_INTEGRATION.md: API guide, data mapping, best practices

4. Fixes:
   - .gitignore: Fixed pattern to allow cmd/goondex/* and cmd/goondexd/*
   - README: Updated to reflect TPDB integration and auto-fetch

Testing:
-  performer-search "Riley Reid" - auto-fetched 2 performers, cached
-  studio-search "Brazzers" - auto-fetched 12 studios, cached
-  Aliases now display correctly as comma-separated list
-  Studio IDs properly converted from numeric to string

API Integration:
- Base URL: https://api.theporndb.net
- Authentication: Bearer token via TPDB_API_KEY env var
- Endpoints: /performers, /sites, /scenes
- Rate limiting handled with warnings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 22:04:23 -05:00
cmd v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs 2025-11-14 22:04:23 -05:00
config Initial release: v0.1.0-dev1 2025-11-14 21:37:26 -05:00
docs v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs 2025-11-14 22:04:23 -05:00
internal v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs 2025-11-14 22:04:23 -05:00
.env.example Initial release: v0.1.0-dev1 2025-11-14 21:37:26 -05:00
.gitignore v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs 2025-11-14 22:04:23 -05:00
go.mod Initial release: v0.1.0-dev1 2025-11-14 21:37:26 -05:00
go.sum Initial release: v0.1.0-dev1 2025-11-14 21:37:26 -05:00
README.md v0.1.0-dev2: Full TPDB integration with auto-fetch and comprehensive docs 2025-11-14 22:04:23 -05:00

Goondex

Fast, local-first media indexer for adult content.

Goondex ingests metadata from external sources (ThePornDB, etc.), normalizes it, and stores it in a small SQLite database for quick search via CLI/TUI and background daemon tasks.

Version

v0.1.0-dev2 - TPDB Integration Release

Features (v0.1.0-dev2)

  • SQLite database with WAL mode for performers, studios, scenes, and tags
  • Full TPDB scraper integration with real API calls
  • CLI import commands - Fetch data directly from ThePornDB
  • CLI search commands for local database queries
  • Automatic relationship management (scenes ↔ performers, scenes ↔ tags)
  • Pluggable scraper architecture
  • Configuration via YAML files
  • Stash-inspired metadata resolution strategies (coming in v0.2.x)

Architecture

Scrapers (TPDB, AE, etc.)
    ↓
Metadata Resolver (field strategies, merge rules)
    ↓
SQLite DB (performers, studios, scenes, tags)
    ↓
CLI/TUI + Daemon (search, identify, sync)

Installation

# Clone the repository
git clone https://git.leaktechnologies.dev/stu/Goondex.git
cd Goondex

# Build the CLI
go build -o bin/goondex ./cmd/goondex

# (Optional) Build the daemon
go build -o bin/goondexd ./cmd/goondexd

Configuration

Set your API keys as environment variables:

export TPDB_API_KEY="your-tpdb-api-key"

Edit configuration files in config/:

  • goondex.yml - Runtime settings (database path, cache dir, timeouts)
  • metadata.yml - Field strategies (MERGE/OVERWRITE/IGNORE)
  • source.yml - Scraper sources and priorities

Usage

Quick Start

# 1. Set your TPDB API key
export TPDB_API_KEY="your-api-key-here"

# 2. Import some data from ThePornDB
./bin/goondex import performer "Riley Reid"
./bin/goondex import studio "Brazzers"
./bin/goondex import scene "Big Wet Butts"

# 3. Search your local database
./bin/goondex performer-search "Riley"
./bin/goondex studio-search "Brazzers"
./bin/goondex scene-search "Big Wet"

All Commands

Import from ThePornDB (requires TPDB_API_KEY):

./bin/goondex import performer [query]  # Import performers
./bin/goondex import studio [query]     # Import studios
./bin/goondex import scene [query]      # Import scenes (+ performers, tags, studio)

Search Local Database:

./bin/goondex performer-search [query]  # Search performers
./bin/goondex studio-search [query]     # Search studios
./bin/goondex scene-search [query]      # Search scenes

Other:

./bin/goondex version                   # Show version
./bin/goondex --help                    # Show help

See CLI Reference for complete documentation.

Database Schema

  • performers: id, name, aliases, nationality, country, gender, images, bio
  • studios: id, name, parent_id, images, description
  • tags: id, name
  • scenes: id, title, code, date, studio_id, description, images, director, url
  • scene_performers: junction table for scenes ↔ performers
  • scene_tags: junction table for scenes ↔ tags

Documentation

Comprehensive documentation is available in the docs/ directory:

Roadmap

v0.1.x (Current)

  • CLI search commands
  • SQLite stores for all entities
  • TPDB scraper implementation with real API integration
  • Import commands (performer, studio, scene)
  • Comprehensive documentation
  • Image cache

v0.2.x

  • Identify/import commands
  • TUI list + preview
  • Alias normalization
  • Full-text search (FTS5)

v0.3.x

  • Daemon (goondexd) with schedules
  • Incremental updates
  • Duplicate scene detection
  • Preview sprite generation

v0.4.x

  • Plugin scrapers
  • Headless HTTP API
  • Web UI

Development

# Run tests (when available)
go test ./...

# Build for development
go build -o bin/goondex ./cmd/goondex

# Run without installing
go run ./cmd/goondex performer-search "test"

Contributing

This is a personal project, but contributions are welcome! Please open an issue before submitting large changes.

License

[Your License Here]

Acknowledgments

Inspired by Stash and its metadata identification flow.