Major Features: - ✅ Complete TPDB scraper implementation with real API calls - ✅ Auto-fetch on cache miss: search commands now automatically import from TPDB when not found locally - ✅ Comprehensive documentation (5 markdown files for Bookstack) - ✅ Import commands for performers, studios, and scenes - ✅ Fixed JSON type mismatches (aliases array, studio numeric IDs) Changes: 1. TPDB Scraper (internal/scraper/tpdb/): - types.go: Full API response structures with correct types - PerformerResponse.Aliases: string → []string (TPDB returns array) - StudioResponse.ID: string → int (TPDB returns numeric IDs) - SiteInfo.ID: string → int (scenes reference studios by number) - mapper.go: Maps TPDB responses to internal models - Converts aliases array to comma-separated string - Converts numeric studio IDs to strings using strconv.Itoa() - scraper.go: Real HTTP client with Bearer token auth - SearchPerformers, SearchStudios, SearchScenes implemented - GetPerformerByID, GetStudioByID, GetSceneByID implemented 2. CLI Auto-Fetch (cmd/goondex/main.go): - performer-search: Auto-fetches from TPDB if local DB empty - studio-search: Auto-fetches from TPDB if local DB empty - scene-search: Auto-fetches basic metadata (no relationships) - Graceful handling of missing TPDB_API_KEY - Import → search again to get local IDs 3. Documentation (docs/): - INDEX.md: Documentation overview and navigation - ARCHITECTURE.md: System design, data flow, component diagrams - DATABASE_SCHEMA.md: Complete schema with relationships and indexes - CLI_REFERENCE.md: All commands with examples - TPDB_INTEGRATION.md: API guide, data mapping, best practices 4. Fixes: - .gitignore: Fixed pattern to allow cmd/goondex/* and cmd/goondexd/* - README: Updated to reflect TPDB integration and auto-fetch Testing: - ✅ performer-search "Riley Reid" - auto-fetched 2 performers, cached - ✅ studio-search "Brazzers" - auto-fetched 12 studios, cached - ✅ Aliases now display correctly as comma-separated list - ✅ Studio IDs properly converted from numeric to string API Integration: - Base URL: https://api.theporndb.net - Authentication: Bearer token via TPDB_API_KEY env var - Endpoints: /performers, /sites, /scenes - Rate limiting handled with warnings 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| cmd | ||
| config | ||
| docs | ||
| internal | ||
| .env.example | ||
| .gitignore | ||
| go.mod | ||
| go.sum | ||
| README.md | ||
Goondex
Fast, local-first media indexer for adult content.
Goondex ingests metadata from external sources (ThePornDB, etc.), normalizes it, and stores it in a small SQLite database for quick search via CLI/TUI and background daemon tasks.
Version
v0.1.0-dev2 - TPDB Integration Release
Features (v0.1.0-dev2)
- ✅ SQLite database with WAL mode for performers, studios, scenes, and tags
- ✅ Full TPDB scraper integration with real API calls
- ✅ CLI import commands - Fetch data directly from ThePornDB
- ✅ CLI search commands for local database queries
- ✅ Automatic relationship management (scenes ↔ performers, scenes ↔ tags)
- ✅ Pluggable scraper architecture
- ✅ Configuration via YAML files
- ⏳ Stash-inspired metadata resolution strategies (coming in v0.2.x)
Architecture
Scrapers (TPDB, AE, etc.)
↓
Metadata Resolver (field strategies, merge rules)
↓
SQLite DB (performers, studios, scenes, tags)
↓
CLI/TUI + Daemon (search, identify, sync)
Installation
# Clone the repository
git clone https://git.leaktechnologies.dev/stu/Goondex.git
cd Goondex
# Build the CLI
go build -o bin/goondex ./cmd/goondex
# (Optional) Build the daemon
go build -o bin/goondexd ./cmd/goondexd
Configuration
Set your API keys as environment variables:
export TPDB_API_KEY="your-tpdb-api-key"
Edit configuration files in config/:
goondex.yml- Runtime settings (database path, cache dir, timeouts)metadata.yml- Field strategies (MERGE/OVERWRITE/IGNORE)source.yml- Scraper sources and priorities
Usage
Quick Start
# 1. Set your TPDB API key
export TPDB_API_KEY="your-api-key-here"
# 2. Import some data from ThePornDB
./bin/goondex import performer "Riley Reid"
./bin/goondex import studio "Brazzers"
./bin/goondex import scene "Big Wet Butts"
# 3. Search your local database
./bin/goondex performer-search "Riley"
./bin/goondex studio-search "Brazzers"
./bin/goondex scene-search "Big Wet"
All Commands
Import from ThePornDB (requires TPDB_API_KEY):
./bin/goondex import performer [query] # Import performers
./bin/goondex import studio [query] # Import studios
./bin/goondex import scene [query] # Import scenes (+ performers, tags, studio)
Search Local Database:
./bin/goondex performer-search [query] # Search performers
./bin/goondex studio-search [query] # Search studios
./bin/goondex scene-search [query] # Search scenes
Other:
./bin/goondex version # Show version
./bin/goondex --help # Show help
See CLI Reference for complete documentation.
Database Schema
- performers: id, name, aliases, nationality, country, gender, images, bio
- studios: id, name, parent_id, images, description
- tags: id, name
- scenes: id, title, code, date, studio_id, description, images, director, url
- scene_performers: junction table for scenes ↔ performers
- scene_tags: junction table for scenes ↔ tags
Documentation
Comprehensive documentation is available in the docs/ directory:
- Index - Documentation overview
- Architecture - System design and components
- Database Schema - Complete schema reference
- CLI Reference - All commands and usage
- TPDB Integration - ThePornDB API guide
Roadmap
v0.1.x (Current)
- CLI search commands
- SQLite stores for all entities
- TPDB scraper implementation with real API integration
- Import commands (performer, studio, scene)
- Comprehensive documentation
- Image cache
v0.2.x
- Identify/import commands
- TUI list + preview
- Alias normalization
- Full-text search (FTS5)
v0.3.x
- Daemon (goondexd) with schedules
- Incremental updates
- Duplicate scene detection
- Preview sprite generation
v0.4.x
- Plugin scrapers
- Headless HTTP API
- Web UI
Development
# Run tests (when available)
go test ./...
# Build for development
go build -o bin/goondex ./cmd/goondex
# Run without installing
go run ./cmd/goondex performer-search "test"
Contributing
This is a personal project, but contributions are welcome! Please open an issue before submitting large changes.
License
[Your License Here]
Acknowledgments
Inspired by Stash and its metadata identification flow.