f7d82cd207
Major Features: - ✅ Complete TPDB scraper implementation with real API calls - ✅ Auto-fetch on cache miss: search commands now automatically import from TPDB when not found locally - ✅ Comprehensive documentation (5 markdown files for Bookstack) - ✅ Import commands for performers, studios, and scenes - ✅ Fixed JSON type mismatches (aliases array, studio numeric IDs) Changes: 1. TPDB Scraper (internal/scraper/tpdb/): - types.go: Full API response structures with correct types - PerformerResponse.Aliases: string → []string (TPDB returns array) - StudioResponse.ID: string → int (TPDB returns numeric IDs) - SiteInfo.ID: string → int (scenes reference studios by number) - mapper.go: Maps TPDB responses to internal models - Converts aliases array to comma-separated string - Converts numeric studio IDs to strings using strconv.Itoa() - scraper.go: Real HTTP client with Bearer token auth - SearchPerformers, SearchStudios, SearchScenes implemented - GetPerformerByID, GetStudioByID, GetSceneByID implemented 2. CLI Auto-Fetch (cmd/goondex/main.go): - performer-search: Auto-fetches from TPDB if local DB empty - studio-search: Auto-fetches from TPDB if local DB empty - scene-search: Auto-fetches basic metadata (no relationships) - Graceful handling of missing TPDB_API_KEY - Import → search again to get local IDs 3. Documentation (docs/): - INDEX.md: Documentation overview and navigation - ARCHITECTURE.md: System design, data flow, component diagrams - DATABASE_SCHEMA.md: Complete schema with relationships and indexes - CLI_REFERENCE.md: All commands with examples - TPDB_INTEGRATION.md: API guide, data mapping, best practices 4. Fixes: - .gitignore: Fixed pattern to allow cmd/goondex/* and cmd/goondexd/* - README: Updated to reflect TPDB integration and auto-fetch Testing: - ✅ performer-search "Riley Reid" - auto-fetched 2 performers, cached - ✅ studio-search "Brazzers" - auto-fetched 12 studios, cached - ✅ Aliases now display correctly as comma-separated list - ✅ Studio IDs properly converted from numeric to string API Integration: - Base URL: https://api.theporndb.net - Authentication: Bearer token via TPDB_API_KEY env var - Endpoints: /performers, /sites, /scenes - Rate limiting handled with warnings 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
174 lines
4.5 KiB
Markdown
174 lines
4.5 KiB
Markdown
# Goondex
|
|
|
|
**Fast, local-first media indexer for adult content.**
|
|
|
|
Goondex ingests metadata from external sources (ThePornDB, etc.), normalizes it, and stores it in a small SQLite database for quick search via CLI/TUI and background daemon tasks.
|
|
|
|
## Version
|
|
|
|
**v0.1.0-dev2** - TPDB Integration Release
|
|
|
|
## Features (v0.1.0-dev2)
|
|
|
|
- ✅ SQLite database with WAL mode for performers, studios, scenes, and tags
|
|
- ✅ **Full TPDB scraper integration** with real API calls
|
|
- ✅ **CLI import commands** - Fetch data directly from ThePornDB
|
|
- ✅ CLI search commands for local database queries
|
|
- ✅ Automatic relationship management (scenes ↔ performers, scenes ↔ tags)
|
|
- ✅ Pluggable scraper architecture
|
|
- ✅ Configuration via YAML files
|
|
- ⏳ Stash-inspired metadata resolution strategies (coming in v0.2.x)
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Scrapers (TPDB, AE, etc.)
|
|
↓
|
|
Metadata Resolver (field strategies, merge rules)
|
|
↓
|
|
SQLite DB (performers, studios, scenes, tags)
|
|
↓
|
|
CLI/TUI + Daemon (search, identify, sync)
|
|
```
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://git.leaktechnologies.dev/stu/Goondex.git
|
|
cd Goondex
|
|
|
|
# Build the CLI
|
|
go build -o bin/goondex ./cmd/goondex
|
|
|
|
# (Optional) Build the daemon
|
|
go build -o bin/goondexd ./cmd/goondexd
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Set your API keys as environment variables:
|
|
|
|
```bash
|
|
export TPDB_API_KEY="your-tpdb-api-key"
|
|
```
|
|
|
|
Edit configuration files in `config/`:
|
|
- `goondex.yml` - Runtime settings (database path, cache dir, timeouts)
|
|
- `metadata.yml` - Field strategies (MERGE/OVERWRITE/IGNORE)
|
|
- `source.yml` - Scraper sources and priorities
|
|
|
|
## Usage
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
# 1. Set your TPDB API key
|
|
export TPDB_API_KEY="your-api-key-here"
|
|
|
|
# 2. Import some data from ThePornDB
|
|
./bin/goondex import performer "Riley Reid"
|
|
./bin/goondex import studio "Brazzers"
|
|
./bin/goondex import scene "Big Wet Butts"
|
|
|
|
# 3. Search your local database
|
|
./bin/goondex performer-search "Riley"
|
|
./bin/goondex studio-search "Brazzers"
|
|
./bin/goondex scene-search "Big Wet"
|
|
```
|
|
|
|
### All Commands
|
|
|
|
**Import from ThePornDB** (requires `TPDB_API_KEY`):
|
|
```bash
|
|
./bin/goondex import performer [query] # Import performers
|
|
./bin/goondex import studio [query] # Import studios
|
|
./bin/goondex import scene [query] # Import scenes (+ performers, tags, studio)
|
|
```
|
|
|
|
**Search Local Database**:
|
|
```bash
|
|
./bin/goondex performer-search [query] # Search performers
|
|
./bin/goondex studio-search [query] # Search studios
|
|
./bin/goondex scene-search [query] # Search scenes
|
|
```
|
|
|
|
**Other**:
|
|
```bash
|
|
./bin/goondex version # Show version
|
|
./bin/goondex --help # Show help
|
|
```
|
|
|
|
See [CLI Reference](docs/CLI_REFERENCE.md) for complete documentation.
|
|
|
|
## Database Schema
|
|
|
|
- **performers**: id, name, aliases, nationality, country, gender, images, bio
|
|
- **studios**: id, name, parent_id, images, description
|
|
- **tags**: id, name
|
|
- **scenes**: id, title, code, date, studio_id, description, images, director, url
|
|
- **scene_performers**: junction table for scenes ↔ performers
|
|
- **scene_tags**: junction table for scenes ↔ tags
|
|
|
|
## Documentation
|
|
|
|
Comprehensive documentation is available in the [`docs/`](docs/) directory:
|
|
|
|
- **[Index](docs/INDEX.md)** - Documentation overview
|
|
- **[Architecture](docs/ARCHITECTURE.md)** - System design and components
|
|
- **[Database Schema](docs/DATABASE_SCHEMA.md)** - Complete schema reference
|
|
- **[CLI Reference](docs/CLI_REFERENCE.md)** - All commands and usage
|
|
- **[TPDB Integration](docs/TPDB_INTEGRATION.md)** - ThePornDB API guide
|
|
|
|
## Roadmap
|
|
|
|
### v0.1.x (Current)
|
|
- [x] CLI search commands
|
|
- [x] SQLite stores for all entities
|
|
- [x] **TPDB scraper implementation with real API integration**
|
|
- [x] **Import commands (performer, studio, scene)**
|
|
- [x] **Comprehensive documentation**
|
|
- [ ] Image cache
|
|
|
|
### v0.2.x
|
|
- [ ] Identify/import commands
|
|
- [ ] TUI list + preview
|
|
- [ ] Alias normalization
|
|
- [ ] Full-text search (FTS5)
|
|
|
|
### v0.3.x
|
|
- [ ] Daemon (goondexd) with schedules
|
|
- [ ] Incremental updates
|
|
- [ ] Duplicate scene detection
|
|
- [ ] Preview sprite generation
|
|
|
|
### v0.4.x
|
|
- [ ] Plugin scrapers
|
|
- [ ] Headless HTTP API
|
|
- [ ] Web UI
|
|
|
|
## Development
|
|
|
|
```bash
|
|
# Run tests (when available)
|
|
go test ./...
|
|
|
|
# Build for development
|
|
go build -o bin/goondex ./cmd/goondex
|
|
|
|
# Run without installing
|
|
go run ./cmd/goondex performer-search "test"
|
|
```
|
|
|
|
## Contributing
|
|
|
|
This is a personal project, but contributions are welcome! Please open an issue before submitting large changes.
|
|
|
|
## License
|
|
|
|
[Your License Here]
|
|
|
|
## Acknowledgments
|
|
|
|
Inspired by [Stash](https://github.com/stashapp/stash) and its metadata identification flow.
|