Goondex/docs/TPDB_INTEGRATION.md

# ThePornDB Integration Guide

## Overview

Goondex integrates with [ThePornDB](https://theporndb.net) (TPDB) to fetch high-quality metadata for performers, studios, and scenes. TPDB is a community-driven database similar to IMDB but for adult content.

## Prerequisites

### 1. TPDB Account

1. Register at https://theporndb.net/register
2. Verify your email
3. Log in to your account

### 2. API Token

1. Navigate to https://theporndb.net/user/api-tokens
2. Click "Create New Token"
3. Give it a name (e.g., "Goondex")
4. Copy the generated token
5. **Save it securely** - it won't be shown again

### 3. Set Environment Variable

```bash
# Linux/macOS (add to ~/.bashrc or ~/.zshrc for persistence)
export TPDB_API_KEY="your-api-key-here"

# Windows (PowerShell)
$env:TPDB_API_KEY="your-api-key-here"

# Windows (CMD)
set TPDB_API_KEY=your-api-key-here
```

## API Endpoints

Goondex uses the following TPDB API endpoints:

| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/performers` | GET | Search for performers |
| `/performers/{id}` | GET | Get performer by ID |
| `/sites` | GET | Search for studios/sites |
| `/sites/{id}` | GET | Get studio by ID |
| `/scenes` | GET | Search for scenes |
| `/scenes/{id}` | GET | Get scene by ID |

**Base URL**: `https://api.theporndb.net`

**Authentication**: Bearer token in `Authorization` header

## Data Mapping

### Performer Fields

| TPDB Field | Goondex Field | Notes |
|------------|---------------|-------|
| `id` | `source_id` | TPDB ID stored as source reference |
| `name` | `name` | Primary name |
| `aliases` | `aliases` | Comma-separated alternative names |
| `nationality` | `nationality` + `country` | Both fields populated |
| `gender` | `gender` | male/female/trans/other |
| `image` | `image_url` | Poster image URL |
| `bio` | `bio` | Biography text |

**Additional TPDB fields** (available but not currently stored):
- `birthday`, `astrology`, `birthplace`
- `ethnicity`, `eye_color`, `hair_color`
- `height`, `weight`, `measurements`
- `tattoo_description`, `piercing_description`
- `boob_job`, `active`

### Studio Fields

| TPDB Field | Goondex Field | Notes |
|------------|---------------|-------|
| `id` | `source_id` | TPDB ID |
| `name` | `name` | Studio/site name |
| `description` | `description` | About the studio |
| `logo` | `image_url` | Logo image URL |
| `parent.id` | `parent_id` | Parent network ID (not yet implemented) |
| `url` | - | Currently stored in description |

### Scene Fields

| TPDB Field | Goondex Field | Notes |
|------------|---------------|-------|
| `id` | `source_id` | TPDB ID |
| `uuid` | - | Not stored |
| `title` | `title` | Scene title |
| `description` | `description` | Scene synopsis |
| `url` | `url` | Scene URL |
| `date` | `date` | Release date (ISO 8601) |
| `image` | `image_url` | Cover image URL |
| `director` | `director` | Director name |
| `code` | `code` | DVD code or scene identifier |
| `site.id` | `studio_id` | References studios table |
| `performers[]` | Scene-performer relationship | Many-to-many |
| `tags[]` | Scene-tag relationship | Many-to-many |

**Additional TPDB fields** (available but not currently stored):
- `poster`, `duration`

## Usage Examples

### Import a Performer

```bash
# Search and import Riley Reid
export TPDB_API_KEY="your-key"
goondex import performer "Riley Reid"
```

**What happens**:
1. Goondex queries TPDB API: `GET /performers?q=Riley+Reid`
2. TPDB returns matching performers (usually 1 exact match)
3. Goondex maps TPDB fields to internal model
4. Data is inserted into local SQLite database
5. Local ID is assigned and reported

### Import a Studio

```bash
# Import Brazzers
goondex import studio "Brazzers"
```

**What happens**:
1. Query: `GET /sites?q=Brazzers`
2. TPDB returns matching sites
3. Studios are created in local database

### Import a Scene

```bash
# Import a scene
goondex import scene "Big Wet Butts 24"
```

**What happens**:
1. Query: `GET /scenes?q=Big+Wet+Butts+24`
2. TPDB returns matching scenes with embedded performers, studio, and tags
3. **Studio** is imported/updated first
4. **Scene** is created with reference to studio
5. **Performers** are imported/updated
6. **Tags** are imported/updated
7. **Relationships** are created in junction tables

## API Rate Limiting

TPDB implements rate limiting to prevent abuse.

### Current Limits (as of 2024)
- **Requests per minute**: ~60
- **Requests per hour**: ~1000

### Best Practices

1. **Batch imports**: Import multiple items at once instead of one-by-one
2. **Cache locally**: Don't re-import already imported items
3. **Error handling**: Implement exponential backoff on 429 errors

### Handling Rate Limits

If you receive a `429 Too Many Requests` response:

```bash
# Wait 60 seconds
sleep 60

# Retry the command
goondex import performer "Riley Reid"
```

## Response Format

### Standard Response Wrapper

```json
{
  "data": { ... },  // Single object or array
  "meta": {         // Optional pagination
    "current_page": 1,
    "from": 1,
    "last_page": 5,
    "per_page": 25,
    "to": 25,
    "total": 100
  }
}
```

### Performer Response Example

```json
{
  "data": {
    "id": "12345",
    "name": "Riley Reid",
    "slug": "riley-reid",
    "gender": "female",
    "aliases": "Paige Riley",
    "nationality": "US",
    "image": "https://cdn.theporndb.net/performers/riley-reid.jpg",
    "bio": "Riley Reid is an American adult film actress..."
  }
}
```

### Scene Response Example

```json
{
  "data": {
    "id": "54321",
    "uuid": "abc-def-123",
    "title": "Big Wet Butts 24",
    "date": "2024-01-15",
    "description": "Riley Reid stars in...",
    "code": "BWB-024",
    "image": "https://cdn.theporndb.net/scenes/bwb-024.jpg",
    "site": {
      "id": "100",
      "name": "Brazzers",
      "url": "https://brazzers.com"
    },
    "performers": [
      {
        "id": "12345",
        "name": "Riley Reid",
        "gender": "female"
      }
    ],
    "tags": [
      {
        "id": "1",
        "name": "Anal",
        "slug": "anal"
      }
    ]
  }
}
```

## Error Handling

### Common HTTP Status Codes

| Code | Meaning | Solution |
|------|---------|----------|
| 200 | Success | No action needed |
| 401 | Unauthorized | Check API key |
| 404 | Not Found | Item doesn't exist in TPDB |
| 429 | Too Many Requests | Wait and retry |
| 500 | Server Error | TPDB issue, wait and retry |

### Goondex Error Messages

```bash
# Missing API key
Error: TPDB_API_KEY environment variable is not set

# API error
Error: failed to search TPDB: API returned status 401

# No results
No performers found on TPDB
```

## Advanced Usage

### Searching by ID

While Goondex doesn't currently expose this via CLI, you can fetch by TPDB ID programmatically by importing the TPDB scraper package and calling GetPerformerByID, GetStudioByID, or GetSceneByID.

### Pagination

TPDB returns paginated results. Goondex currently fetches only the first page (default 25 results). Future versions will support pagination.

## Data Quality

### TPDB Strengths
- ✅ High-quality, curated performer data
- ✅ Comprehensive scene metadata
- ✅ Studio/network relationships
- ✅ Active community maintenance
- ✅ Regular updates

### Known Limitations
- ⚠️ Not all performers have complete bio data
- ⚠️ Some older scenes may have limited metadata
- ⚠️ Parent studio relationships not always populated
- ⚠️ Image URLs may expire or change

## Future Enhancements

### v0.2.x
- **Incremental updates**: Re-import to update changed data
- **Image caching**: Download and cache images locally
- **Pagination support**: Fetch all pages of results

### v0.3.x
- **Scheduled syncing**: Automatic daily/weekly updates
- **Duplicate detection**: Merge duplicate entries
- **Conflict resolution**: Handle data conflicts intelligently

### v0.4.x
- **Multi-source priority**: Combine TPDB with other sources
- **Manual overrides**: User-edited fields protected from updates

## API Documentation

Official TPDB API documentation:
- https://api.theporndb.net/docs

## Support

For TPDB-related issues:
- **TPDB Discord**: https://discord.gg/theporndb
- **TPDB GitHub**: https://github.com/ThePornDatabase