Prepare for migration

2026-03-01 13:39:41 +00:00 · 2026-02-05 16:42:47 +02:00 · 2026-02-05 16:42:47 +02:00 · d442b46d7e
commit d442b46d7e
parent dedfdba39b
11 changed files with 451 additions and 0 deletions
--- a/37
+++ b/37
@ -0,0 +1,37 @@
+SCRIPTS_DIR := scripts
+
+.DEFAULT_GOAL := help
+
+.PHONY: help db-dump-all db-dump-folders db-dump-items db-dump-books db-dump-feeds db-dump-settings db-summary
+
+help:
+	@echo "Audiobookshelf Database Migration Tools"
+	@echo ""
+	@echo "Usage: make <target>"
+	@echo ""
+	@echo "Targets:"
+	@echo "  db-summary       - Run full summary of all Docker paths (quick overview)"
+	@echo "  db-dump-all      - Alias for db-summary"
+	@echo "  db-dump-folders  - Dump libraryFolders paths"
+	@echo "  db-dump-items    - Dump libraryItems paths"
+	@echo "  db-dump-books    - Dump books coverPaths"
+	@echo "  db-dump-feeds    - Dump feeds URLs"
+	@echo "  db-dump-settings - Dump settings JSON paths"
+
+db-dump-folders:
+	@bash $(SCRIPTS_DIR)/dump_library_folders.sh
+
+db-dump-items:
+	@bash $(SCRIPTS_DIR)/dump_library_items.sh
+
+db-dump-books:
+	@bash $(SCRIPTS_DIR)/dump_books.sh
+
+db-dump-feeds:
+	@bash $(SCRIPTS_DIR)/dump_feeds.sh
+
+db-dump-settings:
+	@bash $(SCRIPTS_DIR)/dump_settings.sh
+
+db-summary:
+	@bash $(SCRIPTS_DIR)/dump_all.sh
--- a/artifacts/2026-02-05/local_migration.md
+++ b/artifacts/2026-02-05/local_migration.md
@ -0,0 +1,140 @@
+# Local Database Migration Specification
+
+## Overview
+
+This document specifies the automatic path migration process for transferring an Audiobookshelf database from a Docker deployment to a local development environment.
+
+## Problem Statement
+
+The source database was created in a Docker container environment with hardcoded absolute paths. These paths must be automatically detected and remapped to match the local development environment structure.
+
+## Source Database
+
+- **Location**: `/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite`
+- **Version**: 2.32.1 (from settings)
+
+## Identified Path References
+
+### 1. libraryFolders Table
+
+**Table**: `libraryFolders`
+**Columns with paths**: `path`
+
+**Sample data**:
+| id | path | libraryId |
+|----|------|-----------|
+| 9f980819-1371-4c8f-9e7d-a6cbe9ae1ba7 | /audiobooks | a04cbf28-7eb6-4c87-b3e3-421ad8b35923 |
+| 43bf8c8d-07b6-4828-848f-8bb1e3dcca04 | /libraries/romance | dad4448d-77c2-481e-9212-1ffcb4272932 |
+
+**Migration strategy**:
+- Map each unique library folder path to a corresponding local path
+- Preserve the folder structure within each library
+
+### 2. libraryItems Table
+
+**Table**: `libraryItems`
+**Columns with paths**: `path`, `relPath`
+
+**Sample data**:
+| id | path | relPath | title |
+|----|------|---------|-------|
+| 6ec745f9-608e-4556-8f78-b36e2682069b | /audiobooks/A Beginner's Guide to Forever.m4b | A Beginner's Guide to Forever.m4b | A Beginner's Guide to Forever |
+
+**Migration strategy**:
+- The `path` column contains full absolute paths from Docker root
+- The `relPath` column contains paths relative to library folder (less likely to need migration)
+- Update `path` to use local library folder mappings
+
+### 3. books Table
+
+**Table**: `books`
+**Columns with paths**: `coverPath`
+
+**Sample data**:
+| id | coverPath |
+|----|-----------|
+| 68f4e9ca-c8e9-46a1-b667-7b0a409dd72d | /metadata/items/6ec745f9-608e-4556-8f78-b36e2682069b/cover.jpg |
+
+**Migration strategy**:
+- `coverPath` points to `/metadata/items/{libraryItemId}/cover.jpg`
+- May need remapping if local `metadata` directory differs from Docker
+
+### 4. feeds Table
+
+**Table**: `feeds`
+**Columns with paths**: `serverAddress`, `feedURL`, `imageURL`, `siteURL`, `coverPath`
+
+**Migration strategy**:
+- `serverAddress`: The Docker container's server URL (e.g., `http://audiobookshelf:8080`)
+- `feedURL`, `imageURL`, `siteURL`: URLs containing the server address
+- `coverPath`: Local file path to feed cover images
+
+### 5. settings Table
+
+**Table**: `settings`
+**Key with paths**: `server-settings` (JSON value)
+
+**Path settings in JSON**:
+- `backupPath`: Docker path (e.g., `/metadata/backups`)
+- Potentially others in nested JSON structure
+
+## Migration Input/Output
+
+### Input Mapping Format
+
+```yaml
+# path-mapping.yaml
+libraries:
+  /audiobooks: /home/user/audiobooks
+  /libraries/romance: /home/user/libraries/romance
+metadata:
+  source: /metadata
+  target: /home/user/audiobookshelf/metadata
+server:
+  docker_host: http://audiobookshelf:8080
+  local_host: http://localhost:3333
+```
+
+### Output
+
+- Modified SQLite database with all paths updated
+- Migration log with changes made
+- Backup of original database
+
+## Validation Requirements
+
+1. **Path format validation**: Ensure all updated paths follow local filesystem conventions
+2. **Referential integrity**: Verify libraryItems reference valid libraryFolderIds
+3. **URL validation**: Ensure feed URLs use correct local server address
+4. **File existence check** (optional): Verify that mapped paths exist locally
+
+## Implementation Phases
+
+### Phase 1: Path Discovery
+- [ ] Scan all tables for path-like values
+- [ ] Identify all unique paths requiring migration
+- [ ] Categorize paths by type (library folders, metadata, URLs)
+
+### Phase 2: Mapping Configuration
+- [ ] Create mapping configuration file
+- [ ] Define library folder path mappings
+- [ ] Define metadata path mappings
+- [ ] Define server URL mappings
+
+### Phase 3: Migration Script
+- [ ] Implement path update logic for each table
+- [ ] Implement URL update logic for feeds
+- [ ] Implement settings path updates
+- [ ] Add transaction safety with rollback capability
+
+### Phase 4: Validation
+- [ ] Run validation checks on migrated database
+- [ ] Generate migration report
+- [ ] Test database with local Audiobookshelf instance
+
+## Related Files
+
+- **Source database**: `/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite`
+- **Client config**: `/mnt/docker/work/books/audiobookshelf/client/nuxt.config.js`
+- **Server config**: `/mnt/docker/work/books/audiobookshelf/server/Server.js`
+- **Database models**: `/mnt/docker/work/books/audiobookshelf/server/models/`
--- a/artifacts/Makefile
+++ b/artifacts/Makefile
@ -0,0 +1,6 @@
+DATE := $(shell date +%Y-%m-%d)
+
+.PHONY: today
+
+today:
+	mkdir -p $(DATE)
--- a/scripts/Makefile
+++ b/scripts/Makefile
@ -0,0 +1,39 @@
+.PHONY: all library_folders library_items books feeds settings help
+
+all: library_folders library_items books feeds settings
+	@echo "=== Complete dump complete ==="
+
+library_folders:
+	@echo "Dumping library folders..."
+	@bash dump_library_folders.sh
+
+library_items:
+	@echo "Dumping library items..."
+	@bash dump_library_items.sh
+
+books:
+	@echo "Dumping books..."
+	@bash dump_books.sh
+
+feeds:
+	@echo "Dumping feeds..."
+	@bash dump_feeds.sh
+
+settings:
+	@echo "Dumping settings..."
+	@bash dump_settings.sh
+
+summary:
+	@echo "Running full summary..."
+	@bash dump_all.sh
+
+help:
+	@echo "Available targets:"
+	@echo "  make all        - Run all dump scripts"
+	@echo "  make summary    - Run the master summary script"
+	@echo "  make library_folders - Dump libraryFolders table"
+	@echo "  make library_items   - Dump libraryItems table"
+	@echo "  make books       - Dump books table"
+	@echo "  make feeds       - Dump feeds table"
+	@echo "  make settings    - Dump settings table"
+	@echo "  make help        - Show this help message"
--- a/scripts/SCRIPTS.md
+++ b/scripts/SCRIPTS.md
@ -0,0 +1,130 @@
+# Database Debug Scripts
+
+## Overview
+
+Collection of SQLite scripts to analyze the Audiobookshelf database and identify hardcoded Docker paths that need migration.
+
+## Scripts
+
+### dump_library_folders.sh
+Dumps the `libraryFolders` table showing:
+- Full paths to library root folders
+- Library IDs for reference
+
+**Usage:** `bash dump_library_folders.sh`
+
+**Key for migration:** Maps Docker paths like `/audiobooks` to local equivalents.
+
+---
+
+### dump_library_items.sh
+Dumps `libraryItems` table focusing on:
+- Full absolute paths (`path`)
+- Relative paths (`relPath`)
+- Unique path prefixes used across all items
+
+**Usage:** `bash dump_library_items.sh`
+
+**Key for migration:** Identifies which library folder each item belongs to based on path prefix.
+
+---
+
+### dump_books.sh
+Dumps `books` table focusing on:
+- Cover image paths (`coverPath`)
+- Unique cover path prefixes
+
+**Usage:** `bash dump_books.sh`
+
+**Key for migration:** Identifies metadata paths like `/metadata/items/{id}/cover.jpg`.
+
+---
+
+### dump_feeds.sh
+Dumps `feeds` table showing:
+- Server addresses (Docker hostnames)
+- Feed URLs
+- Cover paths for RSS feeds
+
+**Usage:** `bash dump_feeds.sh`
+
+**Key for migration:** Finds Docker hostnames like `http://audiobookshelf:8080` that need local URLs.
+
+---
+
+### dump_settings.sh
+Dumps `settings` table extracting:
+- Path-related values from JSON
+- URLs and server addresses
+- Configuration paths
+
+**Usage:** `bash dump_settings.sh`
+
+**Key for migration:** Finds settings like `backupPath`, server URLs in JSON config.
+
+---
+
+### dump_all.sh (Master Summary)
+Runs all dumps and provides a consolidated view:
+- Library folder paths
+- Path prefixes across items
+- Cover path prefixes
+- Server addresses
+- Settings with paths/URLs
+- Quick scan of all Docker-like paths
+
+**Usage:** `bash dump_all.sh`
+
+---
+
+## Makefile Targets
+
+Run from the `scripts/` directory:
+
+```bash
+cd scripts
+make all           # Run all dumps
+make summary       # Run master summary
+make help          # Show available targets
+```
+
+### Individual Targets
+
+```bash
+make library_folders  # libraryFolders paths
+make library_items    # libraryItems paths
+make books            # books coverPaths
+make feeds            # feeds URLs
+make settings         # settings JSON paths
+```
+
+## Example Output
+
+```
+=== libraryFolders ===
+ID | PATH | LIBRARY_ID
+---|------|-----------
+9f980819-... | /audiobooks | a04cbf28-...
+43bf8c8d-... | /libraries/romance | dad4448d-...
+
+=== Unique Path Prefixes ===
+/audiobooks
+/libraries/romance
+```
+
+## Common Docker Paths Found
+
+| Path Pattern | Description |
+|--------------|-------------|
+| `/audiobooks` | Library root |
+| `/libraries/...` | Additional libraries |
+| `/metadata/items/...` | Item metadata/covers |
+| `/metadata/backups` | Backup directory |
+| `http://audiobookshelf:8080` | Docker service URL |
+
+## Next Steps
+
+1. Run `make summary` to get complete overview
+2. Identify Docker paths specific to your deployment
+3. Create path mapping configuration
+4. Run migration script to update paths
--- a/scripts/dump_all.sh
+++ b/scripts/dump_all.sh
@ -0,0 +1,39 @@
+#!/bin/bash
+# Master summary script - find all Docker paths in the database
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=============================================="
+echo "DOCKER PATH SUMMARY"
+echo "=============================================="
+
+echo ""
+echo ">>> 1. LIBRARY FOLDERS (libraryFolders.path)"
+echo "=============================================="
+sqlite3 -header -column "$DB_PATH" "SELECT id, path, libraryId FROM libraryFolders;"
+
+echo ""
+echo ">>> 2. LIBRARY ITEMS - Path Prefixes"
+echo "=============================================="
+sqlite3 "$DB_PATH" "SELECT DISTINCT SUBSTR(path, 1, INSTR(path || '/', '/') - 1) AS prefix FROM libraryItems ORDER BY prefix;" | sort -u
+
+echo ""
+echo ">>> 3. BOOKS - Cover Path Prefixes"
+echo "=============================================="
+sqlite3 "$DB_PATH" "SELECT DISTINCT SUBSTR(coverPath, 1, INSTR(coverPath || '/', '/') - 1) AS prefix FROM books WHERE coverPath IS NOT NULL ORDER BY prefix;" | sort -u
+
+echo ""
+echo ">>> 4. FEEDS - Server Addresses (Docker hostnames)"
+echo "=============================================="
+sqlite3 "$DB_PATH" "SELECT DISTINCT serverAddress FROM feeds WHERE serverAddress IS NOT NULL;"
+
+echo ""
+echo ">>> 5. SETTINGS - Path-related Values"
+echo "=============================================="
+sqlite3 "$DB_PATH" "SELECT key, SUBSTR(value, 1, 200) FROM settings WHERE value LIKE '%/metadata/%' OR value LIKE '%/audiobooks%' OR value LIKE 'http://%' OR value LIKE 'https://%';"
+
+echo ""
+echo ">>> 6. ALL LIKELY DOCKER PATHS (Quick scan)"
+echo "=============================================="
+echo "Scanning for paths starting with '/' or URLs..."
+sqlite3 "$DB_PATH" "SELECT 'libraryFolders.path' AS source, path AS value FROM libraryFolders WHERE path LIKE '/%' UNION ALL SELECT 'libraryItems.path', path FROM libraryItems WHERE path LIKE '/%' UNION ALL SELECT 'books.coverPath', coverPath FROM books WHERE coverPath LIKE '/%' UNION ALL SELECT 'feeds.serverAddress', serverAddress FROM feeds WHERE serverAddress LIKE 'http://%' OR serverAddress LIKE 'https://%';" 2>/dev/null | head -50
--- a/scripts/dump_books.sh
+++ b/scripts/dump_books.sh
@ -0,0 +1,13 @@
+#!/bin/bash
+# Dump books table cover paths
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=== books.coverPath ==="
+echo "ID | COVER_PATH"
+echo "---|-----------"
+sqlite3 -header -column "$DB_PATH" "SELECT id, coverPath FROM books LIMIT 50;"
+
+echo ""
+echo "=== Unique Cover Path Prefixes ==="
+sqlite3 "$DB_PATH" "SELECT DISTINCT SUBSTR(coverPath, 1, INSTR(coverPath || '/', '/') - 1) FROM books WHERE coverPath IS NOT NULL;" | sort -u
--- a/scripts/dump_feeds.sh
+++ b/scripts/dump_feeds.sh
@ -0,0 +1,13 @@
+#!/bin/bash
+# Dump feeds table - URLs and server addresses
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=== feeds (URLs and Addresses) ==="
+echo "ID | SERVER_ADDRESS | FEED_URL | COVER_PATH"
+echo "---|----------------|----------|-----------"
+sqlite3 -header -column "$DB_PATH" "SELECT id, serverAddress, feedURL, coverPath FROM feeds;"
+
+echo ""
+echo "=== Unique Server Addresses ==="
+sqlite3 "$DB_PATH" "SELECT DISTINCT serverAddress FROM feeds WHERE serverAddress IS NOT NULL;"
--- a/scripts/dump_library_folders.sh
+++ b/scripts/dump_library_folders.sh
@ -0,0 +1,9 @@
+#!/bin/bash
+# Dump libraryFolders table for human consumption
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=== libraryFolders ==="
+echo "ID | PATH | LIBRARY_ID"
+echo "---|------|-----------"
+sqlite3 -header -column "$DB_PATH" "SELECT id, path, libraryId FROM libraryFolders;"
--- a/scripts/dump_library_items.sh
+++ b/scripts/dump_library_items.sh
@ -0,0 +1,13 @@
+#!/bin/bash
+# Dump libraryItems table focusing on paths
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=== libraryItems (Paths) ==="
+echo "ID | PATH | REL_PATH"
+echo "---|------|---------"
+sqlite3 -header -column "$DB_PATH" "SELECT id, path, relPath FROM libraryItems LIMIT 100;"
+
+echo ""
+echo "=== Unique Path Prefixes ==="
+sqlite3 "$DB_PATH" "SELECT DISTINCT SUBSTR(path, 1, INSTR(path || '/', '/') - 1) FROM libraryItems;" | sort -u
--- a/scripts/dump_settings.sh
+++ b/scripts/dump_settings.sh
@ -0,0 +1,12 @@
+#!/bin/bash
+# Dump settings - extract path-related values from JSON
+
+DB_PATH="/mnt/docker/work/books/audiobookshelf/config/absdatabase.sqlite"
+
+echo "=== settings (Path-related JSON values) ==="
+
+# Extract and display server-settings JSON with key paths highlighted
+sqlite3 "$DB_PATH" "SELECT key, value FROM settings;" | while read -r key value; do
+    echo "--- $key ---"
+    echo "$value" | python3 -c "import sys, json; d=json.load(sys.stdin); [print(f'  {k}: {v}') for k,v in d.items() if 'path' in k.lower() or 'url' in k.lower() or 'address' in k.lower()]" 2>/dev/null || echo "  (No path-related keys found or JSON parse error)"
+done