Fix scanner isInvalid flag and recursive path symlink bug

artifacts/2026-02-22/scanner_missing_invalid_fix.md

@@ -0,0 +1,40 @@
+# Artifact Specification: Scanner Missing and Invalid Flag Reset Fix
+
+## 1. Overview
+This specification details the fixes implemented to resolve two major bugs regarding how the application's library scanner processes files and updates item statuses:
+1. **Stuck "Issues" Status:** Books and podcasts migrated from v1 databases could have a legacy `isInvalid: true` flag. Normal scans and rescans successfully cleared `isMissing` when files were found, but `isInvalid` was never cleared. This forced items to permanently show up as having "issues" unless the user performed a heavy "Consolidate" action. 
+2. **Scanner Path Duplication (Symlink Bug):** When libraries were sourced from symlinked directories, `fileUtils.recurseFiles` would inadvertently duplicate paths because the underlying `fs.realpath` resolution of the symlink caused a string mismatch when stripping the root relative path.
+
+## 2. API & Data Contracts
+- **Data Contracts**: No changes to the database schemas. The existing `isMissing` and `isInvalid` schema flags are preserved.
+- **Scanner Output**: `LibraryItemScanData`, `BookScanner`, and `PodcastScanner` will actively push `isInvalid: false` to the DB when files are detected. 
+
+## 3. Traceability (Files Modified)
+
+| Category | File | Change Description |
+| :--- | :--- | :--- |
+| **Backend** | `server/scanner/LibraryItemScanData.js` | Updated `checkLibraryItemData` to assert `isMissing: false` and `isInvalid: false` if either was `true` |
+| **Backend** | `server/scanner/BookScanner.js` | Updated `rescanExistingBookLibraryItem` to assert `isMissing: false` and `isInvalid: false` if either was `true` |
+| **Backend** | `server/scanner/PodcastScanner.js` | Updated `rescanExistingPodcastLibraryItem` to assert `isMissing: false` and `isInvalid: false` if either was `true` |
+| **Backend** | `server/utils/fileUtils.js` | Refactored `recurseFiles` to use `fs.realpath()` for `relPathToReplace` before string manipulation to ensure correct substring replacement inside symlinks |
+
+## 4. Architectural Decisions
+### Scanner Flags
+- **Decision:** Explicitly check `|| existingLibraryItem.isInvalid` before attempting to unset the flags.
+- **Reasoning:** Rather than wiping `isInvalid` indiscriminately, we check if the flag is actively `true` and the filesystem says the files exist and load correctly. If so, the item is restored to a fully healthy state. This aligns behavior seamlessly with the `isMissing` logic.
+
+### Symlink Substring Removal
+- **Decision:** Use `fs.realpath(relPathToReplace)` before stripping prefixes. 
+- **Reasoning:** The `recursiveReaddirAsync` utility was configured with `realPath: true`. It inherently resolves all symlinks for deep-scanned objects. When `fileUtils` tried to truncate the root path `relPathToReplace` using string replacement, it failed on symlinked paths because `item.fullname` was resolved but `relPathToReplace` was not. Ensuring `relPathToReplace` also runs through `fs.realpath` fixes the root mismatch.
+
+## 5. Verification Plan
+1. **Invalid Items Recovery:**
+   - Restart server.
+   - Re-scan an item that was historically tagged "invalid" via single-file root folders.
+   - Verify the UI "issues" count actively clears.
+2. **Symlink Scans:**
+   - Trigger a scan on a directory mapped via symlinks (e.g., `/home/user/...` -> `/mnt/docker/...`).
+   - Validate that item metadata and paths register with the scanner correctly without dumping duplicate paths (`/home/user/mnt/docker/user...`) in logs or the DB.
+
+## 6. Limitations & Future Work
+- Legacy v1 `isInvalid` items are still expected to exist in long-lived databases until they are actively rescanned or a full-library scan captures them under the new rules. No background database migration was run to mass-clear them, to ensure the file scanner correctly assesses integrity.

artifacts/index.md

@@ -27,6 +27,7 @@ This index provides a quick reference for specification and documentation files
 | **2026-02-22** | [match_default_behavior.md](2026-02-22/match_default_behavior.md) | Specification for the new default "Direct Apply" match behavior and Review & Edit button. |
 | **2026-02-22** | [player_keyboard_shortcuts.md](2026-02-22/player_keyboard_shortcuts.md) | Specification for new player keyboard shortcuts including major skip and chapter jumps. |
 | **2026-02-22** | [normalized_title_filter.md](2026-02-22/normalized_title_filter.md) | Specification for the Normalized Title / Duplicate Title filter implementations and database columns. |
+| **2026-02-22** | [scanner_missing_invalid_fix.md](2026-02-22/scanner_missing_invalid_fix.md) | Specification for fixing `isMissing` and legacy `isInvalid` flag reset logic during scans, and fixing symlink path duplication in `fileUtils`. |
 | **General** | [docs/consolidate_feature.md](docs/consolidate_feature.md) | Comprehensive documentation for the "Consolidate" feature, including conflict resolution and technical details. |
 | **General** | [docs/item_restructuring_guide.md](docs/item_restructuring_guide.md) | Guide for Moving, Merging, and Splitting (Promoting) library items. |
 | **General** | [docs/metadata_management_tools.md](docs/metadata_management_tools.md) | Documentation for Reset Metadata and Batch Reset operations. |

server/scanner/BookScanner.js

@@ -404,9 +404,15 @@ class BookScanner {
         existingLibraryItem.isMissing = true
         libraryItemUpdated = true
       }
-    } else if (existingLibraryItem.isMissing) {
+    } else if (existingLibraryItem.isMissing || existingLibraryItem.isInvalid) {
-      libraryScan.addLog(LogLevel.INFO, `Book "${bookMetadata.title}" was missing but now has media files. Setting library item as NOT missing`)
+      if (existingLibraryItem.isMissing) {
-      existingLibraryItem.isMissing = false
+        libraryScan.addLog(LogLevel.INFO, `Book "${bookMetadata.title}" was missing but now has media files. Setting library item as NOT missing`)
+        existingLibraryItem.isMissing = false
+      }
+      if (existingLibraryItem.isInvalid) {
+        libraryScan.addLog(LogLevel.INFO, `Book "${bookMetadata.title}" was invalid but now has media files. Setting library item as NOT invalid`)
+        existingLibraryItem.isInvalid = false
+      }
       libraryItemUpdated = true
     }
 

server/scanner/LibraryItemScanData.js

@@ -211,9 +211,10 @@ class LibraryItemScanData {
       existingLibraryItem.ctime = this.ctimeMs
       this.hasChanges = true
     }
-    if (existingLibraryItem.isMissing) {
+    if (existingLibraryItem.isMissing || existingLibraryItem.isInvalid) {
-      libraryScan.addLog(LogLevel.DEBUG, `Library item "${existingLibraryItem.relPath}" was missing but now found`)
+      libraryScan.addLog(LogLevel.DEBUG, `Library item "${existingLibraryItem.relPath}" was missing or invalid but now found`)
       existingLibraryItem.isMissing = false
+      existingLibraryItem.isInvalid = false
       this.hasChanges = true
     }
 

server/scanner/PodcastScanner.js

@@ -239,6 +239,18 @@ class PodcastScanner {
       libraryItemUpdated = true
     }
 
+    if (existingLibraryItem.isMissing || existingLibraryItem.isInvalid) {
+      if (existingLibraryItem.isMissing) {
+        libraryScan.addLog(LogLevel.INFO, `Podcast "${podcastMetadata.title}" was missing but is now found. Setting library item as NOT missing`)
+        existingLibraryItem.isMissing = false
+      }
+      if (existingLibraryItem.isInvalid) {
+        libraryScan.addLog(LogLevel.INFO, `Podcast "${podcastMetadata.title}" was invalid but is now found. Setting library item as NOT invalid`)
+        existingLibraryItem.isInvalid = false
+      }
+      libraryItemUpdated = true
+    }
+
     // Save Podcast changes to db
     if (hasMediaChanges) {
       await media.save()

server/utils/fileUtils.js

@@ -192,6 +192,15 @@ module.exports.recurseFiles = async (path, relPathToReplace = null) => {
     relPathToReplace = path
   }
 
+  // Handle symlinked directories by using the real path for string replacements
+  let realPathToReplace = relPathToReplace
+  try {
+    realPathToReplace = await fs.realpath(relPathToReplace)
+    realPathToReplace = filePathToPOSIX(realPathToReplace)
+    if (!realPathToReplace.endsWith('/')) realPathToReplace += '/'
+  } catch (err) {}
+
+
   const options = {
     mode: rra.LIST,
     recursive: true,
@@ -219,7 +228,7 @@ module.exports.recurseFiles = async (path, relPathToReplace = null) => {
 
       item.fullname = filePathToPOSIX(item.fullname)
       item.path = filePathToPOSIX(item.path)
-      const relpath = item.fullname.replace(relPathToReplace, '')
+      const relpath = item.fullname.replace(realPathToReplace, '')
       let reldirname = Path.dirname(relpath)
       if (reldirname === '.') reldirname = ''
       const dirname = Path.dirname(item.fullname)
@@ -249,11 +258,11 @@ module.exports.recurseFiles = async (path, relPathToReplace = null) => {
       return true
     })
     .map((item) => {
-      var isInRoot = item.path + '/' === relPathToReplace
+      var isInRoot = item.path + '/' === realPathToReplace
       return {
         name: item.name,
-        path: item.fullname.replace(relPathToReplace, ''),
+        path: item.fullname.replace(realPathToReplace, ''),
-        reldirpath: isInRoot ? '' : item.path.replace(relPathToReplace, ''),
+        reldirpath: isInRoot ? '' : item.path.replace(realPathToReplace, ''),
         fullpath: item.fullname,
         extension: item.extension,
         deep: item.deep