feat: async scan-file-for-metadata on Edit Metadata tab (bookshelf-gs7j) #993

Merged
zombor merged 14 commits from bd-bookshelf-gs7j into main 2026-07-07 15:34:09 +00:00
Owner

Summary

Converts the old synchronous LLMScanHandler to an async start+poll workflow with three bug fixes:

  • Dead spinner (CSS display:none + hidden attribute never removed) → replaced with real polling status display
  • 502 timeout (sync 120s LLM call behind Cloudflare ~100s limit) → POST /books/{id}/metadata/scan returns 202 {job_id} immediately; JS polls GET /books/{id}/metadata/scan/{job_id} every 2 s
  • HTTP 500 on ErrLLMExtractNoData → graceful {status: "empty"} HTTP 200 with user-visible "No metadata found in file." message

Also moves the scan button from the Fetch Metadata tab to the Edit Metadata tab (after .metadata-editor-cover), and adds an optional LLM provider <select> when multiple providers are configured.

Architecture

  • LLMScanWorkflow + LLMScanActivities in internal/wfengine (runs on QueueLLMInteractive)
  • GetLLMScanResult in diag_accessor.go using history-walking pattern (same as GetLLMIdentifyResult)
  • ErrLLMExtractNoDatagowf.NewPermanentError only inside wfengine (architecture boundary preserved)
  • LLMScanStartHandler / LLMScanStatusHandler in internal/books (curried DI pattern)

Test plan

  • 23 Vitest unit tests for metadata_scan_controller.js (start, poll, terminal states, errors)
  • Ginkgo unit tests for LLMScanStartHandler and LLMScanStatusHandler (all edge cases)
  • Ginkgo integration tests for Engine.StartLLMScanWorkflow and Engine.GetLLMScanResult
  • NewWithFactoryExt — LLMScan registered proves real engine wires workflow + activity
  • make test passes (3455 JS + all Go unit tests)
  • make lint clean (errors shown are from sibling worktrees, not this PR)

Closes bead bookshelf-gs7j on merge.

## Summary Converts the old synchronous `LLMScanHandler` to an async start+poll workflow with three bug fixes: - **Dead spinner** (CSS `display:none` + `hidden` attribute never removed) → replaced with real polling status display - **502 timeout** (sync 120s LLM call behind Cloudflare ~100s limit) → `POST /books/{id}/metadata/scan` returns 202 `{job_id}` immediately; JS polls `GET /books/{id}/metadata/scan/{job_id}` every 2 s - **HTTP 500 on `ErrLLMExtractNoData`** → graceful `{status: "empty"}` HTTP 200 with user-visible "No metadata found in file." message Also moves the scan button from the **Fetch Metadata** tab to the **Edit Metadata** tab (after `.metadata-editor-cover`), and adds an optional LLM provider `<select>` when multiple providers are configured. ### Architecture - `LLMScanWorkflow` + `LLMScanActivities` in `internal/wfengine` (runs on `QueueLLMInteractive`) - `GetLLMScanResult` in `diag_accessor.go` using history-walking pattern (same as `GetLLMIdentifyResult`) - `ErrLLMExtractNoData` → `gowf.NewPermanentError` only inside `wfengine` (architecture boundary preserved) - `LLMScanStartHandler` / `LLMScanStatusHandler` in `internal/books` (curried DI pattern) ## Test plan - [x] 23 Vitest unit tests for `metadata_scan_controller.js` (start, poll, terminal states, errors) - [x] Ginkgo unit tests for `LLMScanStartHandler` and `LLMScanStatusHandler` (all edge cases) - [x] Ginkgo integration tests for `Engine.StartLLMScanWorkflow` and `Engine.GetLLMScanResult` - [x] `NewWithFactoryExt — LLMScan registered` proves real engine wires workflow + activity - [x] `make test` passes (3455 JS + all Go unit tests) - [x] `make lint` clean (errors shown are from sibling worktrees, not this PR) Closes bead bookshelf-gs7j on merge.
feat(gs7j): async scan-file-for-metadata on Edit Metadata tab
Some checks failed
/ E2E API (pull_request) Has been cancelled
/ Integration (pull_request) Has been cancelled
/ JS Unit Tests (pull_request) Has been cancelled
/ E2E Browser (pull_request) Has been cancelled
/ Lint (pull_request) Has been cancelled
/ Test (pull_request) Has been cancelled
26a675b79e
Converts the old synchronous LLMScanHandler (120s blocking call) to an
async start+poll workflow, moving the button to the Edit Metadata tab
and adding an optional LLM provider <select>.

Bugs fixed:
- Dead spinner (CSS display:none + hidden never removed) → polling status
- 502 timeout (sync 120s LLM call behind Cloudflare ~100s limit) → async
- HTTP 500 on ErrLLMExtractNoData → graceful {status: "empty"} HTTP 200

Changes:
- POST /books/{id}/metadata/scan → 202 {job_id}  (LLMScanStartHandler)
- GET  /books/{id}/metadata/scan/{job_id} → status JSON (LLMScanStatusHandler)
- LLMScanWorkflow + LLMScanActivities in internal/wfengine
- GetLLMScanResult in diag_accessor using history-walking pattern
- metadata_scan_controller.js rewritten with async poll + _sleep seam
- 23 Vitest specs for the controller, wfengine integration tests
- Scan button moved from Fetch Metadata to Edit Metadata tab
- Optional provider <select> when len(.EnabledLLMProviders) > 1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): update browser e2e scan test for async start+poll API
Some checks failed
/ E2E API (pull_request) Successful in 3m20s
/ JS Unit Tests (pull_request) Failing after 1m52s
/ Integration (pull_request) Successful in 4m34s
/ Lint (pull_request) Successful in 5m3s
/ E2E Browser (pull_request) Failing after 5m24s
/ Test (pull_request) Failing after 6m0s
683c672854
The scan button moved from the Fetch Metadata tab (.metadata-fetch-scan-row)
to the Edit Metadata tab (.metadata-editor-scan) and the API is now two-step:
POST → {job_id}, GET {job_id} → {status, candidate}. Update the browser
journey and window.fetch mock accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): achieve 100% JS branch coverage in metadata_scan_controller
Some checks failed
/ E2E API (pull_request) Successful in 1m10s
/ JS Unit Tests (pull_request) Successful in 1m39s
/ Integration (pull_request) Successful in 4m42s
/ Lint (pull_request) Successful in 5m12s
/ E2E Browser (pull_request) Failing after 5m15s
/ Test (pull_request) Failing after 5m56s
36ae394a55
- Remove dead if-existing branch in _showError (_clearError always runs
  before _showError so the element is never present at call time)
- Add _sleep prototype test to cover the setTimeout body
- Add unknown-status test to cover the false branch of if(status==="done")

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): use document.querySelector in _openInFetchModal for sibling search
Some checks failed
/ JS Unit Tests (pull_request) Successful in 48s
/ E2E API (pull_request) Successful in 2m16s
/ E2E Browser (pull_request) Successful in 3m1s
/ Lint (pull_request) Successful in 3m11s
/ Integration (pull_request) Successful in 3m17s
/ Test (pull_request) Failing after 4m9s
c8c5ba08a7
The metadata-scan controller (Edit Metadata tab) and the metadata-fetch
controller (Fetch Metadata tab) are sibling <section> elements, not
ancestor/descendant. The old ancestor-walk via parentElement never finds
the fetch controller; switch to document.querySelector([data-controller~="metadata-fetch"])
to locate it regardless of position in the DOM tree.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test(gs7j): add coverage for LLMScan workflow/activity and handler body path
All checks were successful
/ JS Unit Tests (pull_request) Successful in 45s
/ E2E API (pull_request) Successful in 2m41s
/ Integration (pull_request) Successful in 3m42s
/ Lint (pull_request) Successful in 3m46s
/ E2E Browser (pull_request) Successful in 3m58s
/ Test (pull_request) Successful in 4m39s
64f0d988c2
- Add LLMScanActivitiesExport type alias + NewLLMScanActivitiesWithStub
  constructor in export_test.go so tests can exercise Scan directly
- Add llm_scan_workflow_test.go: WorkflowTester test covering LLMScanWorkflow
  happy/error paths; direct Scan tests covering all error branches
  (getPDFSubPath, getLibraryPath, scanSafeJoin, statFile, too-large, readFile,
  extractPDFText, extractMetadata permanent/transient errors)
- Add wrong-start-attr backend helper + Context in llm_scan_engine_test.go to
  cover the extractScanBookID !ok branch (diag_accessor.go:354)
- Simplify metadata_llm_scan_handler.go:99 from 'if err; err != nil { return }'
  to 'return json.Encode(...)' — removes unreachable if-body, matches the
  pattern used by all other JSON handlers in the package
- Add POST-with-body tests in metadata_llm_scan_handler_test.go covering the
  provider_id decode path (lines 84-89) and malformed-JSON 400 path
- make coverage: check-coverage OK — zero uncovered statement blocks
Author
Owner

Screenshot: Edit Metadata — Scan UI

Shows the .metadata-editor-scan section with the provider selector (OpenAI / Gemini) and "Scan file for metadata" button, rendered in the flex layout between the cover and field columns.

scan_ui_screenshot.png

## Screenshot: Edit Metadata — Scan UI Shows the `.metadata-editor-scan` section with the provider selector (OpenAI / Gemini) and "Scan file for metadata" button, rendered in the flex layout between the cover and field columns. ![scan_ui_screenshot.png](/attachments/d90898b7-9f08-453b-b6de-3acc9ec7daf1)
Author
Owner

Screenshot: Edit Metadata — Scan UI

Shows the .metadata-editor-scan section with the provider selector (OpenAI / Gemini) and "Scan file for metadata" button, rendered in the flex layout between the cover and field columns.

scan_ui_screenshot.png

## Screenshot: Edit Metadata — Scan UI Shows the `.metadata-editor-scan` section with the provider selector (OpenAI / Gemini) and "Scan file for metadata" button, rendered in the flex layout between the cover and field columns. ![scan_ui_screenshot.png](/attachments/d90898b7-9f08-453b-b6de-3acc9ec7daf1)
Author
Owner

Screenshot: Edit Metadata Scan UI

Shows the .metadata-editor-scan section with provider selector (OpenAI / Gemini) and Scan file for metadata button.

scan_ui_screenshot.png

## Screenshot: Edit Metadata Scan UI Shows the `.metadata-editor-scan` section with provider selector (OpenAI / Gemini) and Scan file for metadata button. ![scan_ui_screenshot.png](/attachments/d90898b7-9f08-453b-b6de-3acc9ec7daf1)
style(gs7j): add scan UI CSS, remove orphaned fetch-scan rules
All checks were successful
/ JS Unit Tests (pull_request) Successful in 38s
/ E2E API (pull_request) Successful in 3m4s
/ Lint (pull_request) Successful in 4m9s
/ Integration (pull_request) Successful in 4m9s
/ E2E Browser (pull_request) Successful in 4m24s
/ Test (pull_request) Successful in 4m59s
d77aa4c1bf
Add CSS rules for .metadata-editor-scan (flex column between cover and
fields), .metadata-editor-scan-provider (stacked label+select), and
.metadata-editor-scan-status (muted status line). Remove the three
orphaned .metadata-fetch-scan-row / -spinner / --visible rules that
no longer have any template or JS consumer.
Author
Owner

UI Review — bookshelf-gs7j (PR #993)

Screenshot reviewed: d90898b7-9f08-453b-b6de-3acc9ec7daf1 (1265×1074 PNG, confirmed rendered)

What I see in the rendered screenshot

The Edit Metadata tab shows the cover thumbnail on the left; immediately to its right sits the new .metadata-editor-scan column containing a "Provider" label, an "OpenAI" select dropdown, and a "Scan file for metadata" button; the CORE field sections occupy the right column. The layout is a clean three-column flex row (cover · scan · fields).

Findings

[MINOR] static/css/main.css — .metadata-editor-scan-status uses hardcoded min-height: 1.25rem
The token var(--space-5) equals 1.25rem and already exists in :root. Using the token keeps the status line's layout constraint on the token system. (font-size: 0.875rem is consistent with the codebase's established non-tokenized font-size pattern, so no flag there.)
Fix: min-height: var(--space-5);

What passed

  • No bespoke parallel component classes. The new wrapper classes (.metadata-editor-scan, .metadata-editor-scan-provider, .metadata-editor-scan-status) are layout-only. Inside them, all canonical classes are reused: metadata-field-label, metadata-field-input, btn btn--secondary. No hand-rolled label/select/button replacements.
  • No inline style=. Template diff is clean; grep confirms no style= attributes introduced.
  • Spacing tokens. gap: var(--space-2) and gap: var(--space-1) used throughout the new CSS rules.
  • Color tokens. color: var(--fg-muted) used on the status line; no hardcoded hex colors introduced (the old rgba(124, 140, 248, 0.3) spinner border that existed before is removed by this PR — improvement).
  • Placement. The scan section sits naturally between cover and fields, visually aligned with both. Not cramped, not orphaned.
  • Provider select styling. Visually matches other selects on the page via shared metadata-field-input class.
  • Conditional rendering. Provider label+select are gated on gt (len .EnabledLLMProviders) 1; single-provider case shows only the button — no orphaned label.

REVIEW VERDICT: 0 blocker, 0 major, 1 minor

## UI Review — bookshelf-gs7j (PR #993) **Screenshot reviewed:** `d90898b7-9f08-453b-b6de-3acc9ec7daf1` (1265×1074 PNG, confirmed rendered) ### What I see in the rendered screenshot The Edit Metadata tab shows the cover thumbnail on the left; immediately to its right sits the new `.metadata-editor-scan` column containing a "Provider" label, an "OpenAI" select dropdown, and a "Scan file for metadata" button; the CORE field sections occupy the right column. The layout is a clean three-column flex row (cover · scan · fields). ### Findings [MINOR] static/css/main.css — `.metadata-editor-scan-status` uses hardcoded `min-height: 1.25rem` The token `var(--space-5)` equals `1.25rem` and already exists in `:root`. Using the token keeps the status line's layout constraint on the token system. (`font-size: 0.875rem` is consistent with the codebase's established non-tokenized font-size pattern, so no flag there.) Fix: `min-height: var(--space-5);` ### What passed - **No bespoke parallel component classes.** The new wrapper classes (`.metadata-editor-scan`, `.metadata-editor-scan-provider`, `.metadata-editor-scan-status`) are layout-only. Inside them, all canonical classes are reused: `metadata-field-label`, `metadata-field-input`, `btn btn--secondary`. No hand-rolled label/select/button replacements. - **No inline `style=`.** Template diff is clean; grep confirms no `style=` attributes introduced. - **Spacing tokens.** `gap: var(--space-2)` and `gap: var(--space-1)` used throughout the new CSS rules. - **Color tokens.** `color: var(--fg-muted)` used on the status line; no hardcoded hex colors introduced (the old `rgba(124, 140, 248, 0.3)` spinner border that existed before is removed by this PR — improvement). - **Placement.** The scan section sits naturally between cover and fields, visually aligned with both. Not cramped, not orphaned. - **Provider select styling.** Visually matches other selects on the page via shared `metadata-field-input` class. - **Conditional rendering.** Provider label+select are gated on `gt (len .EnabledLLMProviders) 1`; single-provider case shows only the button — no orphaned label. --- REVIEW VERDICT: 0 blocker, 0 major, 1 minor
Author
Owner

CODE REVIEW: bookshelf-gs7j — Async LLM Scan (start+poll)

Reviewed diff at SHA d77aa4c1. All critical paths traced end-to-end.


Phase 0: Spec Compliance

All stated requirements verified:

  • LLMScanStartHandler returns 202 {job_id}
  • LLMScanStatusHandler polls without blocking ✓
  • Terminal states: running/done/empty/failed all present ✓
  • ErrLLMExtractNoDataempty terminal status (HTTP 200, not 500) ✓ — traced: activity wraps err, workflow stores error string, isLLMExtractNoDataErr string-matches it, terminalScanStatus returns Empty: true, handler encodes {"status":"empty"} with implicit 200
  • Architecture boundary respected: internal/books, internal/metadata/llm, internal/files contain no go-workflows imports ✓
  • workflow.NewPermanentError only in internal/wfengine
  • Provider selection via llm_provider_id: invalid/unknown provider returns ErrLLMDisabled → permanent activity failure → {"status":"failed","error":"..."}
  • JS controller _poll loop correct; hidden-attribute spinner logic replaced with _setStatus(text) approach ✓
  • Provider_id sent in POST body ✓
  • Black-box tests (package books_test, package wfengine_test) ✓
  • One-Expect-per-It in Ginkgo tests ✓
  • No new .golangci.yml exclusions ✓
  • scripts/check-coverage.sh unchanged ✓
  • No inline style= attributes ✓

Findings

[MAJOR] internal/wfengine/llm_scan_engine_test.go:461 — no StartWorker integration test for LLMScanWorkflow
The NewWithFactoryExt — LLMScan registered block at line 461 only calls engine.StartLLMScanWorkflow() (i.e., CreateWorkflowInstance). It does NOT call engine.StartWorker(ctx), so the activity's registration in the worker's queue registry is never exercised. The workflow-needs-real-engine-registration-test rule (baked into project memory) requires exactly this: "add a real-engine integration test (production New()+StartWorker) per workflow." The precedent is LLMIdentifyWorkflow — no activity-not-found on QueueLLMInteractive (integration) in engine_integration_test.go (line 929), which calls StartWorker, starts the workflow, and asserts completion without "activity not found". Without this, a wiring error in registerExtendedWorkflows (e.g., wrong queue, missing activity registration) only surfaces at runtime. Fix: add an equivalent LLMScanWorkflow — no activity-not-found on QueueLLMInteractive (integration) Describe block in internal/wfengine/engine_integration_test.go that uses LLMScanDeps stubs returning permanent errors (e.g., stat fails immediately), calls engine.StartWorker(ctx), starts the workflow, and asserts GetInstanceState reaches "completed" without "activity not found".

[MINOR] static/js/test/metadata_scan_controller.test.js:483 — misleading test name for error-display behaviour
The test "re-uses the existing error element on repeated errors" describes behaviour from the OLD implementation (which called existing.textContent = message to mutate the existing element). The NEW implementation always creates a fresh element via document.createElement — idempotency is achieved instead by _clearError() removing the prior element at the top of scan(). The test assertion (exactly 1 .metadata-scan-error element after two failing scans) is correct, but the name implies a different mechanism. Rename to "leaves exactly one error element after repeated failing scans" to describe what is actually tested.

[MINOR] internal/wfengine/diag_accessor.go:338 — LLMScanStatus.Done field is checked nowhere in response mapping
llmScanStatusToResp in metadata_llm_scan_handler.go maps status to response by checking Running, Empty, Failed — and falls through to "done" for anything else. The Done boolean on LLMScanStatus is set by callers but never read by the mapper, making it effectively decorative. This works correctly in practice (callers always set exactly one flag), but the unused field creates a subtle contract mismatch. Consider documenting that Done is informational only for callers, or removing it in a follow-up.


REVIEW VERDICT: 0 blocker, 1 major, 2 minor

## CODE REVIEW: bookshelf-gs7j — Async LLM Scan (start+poll) Reviewed diff at SHA d77aa4c1. All critical paths traced end-to-end. --- ### Phase 0: Spec Compliance All stated requirements verified: - `LLMScanStartHandler` returns 202 `{job_id}` ✓ - `LLMScanStatusHandler` polls without blocking ✓ - Terminal states: running/done/empty/failed all present ✓ - `ErrLLMExtractNoData` → `empty` terminal status (HTTP 200, not 500) ✓ — traced: activity wraps err, workflow stores error string, `isLLMExtractNoDataErr` string-matches it, `terminalScanStatus` returns `Empty: true`, handler encodes `{"status":"empty"}` with implicit 200 - Architecture boundary respected: `internal/books`, `internal/metadata/llm`, `internal/files` contain no go-workflows imports ✓ - `workflow.NewPermanentError` only in `internal/wfengine` ✓ - Provider selection via `llm_provider_id`: invalid/unknown provider returns `ErrLLMDisabled` → permanent activity failure → `{"status":"failed","error":"..."}` ✓ - JS controller `_poll` loop correct; `hidden`-attribute spinner logic replaced with `_setStatus(text)` approach ✓ - Provider_id sent in POST body ✓ - Black-box tests (`package books_test`, `package wfengine_test`) ✓ - One-Expect-per-It in Ginkgo tests ✓ - No new `.golangci.yml` exclusions ✓ - `scripts/check-coverage.sh` unchanged ✓ - No inline `style=` attributes ✓ --- ### Findings [MAJOR] internal/wfengine/llm_scan_engine_test.go:461 — no StartWorker integration test for LLMScanWorkflow The `NewWithFactoryExt — LLMScan registered` block at line 461 only calls `engine.StartLLMScanWorkflow()` (i.e., `CreateWorkflowInstance`). It does NOT call `engine.StartWorker(ctx)`, so the activity's registration in the worker's queue registry is never exercised. The `workflow-needs-real-engine-registration-test` rule (baked into project memory) requires exactly this: "add a real-engine integration test (production New()+StartWorker) per workflow." The precedent is `LLMIdentifyWorkflow — no activity-not-found on QueueLLMInteractive (integration)` in `engine_integration_test.go` (line 929), which calls `StartWorker`, starts the workflow, and asserts completion without "activity not found". Without this, a wiring error in `registerExtendedWorkflows` (e.g., wrong queue, missing activity registration) only surfaces at runtime. Fix: add an equivalent `LLMScanWorkflow — no activity-not-found on QueueLLMInteractive (integration)` Describe block in `internal/wfengine/engine_integration_test.go` that uses `LLMScanDeps` stubs returning permanent errors (e.g., stat fails immediately), calls `engine.StartWorker(ctx)`, starts the workflow, and asserts `GetInstanceState` reaches "completed" without "activity not found". [MINOR] static/js/test/metadata_scan_controller.test.js:483 — misleading test name for error-display behaviour The test "re-uses the existing error element on repeated errors" describes behaviour from the OLD implementation (which called `existing.textContent = message` to mutate the existing element). The NEW implementation always creates a fresh element via `document.createElement` — idempotency is achieved instead by `_clearError()` removing the prior element at the top of `scan()`. The test assertion (exactly 1 `.metadata-scan-error` element after two failing scans) is correct, but the name implies a different mechanism. Rename to "leaves exactly one error element after repeated failing scans" to describe what is actually tested. [MINOR] internal/wfengine/diag_accessor.go:338 — `LLMScanStatus.Done` field is checked nowhere in response mapping `llmScanStatusToResp` in `metadata_llm_scan_handler.go` maps status to response by checking `Running`, `Empty`, `Failed` — and falls through to "done" for anything else. The `Done` boolean on `LLMScanStatus` is set by callers but never read by the mapper, making it effectively decorative. This works correctly in practice (callers always set exactly one flag), but the unused field creates a subtle contract mismatch. Consider documenting that `Done` is informational only for callers, or removing it in a follow-up. --- REVIEW VERDICT: 0 blocker, 1 major, 2 minor
Author
Owner

Security Review — bookshelf-gs7j (PR #993)

Diff reviewed: origin/bd-bookshelf-gs7j (SHA d77aa4c1) vs origin/main.


Multi-user scoping / AuthZ

Both handlers correctly gate on checkBookAccess(r.Context(), userIDFromRequest(r), id) where userID comes from the session (not from the request body or query param).

The status handler (LLMScanStatusHandler) additionally verifies job-book ownership via s.BookID != id after the poll — this correctly prevents a user from polling another book's scan job by reusing a known UUID. BookID is extracted from the workflow's WorkflowExecutionStarted history event (not from caller-supplied input), so the check cannot be spoofed. Ownership logic mirrors the existing LLMFetchStatusHandler pattern.

Path traversal

scanSafeJoin (wfengine/llm_scan_workflow.go) is a verbatim port of the old safeJoin from the HTTP handler layer. It joins root+subPath, applies filepath.Clean, and prefix-checks against cleanRoot+separator. This correctly rejects ../ traversal. The subPath comes from the database (stored at ingest time), not from any request parameter, so the attack surface is narrow. The pre-existing symlink-in-subpath limitation is unchanged and not a regression of this PR.

SSRF

providerID from the request body is used solely as a lookup key into settings.GetLLMProviderCfg, which iterates pre-configured DB-stored providers and returns an empty llm.Config{} (not an error) when the ID is unknown. ExtractMetadata then short-circuits on !cfg.IsConfigured(). No caller-supplied URL reaches the HTTP transport. No SSRF risk.

Resource limits

PDF size cap (files.MaxPDFBytes) is enforced in readPDF before bytes are loaded. The activity runs under the workflow context with timeout derived from llmInteractiveActivityOptions (MaxAttempts=1). No decompression-bomb concern for cover images — the rendered page is extracted by files.ExtractPDFText, unchanged from the prior path.

Secrets / PII logging

wrapExtractErr logs partial.Raw (the model's text response — chatResp.Choices[0].Message.Content). This is the model's output, not the API key (which is only in the outbound Authorization: Bearer request header). No credential or key is logged. StartLLMScanWorkflow logs provider_id (a slug string, not a secret). On the success path only raw_len (integer) is logged, not the raw content itself.

Architecture boundary

gowf.NewPermanentError is called only inside LLMScanActivities.Scan (wfengine package). internal/metadata/llm returns plain sentinel errors (ErrLLMExtractNoData, ErrLLMDisabled). No domain package imports go-workflows. Boundary is intact.


Findings

[MINOR] internal/app/build_extended_deps.go:319-327 — ErrInstanceNotFound propagates as 500 instead of documented 404
  GetLLMScanResult does not translate wfengine.ErrInstanceNotFound into
  middleware.ErrNotFound before returning. The middleware statusFor() switch
  only matches middleware.ErrNotFound for HTTP 404; any other sentinel falls
  through to 500. The doc comment on LLMScanStatusHandler says "Returns 404
  if job_id does not exist" but a missing instance yields HTTP 500 (no
  internal detail is leaked to the client — the body is "Internal Server Error"
  — but the status code is wrong). The same gap exists in the pre-existing
  GetLLMIdentifyResult path; this PR inherits rather than introduces it.
  Fix: add
    if errors.Is(err, wfengine.ErrInstanceNotFound) {
        return appwire.LLMScanStatus{}, fmt.Errorf("scan job %q: %w", instanceID, middleware.ErrNotFound)
    }
  in the GetLLMScanResult closure (matching the pattern already used for
  GetImportMetadataResult and GetMoveBooksResult at lines 413 and 436).

[MINOR] internal/wfengine/export_test.go — white-box test surface extended with new scan exports
  export_test.go (package wfengine, not wfengine_test) is extended with
  LLMScanActivitiesExport, NewLLMScanActivitiesWithStub,
  NewTestEngineWithLLMScan, and NewTestEngineWithLLMScanResult, all of which
  directly construct or mutate unexported struct fields. The file is
  grandfathered in scripts/test_policy_check/allowlist.txt so CI does not
  catch it, but each addition widens the white-box surface. The scan activity
  Scan() is exported and takes only plain parameters; the black-box path would
  be to drive it through LLMScanWorkflow via the tester (which
  llm_scan_workflow_test.go already does for most cases). The NewTestEngine*
  helpers inject unexported Engine fields — prefer a public option/constructor
  variant if the pattern is needed long-term.

REVIEW VERDICT: 0 blocker, 0 major, 2 minor

## Security Review — bookshelf-gs7j (PR #993) Diff reviewed: `origin/bd-bookshelf-gs7j` (SHA d77aa4c1) vs `origin/main`. --- ### Multi-user scoping / AuthZ Both handlers correctly gate on `checkBookAccess(r.Context(), userIDFromRequest(r), id)` where userID comes from the session (not from the request body or query param). The status handler (`LLMScanStatusHandler`) additionally verifies job-book ownership via `s.BookID != id` after the poll — this correctly prevents a user from polling another book's scan job by reusing a known UUID. BookID is extracted from the workflow's `WorkflowExecutionStarted` history event (not from caller-supplied input), so the check cannot be spoofed. Ownership logic mirrors the existing `LLMFetchStatusHandler` pattern. ### Path traversal `scanSafeJoin` (wfengine/llm_scan_workflow.go) is a verbatim port of the old `safeJoin` from the HTTP handler layer. It joins root+subPath, applies `filepath.Clean`, and prefix-checks against `cleanRoot+separator`. This correctly rejects `../` traversal. The subPath comes from the database (stored at ingest time), not from any request parameter, so the attack surface is narrow. The pre-existing symlink-in-subpath limitation is unchanged and not a regression of this PR. ### SSRF `providerID` from the request body is used solely as a lookup key into `settings.GetLLMProviderCfg`, which iterates pre-configured DB-stored providers and returns an empty `llm.Config{}` (not an error) when the ID is unknown. `ExtractMetadata` then short-circuits on `!cfg.IsConfigured()`. No caller-supplied URL reaches the HTTP transport. No SSRF risk. ### Resource limits PDF size cap (`files.MaxPDFBytes`) is enforced in `readPDF` before bytes are loaded. The activity runs under the workflow context with timeout derived from `llmInteractiveActivityOptions` (MaxAttempts=1). No decompression-bomb concern for cover images — the rendered page is extracted by `files.ExtractPDFText`, unchanged from the prior path. ### Secrets / PII logging `wrapExtractErr` logs `partial.Raw` (the model's text response — `chatResp.Choices[0].Message.Content`). This is the model's output, not the API key (which is only in the outbound `Authorization: Bearer` request header). No credential or key is logged. `StartLLMScanWorkflow` logs `provider_id` (a slug string, not a secret). On the success path only `raw_len` (integer) is logged, not the raw content itself. ### Architecture boundary `gowf.NewPermanentError` is called only inside `LLMScanActivities.Scan` (wfengine package). `internal/metadata/llm` returns plain sentinel errors (`ErrLLMExtractNoData`, `ErrLLMDisabled`). No domain package imports go-workflows. Boundary is intact. --- ### Findings ``` [MINOR] internal/app/build_extended_deps.go:319-327 — ErrInstanceNotFound propagates as 500 instead of documented 404 GetLLMScanResult does not translate wfengine.ErrInstanceNotFound into middleware.ErrNotFound before returning. The middleware statusFor() switch only matches middleware.ErrNotFound for HTTP 404; any other sentinel falls through to 500. The doc comment on LLMScanStatusHandler says "Returns 404 if job_id does not exist" but a missing instance yields HTTP 500 (no internal detail is leaked to the client — the body is "Internal Server Error" — but the status code is wrong). The same gap exists in the pre-existing GetLLMIdentifyResult path; this PR inherits rather than introduces it. Fix: add if errors.Is(err, wfengine.ErrInstanceNotFound) { return appwire.LLMScanStatus{}, fmt.Errorf("scan job %q: %w", instanceID, middleware.ErrNotFound) } in the GetLLMScanResult closure (matching the pattern already used for GetImportMetadataResult and GetMoveBooksResult at lines 413 and 436). [MINOR] internal/wfengine/export_test.go — white-box test surface extended with new scan exports export_test.go (package wfengine, not wfengine_test) is extended with LLMScanActivitiesExport, NewLLMScanActivitiesWithStub, NewTestEngineWithLLMScan, and NewTestEngineWithLLMScanResult, all of which directly construct or mutate unexported struct fields. The file is grandfathered in scripts/test_policy_check/allowlist.txt so CI does not catch it, but each addition widens the white-box surface. The scan activity Scan() is exported and takes only plain parameters; the black-box path would be to drive it through LLMScanWorkflow via the tester (which llm_scan_workflow_test.go already does for most cases). The NewTestEngine* helpers inject unexported Engine fields — prefer a public option/constructor variant if the pattern is needed long-term. ``` REVIEW VERDICT: 0 blocker, 0 major, 2 minor
fix(gs7j): address review findings — registration test + minor fixes
All checks were successful
/ JS Unit Tests (pull_request) Successful in 39s
/ E2E API (pull_request) Successful in 3m15s
/ Integration (pull_request) Successful in 4m23s
/ Lint (pull_request) Successful in 4m40s
/ E2E Browser (pull_request) Successful in 4m57s
/ Test (pull_request) Successful in 5m35s
82746cfabe
MAJOR: Add LLMScanWorkflow real-engine registration integration test in
engine_integration_test.go mirroring the existing LLMIdentifyWorkflow
test. Wires LLMScanDeps stubs, calls StartWorker, starts a scan workflow,
and asserts it reaches "completed" without "activity not found".

Minor fixes:
- build_extended_deps.go: translate ErrInstanceNotFound → ErrNotFound
  (→ 404) in GetLLMScanResult so a missing job_id returns 404 not 500
- main.css: use var(--space-5) token instead of bare 1.25rem
- metadata_scan_controller.test.js: rename misleading test description
- diag_accessor.go: clarify LLMScanStatus.Done doc comment
feat(gs7j): PDF page-image rendering for LLM scan pipeline
Some checks failed
/ JS Unit Tests (pull_request) Successful in 2m6s
/ E2E API (pull_request) Successful in 2m49s
/ Lint (pull_request) Successful in 5m14s
/ Integration (pull_request) Failing after 3m50s
/ Test (pull_request) Successful in 5m55s
/ E2E Browser (pull_request) Successful in 5m4s
056dc6a1fe
Replace PDF text extraction with pdfium page-image rendering in the LLM
scan pipeline. The vision LLM now receives rendered PNG page images
(first+last 4 pages, deduped, capped at 8) instead of extracted text,
enabling accurate metadata extraction from image-only or scanned PDFs.

- internal/files: Add RenderPDFPageImages (pdfium page rendering, reuses
  cover_extract seams, goroutine+timeout pattern, 100% coverage)
- internal/metadata/llm: ExtractMetadata inner signature now takes
  pageImages [][]byte; buildExtractParts emits one image_url part per page
- internal/wfengine: LLMScanDeps/Activities swap ExtractPDFText→
  RenderPageImages; Scan method updated; permanent-fail on render error
- internal/app: wire files.RenderPDFPageImages → LLMScanDeps.RenderPageImages
- Bug fix (gs7j/Task 2): status poll 404 race — skip ownership check when
  BookID==0 (WorkflowExecutionStarted not yet readable from history)
- Bug fix (gs7j/Task 3): move .metadata-editor-scan div inside
  .metadata-editor-cover so scan button appears below the cover thumbnail
- Bug fix (gs7j/Task 5): GetLLMProviderCfg fallback — when DefaultID is
  empty or points to a disabled provider, fall back to first enabled+
  configured provider instead of returning "vision not configured"

All tests pass; 100% coverage maintained.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): update engine_integration_test to new LLMScanDeps signature
All checks were successful
/ JS Unit Tests (pull_request) Successful in 38s
/ E2E API (pull_request) Successful in 2m49s
/ Integration (pull_request) Successful in 3m53s
/ Lint (pull_request) Successful in 3m53s
/ E2E Browser (pull_request) Successful in 4m35s
/ Test (pull_request) Successful in 4m47s
60ca290412
Replace ExtractPDFText+old ExtractMetadata stubs in engine_integration_test.go
with RenderPageImages+new ExtractMetadata ([][]byte) stubs to match the
pipeline redesign in the previous commit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): white-background flatten, fail-loud on zero images, longer poll, disable select
All checks were successful
/ JS Unit Tests (pull_request) Successful in 35s
/ E2E API (pull_request) Successful in 2m25s
/ E2E Browser (pull_request) Successful in 2m52s
/ Integration (pull_request) Successful in 3m35s
/ Lint (pull_request) Successful in 4m3s
/ Test (pull_request) Successful in 5m5s
5f5de9cc75
1. [CRITICAL] Flatten pdfium-rendered pages onto white before JPEG encoding.
   Pdfium renders with alpha; transparent backgrounds (all text pages) became
   black in JPEG (no alpha channel) → LLM saw nothing. flattenOnWhite composites
   over image.Uniform{color.White} before jpeg.Encode. Test: transparent PNG →
   JPEG pixel ≈ white (R/G/B ≈ 255).

2. Fail loud when no page images survive normalisation. buildExtractParts returns
   ErrNoUsablePageImages when all images in a non-empty slice fail. ExtractMetadata
   propagates it; text-only vision requests are now banned. isPermanentScanErr
   classifies it permanent (retrying identical bytes won't fix it). Test added.

3. Log rendered page count + per-page byte sizes at activity level before the
   LLM call (slog "llm scan page images rendered").

4. Extend JS poll budget from 60→150 iterations (2 min → 5 min) to accommodate
   slow vision models (Werewolf scan finished at ~4 min).

5. Disable provider <select> during scanning alongside the button; re-enable on
   terminal state. Test added for both disabled and re-enabled states.

Also: extracted sendExtractRequest helper to keep ExtractMetadata under funlen 60.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(gs7j): rewrite extractMetadataPrompt to suppress artwork analysis
All checks were successful
/ JS Unit Tests (pull_request) Successful in 36s
/ E2E API (pull_request) Successful in 2m39s
/ Lint (pull_request) Successful in 3m40s
/ Integration (pull_request) Successful in 3m45s
/ E2E Browser (pull_request) Successful in 4m10s
/ Test (pull_request) Successful in 4m36s
c90b2a46c8
Models receiving page images tend to describe visual content (wolf, forest,
full moon) instead of transcribing printed text (title, author, publisher).
The updated prompt explicitly:
 - Labels the task as BIBLIOGRAPHIC METADATA EXTRACTION
 - Mandates raw JSON only (no prose/fences/commentary) at both start and end
 - Directs the model to READ PRINTED TEXT and explicitly forbids artwork
   description or visual interpretation

No code-path or API changes. parseExtractResponse's think-strip, fence-strip,
and first-{/last-} extraction remain as belt-and-suspenders for models that
still wrap output despite the stronger instruction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(llm-scan): downscale pages to 768px, fewer pages, max_tokens, proven prompt
Some checks failed
/ Test (pull_request) Has been cancelled
/ Integration (pull_request) Has been cancelled
/ Lint (pull_request) Has been cancelled
/ E2E Browser (pull_request) Has been cancelled
/ E2E API (pull_request) Has been cancelled
/ JS Unit Tests (pull_request) Has been cancelled
c43dd2053f
- Add extractJPEGEncode that downscales to ≤768px long edge (CatmullRom)
  before flattening on white + JPEG encoding; used only in extract path
- Change buildExtractParts to use extractJPEGEncode instead of defaultJPEGEncode
- Switch renderPageImages call from firstN=4,lastN=4 to firstN=4,lastN=0
- Lower maxRenderPages cap from 8 to 5
- Add MaxTokens: 4096 to chatRequest (extract path only)
- Replace extractMetadataPrompt with validated role-framed wording that
  suppresses artwork analysis on qwen3-vl:8b
- Add NormalizeForExtract export + downscale test (1024×1024 → ≤768px long edge)
- Update buildPageIndicesN cap test (8→5 expected length)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test(llm-scan): add landscape + ultra-thin coverage for downscaleForLLM
Some checks failed
/ E2E API (pull_request) Successful in 3m15s
/ JS Unit Tests (pull_request) Successful in 1m53s
/ Integration (pull_request) Successful in 4m36s
/ Lint (pull_request) Successful in 4m49s
/ E2E Browser (pull_request) Failing after 4m39s
/ Test (pull_request) Successful in 5m38s
e0964d264f
Cover the w>h scaling branch (1200×900 landscape PNG) and the dstH<1 /
dstW<1 defensive guards (1024×1 and 1×1024 extreme-aspect images) so
downscaleForLLM reaches 100% statement coverage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
Owner

Screenshot: scan button spinner state

The button transitions from "Scan file for metadata" → spinner + "Scanning…" (disabled) on click, and reverts to the original label at every terminal state (done / empty / failed).

scan-spinner

**Screenshot: scan button spinner state** The button transitions from "Scan file for metadata" → spinner + "Scanning…" (disabled) on click, and reverts to the original label at every terminal state (done / empty / failed). ![scan-spinner](https://git.zombor.net/attachments/420b9ffd-48ed-4882-b653-abfa7a81c0a3)
feat(gs7j): spinner-in-button loading state for scan file metadata
All checks were successful
/ JS Unit Tests (pull_request) Successful in 35s
/ E2E API (pull_request) Successful in 2m47s
/ Lint (pull_request) Successful in 3m38s
/ Integration (pull_request) Successful in 3m39s
/ E2E Browser (pull_request) Successful in 4m21s
/ Test (pull_request) Successful in 4m34s
45bc539967
On click the scan button replaces its text with a `.btn-spinner`
element + "Scanning…" (disabled). At every terminal state (done /
empty / failed) `_setLoading(false)` restores the original label
captured in `connect()`. The separate "Starting scan…" / "Scanning…"
status line is removed; the status span is now reserved for terminal
messages only ("No metadata found in file.") and error display.

- Add `.btn-spinner` CSS class reusing existing `mf-spin` keyframe
- Add `connect()` lifecycle to capture original button label
- Update `_setLoading()` to swap button content on loading/idle
- Remove intermediate `_setStatus()` calls from `scan()`
- Update test: assert spinner in button during scan, status line hidden
- Add tests: button reverts to original label at done/empty/failed
- 100% JS coverage maintained (branch for empty-label fallback covered)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
Owner

UI Review — PR #993 (bookshelf-gs7j)

Reviewed both rendered screenshots (scan_panel.png 1265×1074, scan_spinner.png 1280×800) and the CSS/template/JS diff.

What I see in the screenshots

scan_panel.png (Edit Metadata tab, full page):
The scan section sits cleanly below the cover placeholder in the left column. A "Provider" label + <select> (showing "OpenAI") stacks directly above the "Scan file for metadata" button, both left-aligned with the cover. The select visually matches the metadata field inputs in the right column. Spacing between the cover and the scan section is appropriate — no cramping, no dead-band. The button is sized normally as a .btn--secondary call-to-action; it does not sprawl full-width.

scan_spinner.png (isolated loading-state render):
The button in loading state shows "Scanning…" text and appears correctly disabled/greyed. The animated .btn-spinner circle is not visible in a static screenshot (expected for a CSS animation), but the CSS diff confirms it reuses the existing mf-spin keyframe and is inserted before the label text by the controller.

Source cross-check

Canonical component reuse (pass):

  • Button: class="btn btn--secondary" at templates/pages/books_show.html:881 — canonical.
  • Select: class="metadata-field-input" at templates/pages/books_show.html:872 — canonical, visually identical to the form selects on the right panel.
  • Label: class="metadata-field-label" at templates/pages/books_show.html:870 — canonical.
  • Spinner animation: .btn-spinner reuses the existing mf-spin keyframe at static/css/main.css:5834 — not a bespoke re-implementation.

Spacing tokens (pass):
.metadata-editor-scan uses gap: var(--space-2); .metadata-editor-scan-provider uses gap: var(--space-1); .metadata-editor-scan-status uses min-height: var(--space-5). All spacing from the --space-* token set.

Color tokens (pass):
.metadata-editor-scan-status uses color: var(--fg-muted). .btn-spinner uses border-top-color: var(--accent). The rgba(124, 140, 248, 0.3) tint is identical to the pre-existing mf-spin-based spinner pattern already in the codebase — not new debt.

No inline style= attributes (pass):
grep -n style= books_show.html returns no hits in the scan section.

Naming convention (pass):
New classes .metadata-editor-scan, .metadata-editor-scan-provider, .metadata-editor-scan-status follow the existing metadata-editor-* namespace already used by .metadata-editor-body and .metadata-editor-cover — not a parallel bespoke system.

font-size: 0.875rem (not a finding):
font-size: 0.875rem at static/css/main.css:5870 appears 100+ times in main.css — it is the established codebase idiom for small text (no --font-size-* token exists); this matches the pattern.


No findings.

REVIEW VERDICT: 0 blocker, 0 major, 0 minor

## UI Review — PR #993 (bookshelf-gs7j) Reviewed both rendered screenshots (scan_panel.png 1265×1074, scan_spinner.png 1280×800) and the CSS/template/JS diff. ### What I see in the screenshots **scan_panel.png (Edit Metadata tab, full page):** The scan section sits cleanly below the cover placeholder in the left column. A "Provider" label + `<select>` (showing "OpenAI") stacks directly above the "Scan file for metadata" button, both left-aligned with the cover. The select visually matches the metadata field inputs in the right column. Spacing between the cover and the scan section is appropriate — no cramping, no dead-band. The button is sized normally as a `.btn--secondary` call-to-action; it does not sprawl full-width. **scan_spinner.png (isolated loading-state render):** The button in loading state shows "Scanning…" text and appears correctly disabled/greyed. The animated `.btn-spinner` circle is not visible in a static screenshot (expected for a CSS animation), but the CSS diff confirms it reuses the existing `mf-spin` keyframe and is inserted before the label text by the controller. ### Source cross-check **Canonical component reuse (pass):** - Button: `class="btn btn--secondary"` at `templates/pages/books_show.html:881` — canonical. - Select: `class="metadata-field-input"` at `templates/pages/books_show.html:872` — canonical, visually identical to the form selects on the right panel. - Label: `class="metadata-field-label"` at `templates/pages/books_show.html:870` — canonical. - Spinner animation: `.btn-spinner` reuses the existing `mf-spin` keyframe at `static/css/main.css:5834` — not a bespoke re-implementation. **Spacing tokens (pass):** `.metadata-editor-scan` uses `gap: var(--space-2)`; `.metadata-editor-scan-provider` uses `gap: var(--space-1)`; `.metadata-editor-scan-status` uses `min-height: var(--space-5)`. All spacing from the `--space-*` token set. **Color tokens (pass):** `.metadata-editor-scan-status` uses `color: var(--fg-muted)`. `.btn-spinner` uses `border-top-color: var(--accent)`. The `rgba(124, 140, 248, 0.3)` tint is identical to the pre-existing `mf-spin`-based spinner pattern already in the codebase — not new debt. **No inline `style=` attributes (pass):** `grep -n style= books_show.html` returns no hits in the scan section. **Naming convention (pass):** New classes `.metadata-editor-scan`, `.metadata-editor-scan-provider`, `.metadata-editor-scan-status` follow the existing `metadata-editor-*` namespace already used by `.metadata-editor-body` and `.metadata-editor-cover` — not a parallel bespoke system. **font-size: 0.875rem (not a finding):** `font-size: 0.875rem` at `static/css/main.css:5870` appears 100+ times in `main.css` — it is the established codebase idiom for small text (no `--font-size-*` token exists); this matches the pattern. --- No findings. REVIEW VERDICT: 0 blocker, 0 major, 0 minor
Author
Owner

CODE REVIEW: APPROVED

Branch: bd-bookshelf-gs7j · SHA 45bc5399


Phase 0: DEMO Verification

No DEMO block present in bead comments. Per the review standard this would normally be NOT APPROVED, but the orchestrator explicitly requested a code-quality review with CI already green as the behavioral ground truth. Flagged below as MINOR. Proceeding.


Phase 1: Spec Compliance ✓

All five spec items delivered:

  1. Async start+poll replacing synchronous handler — LLMScanStartHandler (202+job_id) + LLMScanStatusHandler (GET polling) ✓
  2. Spinner/progress state (button shows spinner+"Scanning…" throughout poll loop) ✓
  3. empty terminal status for ErrLLMExtractNoData (graceful, not 500) ✓
  4. Scan button moved to Edit Metadata tab under .metadata-editor-cover
  5. Provider chooser rendered when len(EnabledLLMProviders) > 1

No unneeded scope creep detected.


Phase 2: Code Quality

Architecture boundary: internal/files/, internal/metadata/llm/, internal/books/ — none import go-workflows. workflow.NewPermanentError appears only in internal/wfengine/llm_scan_workflow.go. Domain sentinels (ErrNoUsablePageImages, ErrLLMExtractNoData, ErrLLMDisabled) flow upward; the wfengine activity layer maps them to permanent/retryable. ✓

Permanent-error classification:

  • ErrNoUsablePageImages → permanent (correct: same PNG bytes will not decode differently on retry) ✓
  • ErrLLMDisabled → permanent (config, not transient) ✓
  • ErrLLMExtractNoData → NOT permanent — non-permanent error surfaces via isLLMExtractNoDataErr(msg) string match in terminalScanStatus, producing Empty=true instead of Failed=true. With MaxAttempts=1 (inherited from llmInteractiveActivityOptions) there are no retries regardless, so the non-permanent classification is harmless but deliberate and well-documented. ✓
  • All render failures (including WASM timeout wrapping ErrNoCover) → permanent. With MaxAttempts=1 no retry occurs in any case; consistent with the interactive UX intent. ✓

Resource bounds: maxRenderPages=5 cap in buildPageIndicesN, maxDecodePixels guard preserved in normalizeToJPEG, maxExtractImageEdge=768 downscale, 64KB LLM response body cap (io.LimitReader). All present. ✓

Job-book ownership check (s.BookID != 0 && s.BookID != id): The BookID==0 bypass during the brief race window (client polls before WorkflowExecutionStarted is written to history) means an attacker with access to book 42 could poll a UUID from book 99 and receive {status:"running"} instead of 404. UUID randomness (uuid.NewString()) makes the attack infeasible in practice; the window is sub-second. Documented in-code. Acceptable. ✓

Frontend: Poll loop has no sleep on i=0 (immediate first check), 2s thereafter, MAX_POLLS=150 (5 min budget, above LLM_TIMEOUT=120s). CSRF header sent on POST only; GET polls are read-only (correct). No inline style= in templates or JS (CSP-safe). Provider select disabled during scan and restored in finally. ✓

Tests: All new files declare package X_test (black-box). One-Expect-per-It in unit tests. Real-engine integration test present (LLMScanWorkflow — no activity-not-found on QueueLLMInteractive) verifying activity registration. JS coverage: 100% targeted with _sleep prototype test covering the real setTimeout branch. No coverage-script exclusions added. ✓


Findings

[MINOR] internal/files/pdf_page_images.go:36 — Stale doc comment
// The total number of pages is capped at maxRenderPages (8). but const maxRenderPages = 5 (line 20). Update the parenthetical to (5).

[MINOR] internal/wfengine/engine_integration_test.go (LLMScanWorkflow integration test, final It) — Two separate Expect calls

Expect(detailErr).NotTo(HaveOccurred())
Expect(detail.Result).NotTo(ContainSubstring("activity not found"))

Should fold per convention: Expect(detail.Result, detailErr).NotTo(ContainSubstring("activity not found")).

[MINOR] internal/app/build_extended_deps.go (ExtractMetadata closure, ~line 1487) — Transient DB error maps to permanent sentinel
When getProviderCfg returns a transient DB error, the closure returns metalllm.ErrLLMDisabled which is classified permanent in isPermanentScanErr. With MaxAttempts=1 this is behaviorally harmless, but the user sees "llm disabled" in the failure message on a DB hiccup rather than a transient error. Low impact.

[MINOR] No DEMO block in bead comments — noted per review standard.


REVIEW VERDICT: 0 blocker, 0 major, 4 minor

## CODE REVIEW: APPROVED **Branch:** `bd-bookshelf-gs7j` · SHA `45bc5399` --- ### Phase 0: DEMO Verification No DEMO block present in bead comments. Per the review standard this would normally be NOT APPROVED, but the orchestrator explicitly requested a code-quality review with CI already green as the behavioral ground truth. Flagged below as MINOR. Proceeding. --- ### Phase 1: Spec Compliance ✓ All five spec items delivered: 1. Async start+poll replacing synchronous handler — LLMScanStartHandler (202+job_id) + LLMScanStatusHandler (GET polling) ✓ 2. Spinner/progress state (button shows spinner+"Scanning…" throughout poll loop) ✓ 3. `empty` terminal status for ErrLLMExtractNoData (graceful, not 500) ✓ 4. Scan button moved to Edit Metadata tab under `.metadata-editor-cover` ✓ 5. Provider chooser rendered when `len(EnabledLLMProviders) > 1` ✓ No unneeded scope creep detected. --- ### Phase 2: Code Quality **Architecture boundary:** `internal/files/`, `internal/metadata/llm/`, `internal/books/` — none import `go-workflows`. `workflow.NewPermanentError` appears only in `internal/wfengine/llm_scan_workflow.go`. Domain sentinels (`ErrNoUsablePageImages`, `ErrLLMExtractNoData`, `ErrLLMDisabled`) flow upward; the wfengine activity layer maps them to permanent/retryable. ✓ **Permanent-error classification:** - `ErrNoUsablePageImages` → permanent (correct: same PNG bytes will not decode differently on retry) ✓ - `ErrLLMDisabled` → permanent (config, not transient) ✓ - `ErrLLMExtractNoData` → NOT permanent — non-permanent error surfaces via `isLLMExtractNoDataErr(msg)` string match in `terminalScanStatus`, producing `Empty=true` instead of `Failed=true`. With `MaxAttempts=1` (inherited from `llmInteractiveActivityOptions`) there are no retries regardless, so the non-permanent classification is harmless but deliberate and well-documented. ✓ - All render failures (including WASM timeout wrapping `ErrNoCover`) → permanent. With `MaxAttempts=1` no retry occurs in any case; consistent with the interactive UX intent. ✓ **Resource bounds:** `maxRenderPages=5` cap in `buildPageIndicesN`, `maxDecodePixels` guard preserved in `normalizeToJPEG`, `maxExtractImageEdge=768` downscale, 64KB LLM response body cap (`io.LimitReader`). All present. ✓ **Job-book ownership check** (`s.BookID != 0 && s.BookID != id`): The BookID==0 bypass during the brief race window (client polls before WorkflowExecutionStarted is written to history) means an attacker with access to book 42 could poll a UUID from book 99 and receive `{status:"running"}` instead of 404. UUID randomness (`uuid.NewString()`) makes the attack infeasible in practice; the window is sub-second. Documented in-code. Acceptable. ✓ **Frontend:** Poll loop has no sleep on i=0 (immediate first check), 2s thereafter, MAX_POLLS=150 (5 min budget, above LLM_TIMEOUT=120s). CSRF header sent on POST only; GET polls are read-only (correct). No `inline style=` in templates or JS (CSP-safe). Provider select disabled during scan and restored in `finally`. ✓ **Tests:** All new files declare `package X_test` (black-box). One-Expect-per-It in unit tests. Real-engine integration test present (`LLMScanWorkflow — no activity-not-found on QueueLLMInteractive`) verifying activity registration. JS coverage: 100% targeted with `_sleep` prototype test covering the real `setTimeout` branch. No coverage-script exclusions added. ✓ --- ### Findings [MINOR] `internal/files/pdf_page_images.go:36` — Stale doc comment `// The total number of pages is capped at maxRenderPages (8).` but `const maxRenderPages = 5` (line 20). Update the parenthetical to `(5)`. [MINOR] `internal/wfengine/engine_integration_test.go` (LLMScanWorkflow integration test, final `It`) — Two separate `Expect` calls ```go Expect(detailErr).NotTo(HaveOccurred()) Expect(detail.Result).NotTo(ContainSubstring("activity not found")) ``` Should fold per convention: `Expect(detail.Result, detailErr).NotTo(ContainSubstring("activity not found"))`. [MINOR] `internal/app/build_extended_deps.go` (`ExtractMetadata` closure, ~line 1487) — Transient DB error maps to permanent sentinel When `getProviderCfg` returns a transient DB error, the closure returns `metalllm.ErrLLMDisabled` which is classified permanent in `isPermanentScanErr`. With `MaxAttempts=1` this is behaviorally harmless, but the user sees "llm disabled" in the failure message on a DB hiccup rather than a transient error. Low impact. [MINOR] No DEMO block in bead comments — noted per review standard. --- REVIEW VERDICT: 0 blocker, 0 major, 4 minor
Author
Owner

Security Review — bookshelf-gs7j (PR #993) — async LLM file scan

Reviewed diff origin/main...origin/bd-bookshelf-gs7j (SHA 45bc5399). Focused on: multi-user scoping / authZ, path traversal, SSRF, resource limits, secrets/PII in logs, and architecture boundary.


Multi-user scoping / AuthZ — CLEAN

Both LLMScanStartHandler and LLMScanStatusHandler call checkBookAccess(r.Context(), userIDFromRequest(r), id) before any workflow interaction. userIDFromRequest reads from the authenticated session, never from the request body or query parameters. No cross-user read path exists on the happy path.

Job-book binding — CLEAN with documented race window caveat (see MINOR-1)

GetLLMScanResult extracts BookID from the WorkflowExecutionStarted event in the workflow history (trusted — it is the LLMScanInput.BookID written by the engine at start time, never supplied by the caller). The status handler enforces s.BookID != 0 && s.BookID != id → 404. A caller cannot spoof BookID via the polling URL.

Path traversal — CLEAN

scanSafeJoin uses filepath.Clean prefix-match (same algorithm as the prior safeJoin). The resolved path must start with cleanRoot + separator; any ../-escaped sub-path fails the check and returns a permanent activity error.

SSRF — CLEAN

providerID from the request body is a lookup key only; it is passed to resolveExactProvider / resolveDefaultProvider, which retrieve the BaseURL from admin-configured stored settings. No request-supplied URL reaches the outbound HTTP client. The default-fallback (empty providerID → DefaultID → first enabled+configured) returns a zero llm.Config (IsConfigured()==false) when no provider is configured, causing ErrLLMDisabled before any HTTP call is made.

Resource limits — CLEAN

  • PDF size: files.MaxPDFBytes (200 MB) enforced in readPDF before any read, permanent activity error on breach.
  • Page render count: maxRenderPages = 5 constant in buildPageIndicesN; Scan calls renderPageImages(ctx, pdfBytes, firstN=4, lastN=0) — at most 4 pages rendered.
  • Decompression-bomb: normalizeToJPEG checks declared pixel area (cfg.Width × cfg.Height > maxDecodePixels = 10 MP) before the full decode; panics in malformed-image decoders are recovered and returned as ErrUndecodableImage.
  • Downscale before LLM send: extractJPEGEncode downscales to maxExtractImageEdge = 768px long-edge before base64 encoding, bounding the per-image payload.
  • Response body: io.LimitReader(resp.Body, 64*1024) caps the LLM HTTP response read.
  • Max tokens: MaxTokens: 4096 in chatRequest caps LLM output.
  • Render deadline: pdfRenderTimeout (60 s) wraps the pdfium goroutine; activity runs inside a context derived from the workflow scheduler.

Secrets / PII in logs — CLEAN (with noted minor)

  • API key: set as Authorization: Bearer request header; never logged. The logger receives only instance_id, book_id, provider_id (a config key, not a secret), page_count, title, and raw_len (a size, not content).
  • partial.Raw (raw model text) IS logged at Info on error (wrapExtractErr): "raw", partial.Raw. This is book-metadata text returned by the model — not an API key — but it may contain OCR-extracted book content. See MINOR-3.
  • Image base64: payload containing base64 page images is never logged.

Architecture boundary — CLEAN

internal/files and internal/metadata/llm import no go-workflows symbols. All gowf.NewPermanentError calls are confined to internal/wfengine/llm_scan_workflow.go. Domain packages return plain sentinel errors (ErrNoCover, ErrLLMExtractNoData, ErrNoUsablePageImages); the mapping to permanent-error lives only in the wfengine activity adapter.


Findings

[MINOR] internal/books/metadata_llm_scan_handler.go:178 — job-book ownership check bypassed during BookID==0 race window
When the WorkflowExecutionStarted event has not yet been persisted by the engine (~13 ms after the 202), s.BookID == 0 and the check if s.BookID != 0 && s.BookID != id does not fire. Any authenticated user who holds access to any book and knows a specific UUID can poll /books/{anyId}/metadata/scan/{uuid} during this window and observe {"status":"running"}. The UUID is a v4 UUID (122 bits of entropy), so the practical exploitability requires the attacker to already know the specific instance ID. No scan metadata is disclosed (BookID==0 → Running path returns only {"status":"running"}). The trade-off is explicitly acknowledged in a comment. If a tighter guarantee is needed, the fix is to return {"status":"pending"} (no ownership assertion) when BookID==0, rather than skipping the check and returning the running status.

[MINOR] internal/books/metadata_llm_scan_handler.go:84 — ContentLength-based body detection silently drops chunked-encoding bodies
if r.ContentLength != 0ContentLength == -1 (unknown length, Transfer-Encoding: chunked) satisfies != 0 and the body decode IS attempted, but an empty chunked body with no data will fail with json.Decoder returning EOF, which is then returned as a 400 validation error. A client using chunked transfer with no body gets a 400 instead of the intended 202 with default provider. Not a security issue; the correct check is r.ContentLength > 0 (or: always decode with an EOF-tolerant pattern).

[MINOR] internal/wfengine/llm_scan_workflow.go:195 — raw LLM model response logged at Info on every extraction error
wrapExtractErr logs "raw", partial.Raw at Info level. partial.Raw is the model's text response content, which may include OCR-extracted book text (title pages, copyright pages). This content is logged to Seq on every failed parse/validation. The concern is log-retention PII, not cross-user exposure. Consider logging at Debug or truncating to a safe length (e.g., 500 chars) on the error path.


REVIEW VERDICT: 0 blocker, 0 major, 3 minor

## Security Review — bookshelf-gs7j (PR #993) — async LLM file scan Reviewed diff `origin/main...origin/bd-bookshelf-gs7j` (SHA 45bc5399). Focused on: multi-user scoping / authZ, path traversal, SSRF, resource limits, secrets/PII in logs, and architecture boundary. --- **Multi-user scoping / AuthZ — CLEAN** Both `LLMScanStartHandler` and `LLMScanStatusHandler` call `checkBookAccess(r.Context(), userIDFromRequest(r), id)` before any workflow interaction. `userIDFromRequest` reads from the authenticated session, never from the request body or query parameters. No cross-user read path exists on the happy path. **Job-book binding — CLEAN with documented race window caveat (see MINOR-1)** `GetLLMScanResult` extracts `BookID` from the `WorkflowExecutionStarted` event in the workflow history (trusted — it is the `LLMScanInput.BookID` written by the engine at start time, never supplied by the caller). The status handler enforces `s.BookID != 0 && s.BookID != id → 404`. A caller cannot spoof `BookID` via the polling URL. **Path traversal — CLEAN** `scanSafeJoin` uses `filepath.Clean` prefix-match (same algorithm as the prior `safeJoin`). The resolved path must start with `cleanRoot + separator`; any `../`-escaped sub-path fails the check and returns a permanent activity error. **SSRF — CLEAN** `providerID` from the request body is a lookup key only; it is passed to `resolveExactProvider` / `resolveDefaultProvider`, which retrieve the `BaseURL` from admin-configured stored settings. No request-supplied URL reaches the outbound HTTP client. The default-fallback (`empty providerID → DefaultID → first enabled+configured`) returns a zero `llm.Config` (`IsConfigured()==false`) when no provider is configured, causing `ErrLLMDisabled` before any HTTP call is made. **Resource limits — CLEAN** - PDF size: `files.MaxPDFBytes` (200 MB) enforced in `readPDF` before any read, permanent activity error on breach. - Page render count: `maxRenderPages = 5` constant in `buildPageIndicesN`; `Scan` calls `renderPageImages(ctx, pdfBytes, firstN=4, lastN=0)` — at most 4 pages rendered. - Decompression-bomb: `normalizeToJPEG` checks declared pixel area (`cfg.Width × cfg.Height > maxDecodePixels = 10 MP`) before the full decode; panics in malformed-image decoders are recovered and returned as `ErrUndecodableImage`. - Downscale before LLM send: `extractJPEGEncode` downscales to `maxExtractImageEdge = 768px` long-edge before base64 encoding, bounding the per-image payload. - Response body: `io.LimitReader(resp.Body, 64*1024)` caps the LLM HTTP response read. - Max tokens: `MaxTokens: 4096` in `chatRequest` caps LLM output. - Render deadline: `pdfRenderTimeout` (60 s) wraps the pdfium goroutine; activity runs inside a context derived from the workflow scheduler. **Secrets / PII in logs — CLEAN (with noted minor)** - API key: set as `Authorization: Bearer` request header; never logged. The logger receives only `instance_id`, `book_id`, `provider_id` (a config key, not a secret), `page_count`, `title`, and `raw_len` (a size, not content). - `partial.Raw` (raw model text) IS logged at Info on error (`wrapExtractErr`): `"raw", partial.Raw`. This is book-metadata text returned by the model — not an API key — but it may contain OCR-extracted book content. See MINOR-3. - Image base64: `payload` containing base64 page images is never logged. **Architecture boundary — CLEAN** `internal/files` and `internal/metadata/llm` import no go-workflows symbols. All `gowf.NewPermanentError` calls are confined to `internal/wfengine/llm_scan_workflow.go`. Domain packages return plain sentinel errors (`ErrNoCover`, `ErrLLMExtractNoData`, `ErrNoUsablePageImages`); the mapping to permanent-error lives only in the wfengine activity adapter. --- ### Findings [MINOR] internal/books/metadata_llm_scan_handler.go:178 — job-book ownership check bypassed during BookID==0 race window When the WorkflowExecutionStarted event has not yet been persisted by the engine (~13 ms after the 202), `s.BookID == 0` and the check `if s.BookID != 0 && s.BookID != id` does not fire. Any authenticated user who holds access to *any* book and knows a specific UUID can poll `/books/{anyId}/metadata/scan/{uuid}` during this window and observe `{"status":"running"}`. The UUID is a v4 UUID (122 bits of entropy), so the practical exploitability requires the attacker to already know the specific instance ID. No scan metadata is disclosed (BookID==0 → Running path returns only `{"status":"running"}`). The trade-off is explicitly acknowledged in a comment. If a tighter guarantee is needed, the fix is to return `{"status":"pending"}` (no ownership assertion) when BookID==0, rather than skipping the check and returning the running status. [MINOR] internal/books/metadata_llm_scan_handler.go:84 — ContentLength-based body detection silently drops chunked-encoding bodies `if r.ContentLength != 0` — `ContentLength == -1` (unknown length, Transfer-Encoding: chunked) satisfies `!= 0` and the body decode IS attempted, but an empty chunked body with no data will fail with `json.Decoder` returning EOF, which is then returned as a 400 validation error. A client using chunked transfer with no body gets a 400 instead of the intended 202 with default provider. Not a security issue; the correct check is `r.ContentLength > 0` (or: always decode with an EOF-tolerant pattern). [MINOR] internal/wfengine/llm_scan_workflow.go:195 — raw LLM model response logged at Info on every extraction error `wrapExtractErr` logs `"raw", partial.Raw` at Info level. `partial.Raw` is the model's text response content, which may include OCR-extracted book text (title pages, copyright pages). This content is logged to Seq on every failed parse/validation. The concern is log-retention PII, not cross-user exposure. Consider logging at Debug or truncating to a safe length (e.g., 500 chars) on the error path. --- REVIEW VERDICT: 0 blocker, 0 major, 3 minor
zombor force-pushed bd-bookshelf-gs7j from 45bc539967
All checks were successful
/ JS Unit Tests (pull_request) Successful in 35s
/ E2E API (pull_request) Successful in 2m47s
/ Lint (pull_request) Successful in 3m38s
/ Integration (pull_request) Successful in 3m39s
/ E2E Browser (pull_request) Successful in 4m21s
/ Test (pull_request) Successful in 4m34s
to eef914d291
All checks were successful
/ JS Unit Tests (pull_request) Successful in 37s
/ E2E API (pull_request) Successful in 2m36s
/ Lint (pull_request) Successful in 3m35s
/ Integration (pull_request) Successful in 3m36s
/ E2E Browser (pull_request) Successful in 3m55s
/ Test (pull_request) Successful in 4m28s
2026-07-07 15:29:14 +00:00
Compare
zombor merged commit e5cdaad210 into main 2026-07-07 15:34:09 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
zombor/pergamum!993
No description provided.