• 谢宇轩's avatar
    Refactor: scripts own file I/O, agent only produces summaries · bd73bb2c
    谢宇轩 authored
    Root cause: harvest.js fetched content but wrote NO files — it dumped a
    giant manifest (with full body) to stdout and left all document I/O to
    the agent. This caused silent save failures (agent hand-wrote fragile
    Node scripts) and token waste (full body in context → 13 python re-parses).
    
    Architecture change — scripts do all deterministic I/O, full body never
    enters the agent context:
    
      Archive:  harvest.js <src> <n>  → writes 原文.md + registry + index
                                         (sentinel placeholder for 中文标题)
                                         + compact manifest (bodyPreview only)
                agent writes .summaries.json (中文标题 + 四段摘要 + 分类)
                finalize.js <dayDir>    → writes 摘要.md, patches sentinels,
                                         updates registry/index, cleans up
    
      Preview:  harvest.js <src> <n> --preview [--full]  → read-only, no writes
    
    Changes:
    - harvest.js: wire up generateOriginalDoc/updateIndex/appendRegistry
      (were defined but never called); compact manifest (no full body);
      sentinel placeholder links; --preview/--full read-only mode; real
      asset count (was "articles with images"); empty-alt guard (no bare **);
      manifest merge across runs (fixes dangling-sentinel data loss);
      --help; module exports for finalize.js
    - finalize.js (new): consumes .manifest.json + .summaries.json, writes
      summary docs, patches sentinel links, updates registry/index, cleans
      up temp files; missing summary → warn + skip (non-fatal); --help
    - fetcher-direct.js: author extraction via JSON-LD → <meta name=author>
      → rel=author (was matching arbitrary JSON-LD names → empty authors);
      body noise filter (share buttons, boilerplate); image tracking-pixel
      + duplicate filtering
    - fetcher-browser.js: CDP port 9223 → 9222 (match docs/chromium launch)
    - SKILL.md: 3-step archive workflow, --preview flow, First-Run Setup
      onboarding, guardrails, .summaries.json contract, sentinel docs
    - README.md: architecture diagram, division of labor, task flows, limits
    bd73bb2c
Name
Last commit
Last update
references Loading commit data...
scripts Loading commit data...
.gitignore Loading commit data...
README.md Loading commit data...
SKILL.md Loading commit data...
package-lock.json Loading commit data...
package.json Loading commit data...