• 谢宇轩's avatar
    Refactor: scripts own file I/O, agent only produces summaries · bd73bb2c
    谢宇轩 authored
    Root cause: harvest.js fetched content but wrote NO files — it dumped a
    giant manifest (with full body) to stdout and left all document I/O to
    the agent. This caused silent save failures (agent hand-wrote fragile
    Node scripts) and token waste (full body in context → 13 python re-parses).
    
    Architecture change — scripts do all deterministic I/O, full body never
    enters the agent context:
    
      Archive:  harvest.js <src> <n>  → writes 原文.md + registry + index
                                         (sentinel placeholder for 中文标题)
                                         + compact manifest (bodyPreview only)
                agent writes .summaries.json (中文标题 + 四段摘要 + 分类)
                finalize.js <dayDir>    → writes 摘要.md, patches sentinels,
                                         updates registry/index, cleans up
    
      Preview:  harvest.js <src> <n> --preview [--full]  → read-only, no writes
    
    Changes:
    - harvest.js: wire up generateOriginalDoc/updateIndex/appendRegistry
      (were defined but never called); compact manifest (no full body);
      sentinel placeholder links; --preview/--full read-only mode; real
      asset count (was "articles with images"); empty-alt guard (no bare **);
      manifest merge across runs (fixes dangling-sentinel data loss);
      --help; module exports for finalize.js
    - finalize.js (new): consumes .manifest.json + .summaries.json, writes
      summary docs, patches sentinel links, updates registry/index, cleans
      up temp files; missing summary → warn + skip (non-fatal); --help
    - fetcher-direct.js: author extraction via JSON-LD → <meta name=author>
      → rel=author (was matching arbitrary JSON-LD names → empty authors);
      body noise filter (share buttons, boilerplate); image tracking-pixel
      + duplicate filtering
    - fetcher-browser.js: CDP port 9223 → 9222 (match docs/chromium launch)
    - SKILL.md: 3-step archive workflow, --preview flow, First-Run Setup
      onboarding, guardrails, .summaries.json contract, sentinel docs
    - README.md: architecture diagram, division of labor, task flows, limits
    bd73bb2c
README.md 10.1 KB