- 30 Jul, 2026 2 commits
-
-
谢宇轩 authored
Export reuters-business, reuters-markets, reuters-world as shareable .nhsource.json bundles (the committed unit for sources). Untrack the helpers/ directory — recipes/fixtures are now installed from bundles or authored locally; only bundles are committed. Add helpers/ to .gitignore so product files stay on disk for harvest but out of git.
-
谢宇轩 authored
Reuters: - Remove homepage source (reuters) from config; keep business/markets/world - reuters-markets: fix stale hashed CSS class (only 9/20 articles) → composite data-testid listSelector (MediaStoryCard/BasicCard/HeroCard/ HubCard/AuthorCard/OurColumnists), now captures all 20 - All three: add validateArticle (og:type/JSON-LD/body), broaden urlPattern to site-wide date-suffix match, exclude podcasts/newsletter - Verified: Business 20, Markets 20 (was 9), World 19 article links nikkei-tech: bring recipe + fixtures into tree (composite listSelector, validateArticle, 7-article/5-section regression fixtures).
-
- 29 Jul, 2026 3 commits
-
-
谢宇轩 authored
Add export/install to source-manage.js: pack a source (config entry + recipe + fixtures) into a single self-contained .nhsource.json, install on another machine in one command. Bundle excludes machine-specific fields (vaultPath/enabled/articleUrlPattern); --force overwrites on id conflict (matches the helper command convention). Path-traversal guarded. Includes bundles/nikkei-tech.nhsource.json as a ready-to-install example (the bundle itself carries the recipe + fixtures; the unpacked helpers/nikkei-tech.* files stay out of this commit). Documents the flow in SKILL.md (Quick Reference + Sharing sources) and references/source-management.md.
-
谢宇轩 authored
-
谢宇轩 authored
-
- 27 Jul, 2026 1 commit
-
-
谢宇轩 authored
-
- 24 Jul, 2026 3 commits
-
-
谢宇轩 authored
Chromium launch was fragile: the documented `chromium ... & sleep 5` assumed `chromium` was on PATH, but on macOS the binary lives inside an .app bundle and the command silently failed (&>/dev/null swallowed the error). sleep 5 was not a readiness check, getWsUrl() had zero retries, and the health check depended on python3. Changes: - Add scripts/ensure-chromium.js: cross-platform binary discovery (macOS .app bundles, Linux PATH, Windows install dirs), stale-SingletonLock cleanup, spawn with CDP, poll /json/version until ready (30s), and a diagnostic stderr-log tail on timeout. Pure node http, no python3. Supports config.chromium.{cdpPort,headless,executablePath,userDataDir}. - harvest.js: auto-call ensureChromium before browser-source fetch (no-op if already running). Sources needing login can set chromium.userDataDir to a logged-in profile; auto-launches then carry the cookies. - fetcher-browser.js: getWsUrl() now retries (6x500ms) and reads the CDP port from config (env > config.chromium.cdpPort > 9222). - config-template.json: add optional chromium block. - SKILL.md / README.md: replace the manual launch command with the auto-launch explanation; add --help docs and dedicated-profile guidance. Also clean up the skill flow to cut a 16-step run down to ~4: - All commands now use the `scripts/` prefix (bare `node harvest.js` failed with MODULE_NOT_FOUND — 5x in the trace). - First-Run Setup becomes a single pre-flight gate: print OK -> skip to Workflow, don't re-check vault/source/config mid-task. - Workflow adds a "stay lean" note: if the user named a source, don't run source-manage.js list; the archive flow is exactly 3 script calls. -
谢宇轩 authored
- Point config setup + Agent 自主配置 to SKILL.md's First-Run Setup (was duplicating onboarding steps that now live in SKILL.md) - Add --help mention for both scripts - Update 归档产物结构 with sentinel intermediate state + temp files (.manifest.json / .summaries.json) - Update budget reference: full body no longer in context → lower cost (~$1.0 archive, ~$0.3 preview); max-budget $5 → $3
-
谢宇轩 authored
Root cause: harvest.js fetched content but wrote NO files — it dumped a giant manifest (with full body) to stdout and left all document I/O to the agent. This caused silent save failures (agent hand-wrote fragile Node scripts) and token waste (full body in context → 13 python re-parses). Architecture change — scripts do all deterministic I/O, full body never enters the agent context: Archive: harvest.js <src> <n> → writes 原文.md + registry + index (sentinel placeholder for 中文标题) + compact manifest (bodyPreview only) agent writes .summaries.json (中文标题 + 四段摘要 + 分类) finalize.js <dayDir> → writes 摘要.md, patches sentinels, updates registry/index, cleans up Preview: harvest.js <src> <n> --preview [--full] → read-only, no writes Changes: - harvest.js: wire up generateOriginalDoc/updateIndex/appendRegistry (were defined but never called); compact manifest (no full body); sentinel placeholder links; --preview/--full read-only mode; real asset count (was "articles with images"); empty-alt guard (no bare **); manifest merge across runs (fixes dangling-sentinel data loss); --help; module exports for finalize.js - finalize.js (new): consumes .manifest.json + .summaries.json, writes summary docs, patches sentinel links, updates registry/index, cleans up temp files; missing summary → warn + skip (non-fatal); --help - fetcher-direct.js: author extraction via JSON-LD → <meta name=author> → rel=author (was matching arbitrary JSON-LD names → empty authors); body noise filter (share buttons, boilerplate); image tracking-pixel + duplicate filtering - fetcher-browser.js: CDP port 9223 → 9222 (match docs/chromium launch) - SKILL.md: 3-step archive workflow, --preview flow, First-Run Setup onboarding, guardrails, .summaries.json contract, sentinel docs - README.md: architecture diagram, division of labor, task flows, limits
-
- 23 Jul, 2026 3 commits
-
-
谢宇轩 authored
-
xieyichu authored
Discord publishing (per-article threads, weekly summary threads, discordChannelId field, weeklyThreads config) has been removed from the skill. This functionality will be handled by separate skills that depend on the Agent's message tooling. Changes: - Delete scripts/save-weekly-thread.js - harvest.js: remove isoWeekKey/weekDateRange helpers, Discord manifest fields (discordChannelId, weekKey, weekRange, weeklyThreadId), and publishing comment block - source-manage.js: remove CHANNEL column from list, discordChannelId from add/edit/usage - config-template.json: remove weeklyThreads and discordChannelId fields - source-management.md: remove Discord fields and Weekly Thread section - SKILL.md: remove Publishing to Discord section, Workflow step 7, discordThreadId from registry format, Discord from description/intro - README.md: remove all Discord references from capability table, architecture diagram, prerequisites, and limitations - .gitignore: add .zcode/ workspace dir
-
xieyichu authored
A Claude Code skill for harvesting news articles from configured sources, archiving to Obsidian vault with Chinese summaries, and publishing to Discord. - SKILL.md: skill definition and workflow documentation - scripts/: harvest.js, source-manage.js, fetcher-direct.js, fetcher-browser.js, save-weekly-thread.js - references/: config template (sanitized) and source management guide - README.md: installation, usage, and agent self-configuration guide - .gitignore: excludes config.json, data/, node_modules/ (machine-specific runtime files)
-