- 06 Aug, 2026 1 commit
-
-
谢宇轩 authored
-
- 05 Aug, 2026 2 commits
- 04 Aug, 2026 2 commits
- 03 Aug, 2026 1 commit
-
-
谢宇轩 authored
-
- 31 Jul, 2026 3 commits
-
-
谢宇轩 authored
-
谢宇轩 authored
- 前置条件: macOS 手写 Chromium 命令 → ensure-chromium.js --list/--login 跨平台引导 - 添加新源流程: 3步闭环 → 4步闭环 (加 recipe-test 锁定) - 归档产物结构: 摘要文件移到 summary/ 子目录, 加 category/tags/date 字段说明 - 分工说明: tags 从 [category,source] 改为 agent 从摘要提取 ≤3 语义关键词 - 浏览器说明: 加 --login 引导登录态, --list/--check 检测 - 能力概览表: "自动分类" → "分类与标签" (脚本猜测+agent覆盖 / agent提取)
-
谢宇轩 authored
SKILL.md 精简 (476→296行): - Document Formats 移到 references/document-formats.md (每次调用少加载1/3上下文) - 摘要 batching 改成单一决策树 (N≤8 一次写 / N>8 批≤4),消除4处重复 - 添加新源流程统一为 inspect --write 4步闭环,消除与 source-management.md 的冲突 - 新增 References 小节索引三个 references 文件 CDP 登录引导 (跨平台): - ensure-chromium.js 新增 --list (检测平台+可用浏览器) 和 --login (窗口化登录) - --login 用固定 profile (~/.news-harvester/chromium-profile),端口占用时报告退出 - SKILL.md 的 Dedicated profile 从 macOS-only 手写命令改为跨平台3步流程 - --no-sandbox 仅在 root 下添加 (修掉 macOS 的不受支持标记警告) - config-template.json 加 browser method 示例源 tag/category 改造: - tags 从 [category, source] 改为 agent 从摘要提取的 ≤3 语义关键词 - 新增 frontmatter category 字段 (独立于 tags,支持 Obsidian 筛选) - finalize 回写原文.md 的 tags+category frontmatter (与摘要.md 一致) - index.md 文章列表新增"原文标题"列,标签显示提取的 - 摘要文件移到 <dayDir>/summary/ 子目录 (与 assets/ 同级,wikilink 保持裸标题) 日期修复: - frontmatter date: 存完整 ISO 8601 时间戳 (原 substring(0,10) 截断导致跨时区差一天) - fetcher-direct.js 正则放宽,捕获含毫秒/时区偏移的完整 ISO - registry article 加 date 字段 (之前未存,导致 index 日期列空) - callout 发布时间显示上海时区 (UTC+8),来源/URL 各自另起一行 - index 表格用紧凑 YYYY-MM-DD (dateCell),存储用完整 ISO (dateShort)
-
- 30 Jul, 2026 6 commits
-
-
谢宇轩 authored
-
谢宇轩 authored
When a harvest produced 15+ articles, asking the agent to emit the entire .summaries.json in a single response hit the model's output-token ceiling — the response was truncated and the tail articles silently got no summary (their 原文 kept the __SUMMARY_<slug>__ sentinel forever). Reproduced every time a high-volume source ran; ≤9-article sources never hit it. Add scripts/summary-add.js to incrementally merge small batches (≤4) into .summaries.json (keyed by url, overwrite-no-dup), with a --status mode that diffs manifest vs summaries so interrupted runs can resume. Rewrite SKILL.md Flow A Step 2 + Guardrails to mandate batching, and surface a batchHint in harvest.js's manifest so the agent sees the guidance on first contact.
-
谢宇轩 authored
Two new Nikkei Asia section sources, built on the same composite listSelector + validateArticle pattern proven on nikkei-tech: - nikkei-world: https://asia.nikkei.com/location → 日经亚洲/世界 - nikkei-markets: https://asia.nikkei.com/business/markets → 日经亚洲/市场 Both pages share Nikkei's card template (StreamArticleCard + SpotlightArticleCard + SecondaryArticleCard), so they reuse the same recipe. Preview verified: validateArticle correctly skips section pages, real articles have 1700-5000 chars body content.
-
谢宇轩 authored
Export reuters-business, reuters-markets, reuters-world as shareable .nhsource.json bundles (the committed unit for sources). Untrack the helpers/ directory — recipes/fixtures are now installed from bundles or authored locally; only bundles are committed. Add helpers/ to .gitignore so product files stay on disk for harvest but out of git.
-
谢宇轩 authored
Reuters: - Remove homepage source (reuters) from config; keep business/markets/world - reuters-markets: fix stale hashed CSS class (only 9/20 articles) → composite data-testid listSelector (MediaStoryCard/BasicCard/HeroCard/ HubCard/AuthorCard/OurColumnists), now captures all 20 - All three: add validateArticle (og:type/JSON-LD/body), broaden urlPattern to site-wide date-suffix match, exclude podcasts/newsletter - Verified: Business 20, Markets 20 (was 9), World 19 article links nikkei-tech: bring recipe + fixtures into tree (composite listSelector, validateArticle, 7-article/5-section regression fixtures).
-
- 29 Jul, 2026 4 commits
-
-
谢宇轩 authored
Add export/install to source-manage.js: pack a source (config entry + recipe + fixtures) into a single self-contained .nhsource.json, install on another machine in one command. Bundle excludes machine-specific fields (vaultPath/enabled/articleUrlPattern); --force overwrites on id conflict (matches the helper command convention). Path-traversal guarded. Includes bundles/nikkei-tech.nhsource.json as a ready-to-install example (the bundle itself carries the recipe + fixtures; the unpacked helpers/nikkei-tech.* files stay out of this commit). Documents the flow in SKILL.md (Quick Reference + Sharing sources) and references/source-management.md.
-
谢宇轩 authored
-
谢宇轩 authored
- 27 Jul, 2026 1 commit
-
-
谢宇轩 authored
-
- 24 Jul, 2026 3 commits
-
-
谢宇轩 authored
Chromium launch was fragile: the documented `chromium ... & sleep 5` assumed `chromium` was on PATH, but on macOS the binary lives inside an .app bundle and the command silently failed (&>/dev/null swallowed the error). sleep 5 was not a readiness check, getWsUrl() had zero retries, and the health check depended on python3. Changes: - Add scripts/ensure-chromium.js: cross-platform binary discovery (macOS .app bundles, Linux PATH, Windows install dirs), stale-SingletonLock cleanup, spawn with CDP, poll /json/version until ready (30s), and a diagnostic stderr-log tail on timeout. Pure node http, no python3. Supports config.chromium.{cdpPort,headless,executablePath,userDataDir}. - harvest.js: auto-call ensureChromium before browser-source fetch (no-op if already running). Sources needing login can set chromium.userDataDir to a logged-in profile; auto-launches then carry the cookies. - fetcher-browser.js: getWsUrl() now retries (6x500ms) and reads the CDP port from config (env > config.chromium.cdpPort > 9222). - config-template.json: add optional chromium block. - SKILL.md / README.md: replace the manual launch command with the auto-launch explanation; add --help docs and dedicated-profile guidance. Also clean up the skill flow to cut a 16-step run down to ~4: - All commands now use the `scripts/` prefix (bare `node harvest.js` failed with MODULE_NOT_FOUND — 5x in the trace). - First-Run Setup becomes a single pre-flight gate: print OK -> skip to Workflow, don't re-check vault/source/config mid-task. - Workflow adds a "stay lean" note: if the user named a source, don't run source-manage.js list; the archive flow is exactly 3 script calls. -
谢宇轩 authored
- Point config setup + Agent 自主配置 to SKILL.md's First-Run Setup (was duplicating onboarding steps that now live in SKILL.md) - Add --help mention for both scripts - Update 归档产物结构 with sentinel intermediate state + temp files (.manifest.json / .summaries.json) - Update budget reference: full body no longer in context → lower cost (~$1.0 archive, ~$0.3 preview); max-budget $5 → $3
-
谢宇轩 authored
Root cause: harvest.js fetched content but wrote NO files — it dumped a giant manifest (with full body) to stdout and left all document I/O to the agent. This caused silent save failures (agent hand-wrote fragile Node scripts) and token waste (full body in context → 13 python re-parses). Architecture change — scripts do all deterministic I/O, full body never enters the agent context: Archive: harvest.js <src> <n> → writes 原文.md + registry + index (sentinel placeholder for 中文标题) + compact manifest (bodyPreview only) agent writes .summaries.json (中文标题 + 四段摘要 + 分类) finalize.js <dayDir> → writes 摘要.md, patches sentinels, updates registry/index, cleans up Preview: harvest.js <src> <n> --preview [--full] → read-only, no writes Changes: - harvest.js: wire up generateOriginalDoc/updateIndex/appendRegistry (were defined but never called); compact manifest (no full body); sentinel placeholder links; --preview/--full read-only mode; real asset count (was "articles with images"); empty-alt guard (no bare **); manifest merge across runs (fixes dangling-sentinel data loss); --help; module exports for finalize.js - finalize.js (new): consumes .manifest.json + .summaries.json, writes summary docs, patches sentinel links, updates registry/index, cleans up temp files; missing summary → warn + skip (non-fatal); --help - fetcher-direct.js: author extraction via JSON-LD → <meta name=author> → rel=author (was matching arbitrary JSON-LD names → empty authors); body noise filter (share buttons, boilerplate); image tracking-pixel + duplicate filtering - fetcher-browser.js: CDP port 9223 → 9222 (match docs/chromium launch) - SKILL.md: 3-step archive workflow, --preview flow, First-Run Setup onboarding, guardrails, .summaries.json contract, sentinel docs - README.md: architecture diagram, division of labor, task flows, limits
-
- 23 Jul, 2026 3 commits
-
-
谢宇轩 authored
-
xieyichu authored
Discord publishing (per-article threads, weekly summary threads, discordChannelId field, weeklyThreads config) has been removed from the skill. This functionality will be handled by separate skills that depend on the Agent's message tooling. Changes: - Delete scripts/save-weekly-thread.js - harvest.js: remove isoWeekKey/weekDateRange helpers, Discord manifest fields (discordChannelId, weekKey, weekRange, weeklyThreadId), and publishing comment block - source-manage.js: remove CHANNEL column from list, discordChannelId from add/edit/usage - config-template.json: remove weeklyThreads and discordChannelId fields - source-management.md: remove Discord fields and Weekly Thread section - SKILL.md: remove Publishing to Discord section, Workflow step 7, discordThreadId from registry format, Discord from description/intro - README.md: remove all Discord references from capability table, architecture diagram, prerequisites, and limitations - .gitignore: add .zcode/ workspace dir
-
xieyichu authored
A Claude Code skill for harvesting news articles from configured sources, archiving to Obsidian vault with Chinese summaries, and publishing to Discord. - SKILL.md: skill definition and workflow documentation - scripts/: harvest.js, source-manage.js, fetcher-direct.js, fetcher-browser.js, save-weekly-thread.js - references/: config template (sanitized) and source management guide - README.md: installation, usage, and agent self-configuration guide - .gitignore: excludes config.json, data/, node_modules/ (machine-specific runtime files)
-