Initial commit: news-harvester skill

A Claude Code skill for harvesting news articles from configured sources,
archiving to Obsidian vault with Chinese summaries, and publishing to Discord.

- SKILL.md: skill definition and workflow documentation
- scripts/: harvest.js, source-manage.js, fetcher-direct.js, fetcher-browser.js, save-weekly-thread.js
- references/: config template (sanitized) and source management guide
- README.md: installation, usage, and agent self-configuration guide
- .gitignore: excludes config.json, data/, node_modules/ (machine-specific runtime files)

.gitignore

+# Machine-specific runtime config (auto-generated from references/config-template.json)
+config.json
+config.json.bak
+
+# Runtime dedup registry
+data/
+
+# Dependencies
+node_modules/
+
+# Logs / temp
+*.log
+/tmp/

README.md

+# News Harvester Skill
+
+一个 [Claude Code](https://docs.claude.com/en/docs/claude-code) Skill：从配置的新闻源（Reuters、The Conversation、The Guardian 等）采集最新文章，生成中文摘要 + 英文原文，归档到 Obsidian vault，并可选发布到 Discord 论坛频道。支持去重、分类、自动周报。
+
+> 适用于任何能加载 Claude Code Skill 的 Agent 环境（交互模式 / 无头模式 `-p`）。
+
+---
+
+## 能力概览
+
+| 能力 | 说明 |
+|------|------|
+| 🗂 数据源管理 | 添加 / 列出 / 编辑 / 启用禁用新闻源 |
+| 📡 文章采集 | 抓取首页链接 → 提取全文（标题/日期/作者/正文/配图） |
+| 🗄 Obsidian 归档 | 每篇文章生成中文摘要 + 英文原文 + 当日索引 + 去重注册表 |
+| 📢 Discord 发布 | 每篇文章一个论坛帖 + 每周汇总帖（可选） |
+| 🔁 去重 | 滑动窗口（默认 7 天）的 registry，避免重复采集 |
+| 🏷 自动分类 | 基于关键词映射到 7 个分类（地缘政治/科技AI/中美关系/财经/社会/国际冲突/国际） |
+
+---
+
+## 工作原理
+
+```
+config.json ──▶ harvest.js ──▶ fetcher-*.js ──▶ 文章数据(JSON)
+  (数据源)       (主调度器)      (direct/browser)     │
+       ▲              │                               ▼
+       │         dedup 去重                         Agent(LLM)
+       │         (registry)                    翻译/摘要/分类/发布
+       │                                           │      ▼
+  save-weekly-thread.js ◀──────────────────── Agent  ┌─────────┐
+  (持久化周帖ID)                                 ├─▶ Obsidian
+                                                   └─▶ Discord
+```
+
+**脚本与 Agent 分工**：
+- **脚本**（`scripts/`）：确定性工作 —— HTTP 请求、HTML 解析、图片下载、去重、文件写入
+- **Agent**（LLM）：智能工作 —— 中英翻译、生成中文摘要、文章分类、Discord 发布
+
+`harvest.js` 完成采集后输出 JSON manifest，提示 Agent 接管翻译与发布。
+
+---
+
+## 文件结构
+
+```
+news-harvester/
+├── SKILL.md                          # Skill 定义（Agent 加载入口）
+├── README.md                         # 本文件
+├── .gitignore
+├── package.json                      # Node 依赖（ws，仅 browser 方法用）
+├── config.json                       # ⚠️ 运行时生成，不入库
+├── references/
+│   ├── config-template.json          # config.json 的模板（首次运行自动复制）
+│   └── source-management.md          # 数据源管理详细说明
+├── scripts/
+│   ├── harvest.js                    # 主入口：node harvest.js <source_id> <count>
+│   ├── source-manage.js              # 数据源增删改查
+│   ├── fetcher-direct.js             # 纯 HTTP 抓取（开放站点）
+│   ├── fetcher-browser.js            # Chromium CDP 抓取（JS 渲染站点）
+│   └── save-weekly-thread.js         # 保存 Discord 周帖 ID
+└── data/                             # ⚠️ 运行时去重数据，不入库
+    └── registry.json
+```
+
+---
+
+## 安装
+
+### 前置条件
+
+- [Node.js](https://nodejs.org/) ≥ 18
+- 一个可写的 Obsidian vault 目录
+- （可选）Chromium —— 仅当使用 `browser` 方法的源（如 reuters）时需要
+- （可选）Discord Bot —— 仅当需要发布到 Discord 时需要
+
+### 1. 克隆并安装依赖
+
+```bash
+git clone <your-repo-url> news-harvester
+cd news-harvester
+npm install
+```
+
+### 2. 注册为 Claude Code Skill
+
+Skill 必须放到 Claude Code 的发现路径，否则交互模式和无头模式都找不到它。两种方式任选：
+
+**方式 A：全局符号链接（推荐）**
+
+```bash
+ln -s "$(pwd)" ~/.claude/skills/news-harvester
+```
+
+**方式 B：项目内引用**
+
+在你的目标项目下创建 `.claude/skills/news-harvester` 符号链接，或在 `~/.claude/skills/` 下放置本目录。
+
+> ⚠️ 仅把 `SKILL.md` 放在项目根目录**不会**被发现。必须注册到上述路径。
+
+### 3. 生成配置
+
+首次运行任意脚本会自动从 `references/config-template.json` 复制生成 `config.json`。但**在首次采集前**，必须编辑 `config.json` 设置你的 vault 路径：
+
+```jsonc
+{
+  "vaultPath": "/your/actual/obsidian/vault/path",  // ← 必改
+  "registryWindowDays": 7,
+  "weeklyThreads": {},
+  "sources": [ ... ]
+}
+```
+
+也可以让 Agent 在运行时自动配置（见下方「Agent 自主配置」）。
+
+---
+
+## 使用方式
+
+### 交互模式 / 无头模式
+
+所有任务都可以通过自然语言驱动，Agent 会加载本 skill 并调用脚本：
+
+#### 任务 1：添加数据源
+
+```bash
+node scripts/source-manage.js add \
+  --id the-guardian \
+  --name "The Guardian" \
+  --url https://www.theguardian.com/ \
+  --method direct \
+  --folder "The Guardian"
+```
+
+或让 Agent 自然语言完成：「添加 The Guardian 作为新闻源」。
+
+#### 任务 2：采集最新文章，返回 JSON（不归档）
+
+适合只取数据、不落盘的场景。直接调用 fetcher：
+
+```bash
+# 抓取首页文章链接
+node -e "require('./scripts/fetcher-direct').fetchHomeLinks('https://theconversation.com/us','theconversation\\\\.com/[\\\\w-]+-\\\\d+$').then(l=>console.log(JSON.stringify(l,null,2)))"
+
+# 抓取单篇文章全文
+node scripts/fetcher-direct.js https://theconversation.com/some-article-12345
+```
+
+#### 任务 3：完整采集并归档到 Obsidian
+
+```bash
+node scripts/harvest.js the-conversation 5
+```
+
+`harvest.js` 会：抓取链接 → 去重 → 逐篇抓取全文 → 下载图片 → 输出 JSON manifest。
+然后 Agent 接管：生成中文摘要 → 写入 Obsidian 文档 → 更新 index.md / registry.json → （可选）发布 Discord。
+
+### 无头模式调用示例
+
+```bash
+claude -p \
+  --output-format json \
+  --permission-mode bypassPermissions \
+  --allowed-tools "Bash" "Write" \
+  --max-budget-usd 5 \
+  "使用 news-harvester skill，采集 the-conversation 最新 3 篇文章并归档到 Obsidian"
+```
+
+> 💡 预算参考（实测）：添加数据源 ~$0.8｜返回 JSON ~$1.0｜完整归档 ~$2.2。多步骤任务建议 `--max-budget-usd` 设为 $3–5。
+
+---
+
+## Agent 自主配置
+
+当其他 Agent 首次加载本 skill 时，`config.json` 可能不存在或含模板占位符。Agent 应按以下流程自主配置：
+
+1. **检测**：运行 `node scripts/source-manage.js list`。若报错或 `vaultPath` 仍为 `/path/to/your/obsidian/vault`，说明未配置。
+2. **设置 vaultPath**：询问用户或从环境推断 Obsidian vault 路径，写入 `config.json`。
+3. **添加数据源**（可选）：用 `source-manage.js add` 或直接编辑 `config.json`。
+4. **验证**：确保 `vaultPath` 目录存在且可写（`mkdir -p <path>`）。
+5. **运行采集**：`node scripts/harvest.js <source_id> <count>`。
+
+配置字段说明见 `SKILL.md` 的 **Configuration** 小节。
+
+---
+
+## 数据源配置字段
+
+| 字段 | 说明 | 示例 |
+|------|------|------|
+| `id` | 唯一标识（小写无空格） | `reuters` |
+| `name` | 显示名 | `路透社` |
+| `homepageUrl` | 首页 URL（抓取链接入口） | `https://www.reuters.com/` |
+| `method` | `direct`（纯 HTTP）或 `browser`（需 Chromium） | `direct` |
+| `articleUrlPattern` | 文章 URL 匹配正则 | `theconversation\\.com/[\\w-]+-\\d+$` |
+| `vaultFolder` | vault 下的子文件夹名 | `The Conversation` |
+| `discordChannelId` | Discord 论坛频道 ID（可空） | `1234567890123456789` |
+| `enabled` | 是否启用 | `true` |
+
+---
+
+## 归档产物结构
+
+每次采集写入 `<vaultPath>/<vaultFolder>/YYYYMMDD/`：
+
+```
+YYYYMMDD/
+├── <中文标题>.md                    # 中文摘要（事件概述/背景/关键引语/影响分析）
+├── <英文标题> - <SourceName>原文.md  # 英文原文（含配图）
+├── index.md                         # 当日索引（分类统计 + 文章列表 + 分类导航）
+├── registry.json                    # 去重注册表
+└── assets/
+    └── <slug>-N.jpg                 # 下载的配图
+```
+
+中英文档通过 YAML frontmatter 的 `related` 字段双向 `[[链接]]`。
+
+---
+
+## 限制与注意
+
+- **正文提取依赖正则**：网站改版可能导致提取失败（正文 < 100 字会被跳过）
+- **中文标题需 Agent 翻译**：`harvest.js` 只生成占位标题 `[需翻译] ...`，真正翻译由 Agent 完成
+- **browser 方法依赖 Chromium CDP**（端口 9222），需预先启动：
+  ```bash
+  chromium --no-sandbox --remote-debugging-port=9222 \
+    --user-data-dir=/tmp/chromium-harvest --disable-gpu "about:blank" &>/dev/null &
+  ```
+- **图片下载无重试**：失败仅跳过
+- **Discord 发布由 Agent 完成**：脚本不直接调 Discord API
+
+---
+
+## License
+
+MIT

SKILL.md

+---
+name: news-harvester
+description: Harvest, archive, and publish news articles from configured sources (Reuters, The Conversation, etc.) to Obsidian vault and Discord forum channels. Use this skill whenever the user wants to: fetch latest news from a source, add/list/edit news sources, run a harvest job, check what's been archived, avoid duplicate articles, or set up automated news collection. Triggers on phrases like "抓取新闻", "harvest news", "fetch articles", "add source", "run harvest", "news sources", or any request to collect/archive/publish articles from a website.
+---
+
+# News Harvester Skill
+
+Fetch latest articles from configured news sources, archive them to Obsidian vault with Chinese summaries + original text, and publish to Discord forum channels. Supports browser-based (Chromium CDP) and direct HTTP fetching. Deduplicates via per-day registry with a sliding window.
+
+## Quick Reference
+
+| Task | Command |
+|------|---------|
+| Run harvest | `node harvest.js <source_id> <count>` |
+| Add source | `node source-manage.js add` |
+| List sources | `node source-manage.js list` |
+| Edit source | `node source-manage.js edit <id>` |
+
+Scripts live in: `scripts/` (relative to this skill's root directory)
+Config file: `config.json` (auto-generated from `references/config-template.json` on first run — see Configuration below)
+
+---
+
+## Configuration
+
+`config.json` is **not shipped** with this skill — it is machine-specific (contains local vault path, Discord channel IDs, and runtime thread IDs). On first run, any script auto-creates `config.json` by copying `references/config-template.json`.
+
+Before running a harvest, the agent MUST verify/edit these fields:
+
+| Field | Required? | Description |
+|-------|-----------|-------------|
+| `vaultPath` | ✅ Yes | Absolute path to the Obsidian vault root. Must exist and be writable. Replace the template placeholder `/path/to/your/obsidian/vault`. |
+| `registryWindowDays` | Optional | Dedup sliding window in days (default `7`). |
+| `sources[].discordChannelId` | Optional | Discord forum channel ID for publishing. Leave empty `""` to skip Discord. |
+| `weeklyThreads` | Auto-managed | Populated at runtime by `scripts/save-weekly-thread.js`; never edit by hand. |
+
+To point a source at a different vault subfolder, set `sources[].vaultFolder`. Articles are archived to `<vaultPath>/<vaultFolder>/YYYYMMDD/`.
+
+---
+
+## Workflow
+
+When asked to harvest news, follow these steps:
+
+### 1. Load config
+Read `config.json`. If it doesn't exist, create it from the default template in `references/config-template.json`.
+
+### 2. Check Chromium (for browser sources)
+```bash
+curl -s http://localhost:9222/json/version 2>/dev/null | python3 -c "import sys,json; d=json.load(sys.stdin); print('OK:', d.get('Browser',''))"
+```
+If not running, start it:
+```bash
+chromium --no-sandbox --remote-debugging-port=9222 --user-data-dir=/tmp/chromium-harvest --disable-gpu "about:blank" &>/dev/null &
+sleep 5
+```
+
+### 3. Build dedup set
+Scan the last `registryWindowDays` (default 7) days of registry files:
+```
+<vaultPath>/<source.vaultFolder>/YYYYMMDD/registry.json
+```
+Collect all URLs into a Set.
+
+### 4. Fetch homepage links
+- **browser method**: use `scripts/fetcher-browser.js`
+- **direct method**: use `scripts/fetcher-direct.js`
+
+### 5. Filter + slice
+Remove URLs already in dedup set. Take first `count` new ones.
+
+### 6. Process each article
+For each URL (in series, not parallel):
+1. Fetch full article (title, date, authors, body, images)
+2. Download images → `<vaultPath>/<vaultFolder>/YYYYMMDD/assets/<slug>-<n>.jpg`
+3. Generate Chinese summary doc → `<vaultPath>/<vaultFolder>/YYYYMMDD/<中文标题>.md`
+4. Generate original doc → `<vaultPath>/<vaultFolder>/YYYYMMDD/<英文标题> - <SourceName>原文.md`
+5. Update `YYYYMMDD/index.md`
+6. Publish to Discord (see Publishing section)
+7. Append to `YYYYMMDD/registry.json`
+
+### 7. Post run summary to weekly thread
+Find or create the weekly Discord thread (see Weekly Thread section), then post a summary reply.
+
+---
+
+## Document Formats
+
+### Chinese Summary Doc
+Filename: `<中文标题>.md` (concise, ≤30 chars)
+
+```markdown
+---
+title: <中文标题>
+date: YYYY-MM-DD
+tags:
+  - <tag1>
+  - <tag2>
+source: "<original URL>"
+publisher: <source name>
+authors: "<author names>"
+type: 摘要
+related: "[[<英文标题> - <SourceName>原文]]"
+saved: YYYY-MM-DD
+---
+
+# <中文标题>
+
+> [!info] 文章信息
+> **发布：** YYYY-MM-DD | **来源：** <source>
+> **作者：** <authors>
+> **原文：** [[<英文标题> - <SourceName>原文]]
+
+---
+
+## 事件概述
+<2-3 sentences summary in Chinese>
+
+## 背景
+<context>
+
+## 关键引语
+> "<translated quote>"
+
+## 影响分析
+<analysis>
+
+---
+[[<英文标题> - <SourceName>原文]]
+```
+
+### Original Doc
+Filename: `<英文标题> - <SourceName>原文.md`
+
+```markdown
+---
+title: "<英文标题>"
+date: YYYY-MM-DD
+tags: [...]
+source: "<url>"
+publisher: <source>
+authors: "<authors>"
+type: 原文
+related: "[[<中文标题>]]"
+saved: YYYY-MM-DD
+---
+
+# <英文标题>
+
+> [!info] Source
+> **Published:** YYYY-MM-DD | **URL:** <url>
+> **中文摘要：** [[<中文标题>]]
+
+---
+
+## 配图
+![[assets/<slug>-1.jpg]]
+*<alt text>*
+
+---
+
+## 正文
+<full article body>
+
+---
+[[<中文标题>]]
+```
+
+---
+
+## index.md Format
+
+File: `<vaultFolder>/YYYYMMDD/index.md`
+
+```markdown
+---
+date: YYYY-MM-DD
+type: daily-index
+source: <source name>
+total_articles: N
+last_updated: "<ISO timestamp>"
+---
+
+# <SourceName> Daily Index · YYYY-MM-DD
+
+> **最近更新：** YYYY-MM-DD HH:MM CST  
+> **当日归档：** N 篇 | **图片资产：** N 张
+
+## 📊 分类统计
+| 分类 | 数量 |
+|------|------|
+| ... | ... |
+
+## 📋 文章列表
+| # | 标题 | 分类 | 标签 | 发布时间 | 状态 |
+|---|------|------|------|----------|------|
+| 1 | [[中文标题]] | ... | ... | YYYY-MM-DD | ✅ |
+
+## 🗂️ 分类导航
+### 分类1（N篇）
+- [[文章1]]
+```
+
+---
+
+## registry.json Format
+
+File: `<vaultFolder>/YYYYMMDD/registry.json`
+
+```json
+{
+  "date": "YYYYMMDD",
+  "source": "<source_id>",
+  "articles": [
+    {
+      "url": "https://...",
+      "titleZh": "中文标题",
+      "titleEn": "English Title",
+      "processedAt": "2026-02-28T14:30:00+08:00",
+      "discordThreadId": "1477xxx",
+      "category": "地缘政治",
+      "tags": ["tag1", "tag2"]
+    }
+  ]
+}
+```
+
+---
+
+## Publishing to Discord
+
+### Per-article thread
+1. `thread-create` on `source.discordChannelId`:
+   - `threadName`: `<emoji> <中文标题>` — **纯文字，不含任何 Markdown 符号（无 `**`、`#` 等）**，Discord 标题不渲染 Markdown
+   - `message`: formatted Chinese summary with sections (Markdown OK here)
+   - End with: `原文见下方回复 👇`
+2. `send` reply to thread:
+   - Original title + `<url>` (wrapped in `<>`)
+   - Image URLs (plain, Discord auto-previews)
+   - Full English body (split if >1800 chars)
+
+### Weekly thread
+- Thread name format: `📰 <SourceName> 周归档 · YYYY-WNN（MM/DD–MM/DD）`
+- Store thread ID in config under `weeklyThreads.<source_id>.<YYYY-WNN>`
+- **ALWAYS check config for existing thread before creating a new one.** The manifest output from `harvest.js` includes `weeklyThreadId` — use it if non-null. If null, create the thread and immediately save the new ID:
+  ```bash
+  node scripts/save-weekly-thread.js <source_id> <week_key> <new_thread_id>
+  ```
+- Each run appends one reply:
+
+```
+── YYYY-MM-DD HH:MM · 新增 N 篇 · 跳过 M 篇 ──
+<emoji> <中文标题1>
+<emoji> <中文标题2>
+...
+```
+
+To find current week key: ISO week number, e.g. `2026-W09`.
+
+---
+
+## Source Management
+
+Read `references/source-management.md` for detailed instructions on add/list/edit operations.
+
+---
+
+## Category Detection
+
+Assign category based on URL path and title keywords:
+
+| Keywords | Category |
+|----------|---------|
+| asia-pacific, pakistan, afghanistan, 巴, 阿富汗 | 地缘政治 |
+| ai, anthropic, openai, tech, 科技, AI | 科技/AI |
+| china, 中国, 中美 | 中美关系 |
+| business, finance, housing, economy, 经济, 财经 | 财经 |
+| sustainability, feminist, society, 社会 | 社会 |
+| ukraine, russia, middle-east, war | 国际冲突 |
+| default | 国际 |
+
+Category emoji mapping: 地缘政治→🌏, 科技/AI→🤖, 中美关系→🇨🇳, 财经→💰, 社会→🧑‍🤝‍🧑, 国际冲突→⚔️, 国际→📰

package-lock.json

+{
+  "name": "news-harvester",
+  "version": "1.0.0",
+  "lockfileVersion": 3,
+  "requires": true,
+  "packages": {
+    "": {
+      "name": "news-harvester",
+      "version": "1.0.0",
+      "license": "ISC",
+      "dependencies": {
+        "ws": "^8.19.0"
+      }
+    },
+    "node_modules/ws": {
+      "version": "8.19.0",
+      "resolved": "https://registry.npmjs.org/ws/-/ws-8.19.0.tgz",
+      "integrity": "sha512-blAT2mjOEIi0ZzruJfIhb3nps74PRWTCz1IjglWEEpQl5XS/UNama6u2/rjFkDDouqr4L67ry+1aGIALViWjDg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=10.0.0"
+      },
+      "peerDependencies": {
+        "bufferutil": "^4.0.1",
+        "utf-8-validate": ">=5.0.2"
+      },
+      "peerDependenciesMeta": {
+        "bufferutil": {
+          "optional": true
+        },
+        "utf-8-validate": {
+          "optional": true
+        }
+      }
+    }
+  }
+}

package.json

+{
+  "name": "news-harvester",
+  "version": "1.0.0",
+  "description": "",
+  "main": "index.js",
+  "scripts": {
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "type": "commonjs",
+  "dependencies": {
+    "ws": "^8.19.0"
+  }
+}

references/config-template.json

+{
+  "vaultPath": "/path/to/your/obsidian/vault",
+  "registryWindowDays": 7,
+  "weeklyThreads": {},
+  "sources": [
+    {
+      "id": "the-conversation",
+      "name": "The Conversation",
+      "homepageUrl": "https://theconversation.com/us",
+      "method": "direct",
+      "articleUrlPattern": "theconversation\\.com/[\\w-]+-\\d+$",
+      "vaultFolder": "The Conversation",
+      "discordChannelId": "",
+      "enabled": true
+    }
+  ]
+}

references/source-management.md

+# Source Management
+
+## List Sources
+
+Read `config.json` and display:
+
+```
+ID             NAME              METHOD    CHANNEL              ENABLED
+reuters        路透社             browser   1234567890123456789  ✅
+theconversation The Conversation  direct    (not set)            ❌
+```
+
+## Add Source
+
+Prompt the user for (or infer from context):
+
+| Field | Description | Example |
+|-------|-------------|---------|
+| `id` | Unique slug (lowercase, no spaces) | `reuters` |
+| `name` | Display name | `路透社` |
+| `homepageUrl` | URL to scrape for links | `https://www.reuters.com/` |
+| `method` | `browser` (needs JS) or `direct` (plain HTTP) | `browser` |
+| `articleUrlPattern` | Regex to match article URLs | `reuters\\.com/.+/\\d{4}-\\d{2}-\\d{2}/` |
+| `vaultFolder` | Subfolder under vaultPath | `路透社` |
+| `discordChannelId` | Forum channel ID | `1234567890123456789` |
+| `enabled` | Start enabled? | `true` |
+
+Auto-detect `articleUrlPattern` if not provided:
+- Reuters: `reuters\\.com/.+/\\d{4}-\\d{2}-\\d{2}/`
+- The Conversation: `theconversation\\.com/.+-\\d+$`
+- Generic: derive from homepage domain + path depth
+
+After collecting all fields, append to `config.json` sources array and confirm:
+```
+✅ Added source "路透社" (id: reuters)
+   Method: browser | Channel: 1234567890123456789
+   Run: node harvest.js reuters 5
+```
+
+## Edit Source
+
+Usage: `node source-manage.js edit <id>`
+
+Show current values, prompt for field to change, update `config.json`.
+
+Editable fields: `name`, `homepageUrl`, `method`, `articleUrlPattern`, `vaultFolder`, `discordChannelId`, `enabled`, `countPerRun` (if set as default).
+
+## Enable / Disable Source
+
+Toggle `enabled` field:
+```bash
+node source-manage.js enable reuters
+node source-manage.js disable reuters
+```
+
+## Weekly Thread Management
+
+Weekly thread IDs are stored in config under `weeklyThreads`:
+```json
+{
+  "weeklyThreads": {
+    "reuters": {
+      "2026-W09": "1234567890123456789"
+    }
+  }
+}
+```
+
+To find current ISO week:
+```python
+from datetime import datetime, date
+d = date.today()
+week = d.isocalendar()[1]
+year = d.isocalendar()[0]
+key = f"{year}-W{week:02d}"
+```
+
+If no thread exists for current week, create one with `thread-create` and save the ID.
+Thread name format: `📰 <SourceName> 周归档 · YYYY-WNN（MM/DD–MM/DD）`
+Include week date range in the name (Monday–Sunday).

scripts/fetcher-browser.js

+/**
+ * fetcher-browser.js
+ * Fetch article content using Chromium CDP (for JS-heavy sites like Reuters)
+ * Usage: node fetcher-browser.js <url> [waitMs]
+ */
+
+const http = require('http');
+const WebSocket = require('ws');
+
+const CDP_PORT = parseInt(process.env.CDP_PORT_OVERRIDE || '9223');
+
+function getWsUrl() {
+  return new Promise((resolve, reject) => {
+    http.get(`http://localhost:${CDP_PORT}/json`, (res) => {
+      let data = '';
+      res.on('data', d => data += d);
+      res.on('end', () => {
+        try {
+          const tabs = JSON.parse(data);
+          // 优先找 page 类型的 tab，避免拿到 iframe/service_worker
+          const tab = tabs.find(t => t.type === 'page') || tabs[0];
+          resolve(tab.webSocketDebuggerUrl);
+        } catch (e) { reject(e); }
+      });
+    }).on('error', reject);
+  });
+}
+
+async function sleep(ms) { return new Promise(r => setTimeout(r, ms)); }
+
+async function fetchPage(url, waitMs = 12000) {
+  const wsUrl = await getWsUrl();
+  const ws = new WebSocket(wsUrl);
+  await new Promise((resolve, reject) => { ws.on('open', resolve); ws.on('error', reject); });
+
+  // Enable Page events
+  await new Promise(resolve => {
+    ws.send(JSON.stringify({ id: 1, method: 'Page.enable' }));
+    ws.once('message', () => resolve());
+  });
+
+  // Navigate + wait for load
+  let loaded = false;
+  const loadPromise = new Promise(resolve => {
+    const handler = (raw) => {
+      const msg = JSON.parse(raw.toString());
+      if (msg.method === 'Page.loadEventFired' && !loaded) {
+        loaded = true;
+        ws.removeListener('message', handler);
+        resolve();
+      }
+    };
+    ws.on('message', handler);
+  });
+  ws.send(JSON.stringify({ id: 2, method: 'Page.navigate', params: { url } }));
+  await Promise.race([loadPromise, sleep(25000)]);
+  await sleep(waitMs); // wait for React/JS to fully render
+
+  // Extract content
+  const result = await new Promise((resolve) => {
+    const id = Math.floor(Math.random() * 100000) + 90000;
+    const handler = (raw) => {
+      const msg = JSON.parse(raw.toString());
+      if (msg.id === id) {
+        ws.removeListener('message', handler);
+        resolve(msg.result && msg.result.result && msg.result.result.value);
+      }
+    };
+    ws.on('message', handler);
+    ws.send(JSON.stringify({ id, method: 'Runtime.evaluate', params: {
+      expression: `(function(){
+        var title = document.title.replace(/ \\| Reuters| \\| The Conversation| - Bloomberg$/g,'').trim();
+        var dateEl = document.querySelector('time[datetime]');
+        var date = dateEl ? dateEl.getAttribute('datetime') : '';
+        var authors = Array.from(document.querySelectorAll('[rel="author"], [class*="author"] a'))
+          .map(a => a.innerText.trim()).filter(Boolean)
+          .filter((v,i,a) => a.indexOf(v)===i).join(', ');
+        var imgs = Array.from(document.querySelectorAll('img')).map(img => ({
+          src: img.src,
+          alt: img.alt || '',
+          width: img.naturalWidth || img.width || 0
+        })).filter(i => i.src && i.src.startsWith('http') && i.width > 200
+          && !i.src.includes('logo') && !i.src.includes('icon') && !i.src.includes('avatar')
+          && !i.src.includes('profile')).slice(0, 6);
+        var body = '';
+        var selectors = [
+          '[data-testid^="paragraph"]',
+          '[class*="articleBodyContent"] p',
+          '[class*="article-body"] p',
+          '[class*="ArticleBody"] p',
+          '[class*="story-body"] p',
+          'article p'
+        ];
+        for (var sel of selectors) {
+          var els = document.querySelectorAll(sel);
+          if (els.length >= 3) {
+            body = Array.from(els).map(e => e.innerText.trim()).filter(t => t.length > 40).join('\\n\\n');
+            break;
+          }
+        }
+        if (!body) {
+          var noise = ['Reporting by','Editing by','Access unmatched','Browse an unrivalled',
+            'Screen for heightened','Sign up here','Reuters, the news','All quotes delayed','See here for'];
+          body = Array.from(document.querySelectorAll('p'))
+            .map(e => e.innerText.trim())
+            .filter(t => t.length > 60 && !noise.some(n => t.startsWith(n)))
+            .join('\\n\\n');
+        }
+        return JSON.stringify({ title, date, authors, imgs, body: body.substring(0, 15000) });
+      })()`
+    }}));
+  });
+
+  ws.close();
+  return result ? JSON.parse(result) : null;
+}
+
+// Extract article links from homepage
+async function fetchHomeLinks(url, pattern, waitMs = 15000) {
+  const wsUrl = await getWsUrl();
+  const ws = new WebSocket(wsUrl);
+  await new Promise((resolve, reject) => { ws.on('open', resolve); ws.on('error', reject); });
+
+  await new Promise(resolve => {
+    ws.send(JSON.stringify({ id: 1, method: 'Page.enable' }));
+    ws.once('message', () => resolve());
+  });
+
+  let loaded = false;
+  const loadPromise = new Promise(resolve => {
+    const handler = (raw) => {
+      const msg = JSON.parse(raw.toString());
+      if (msg.method === 'Page.loadEventFired' && !loaded) {
+        loaded = true;
+        ws.removeListener('message', handler);
+        resolve();
+      }
+    };
+    ws.on('message', handler);
+  });
+  ws.send(JSON.stringify({ id: 2, method: 'Page.navigate', params: { url } }));
+  await Promise.race([loadPromise, sleep(25000)]);
+  await sleep(waitMs);
+
+  const result = await new Promise((resolve) => {
+    const id = Math.floor(Math.random() * 100000) + 80000;
+    const patternStr = pattern.toString();
+    const handler = (raw) => {
+      const msg = JSON.parse(raw.toString());
+      if (msg.id === id) {
+        ws.removeListener('message', handler);
+        resolve(msg.result && msg.result.result && msg.result.result.value);
+      }
+    };
+    ws.on('message', handler);
+    ws.send(JSON.stringify({ id, method: 'Runtime.evaluate', params: {
+      expression: `JSON.stringify(
+        Array.from(document.querySelectorAll('a[href]'))
+          .map(a => a.href)
+          .filter(h => h && new RegExp(${JSON.stringify(pattern)}).test(h))
+          .filter((h, i, arr) => arr.indexOf(h) === i)
+          .slice(0, 30)
+      )`
+    }}));
+  });
+
+  ws.close();
+  return result ? JSON.parse(result) : [];
+}
+
+module.exports = { fetchPage, fetchHomeLinks };
+
+// CLI usage
+if (require.main === module) {
+  const url = process.argv[2];
+  const waitMs = parseInt(process.argv[3] || '12000');
+  if (!url) { console.error('Usage: node fetcher-browser.js <url> [waitMs]'); process.exit(1); }
+  fetchPage(url, waitMs).then(data => {
+    process.stdout.write(JSON.stringify(data, null, 2));
+  }).catch(e => { console.error('Error:', e.message); process.exit(1); });
+}

scripts/fetcher-direct.js

+/**
+ * fetcher-direct.js
+ * Fetch article content via direct HTTP (for open sites like The Conversation)
+ * Usage: node fetcher-direct.js <url>
+ */
+
+const https = require('https');
+const http = require('http');
+
+function fetchHtml(url) {
+  return new Promise((resolve, reject) => {
+    const lib = url.startsWith('https') ? https : http;
+    const req = lib.get(url, {
+      headers: {
+        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
+        'Accept': 'text/html,application/xhtml+xml',
+        'Accept-Language': 'en-US,en;q=0.9'
+      }
+    }, (res) => {
+      // Handle redirects
+      if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
+        return resolve(fetchHtml(res.headers.location));
+      }
+      let data = '';
+      res.on('data', d => data += d);
+      res.on('end', () => resolve(data));
+    });
+    req.on('error', reject);
+    req.setTimeout(15000, () => { req.destroy(); reject(new Error('Timeout')); });
+  });
+}
+
+function extractArticle(html, url) {
+  // Title
+  const titleMatch = html.match(/<title[^>]*>(.*?)<\/title>/is);
+  let title = titleMatch ? titleMatch[1].trim() : '';
+  title = title.replace(/ \| The Conversation| \| Reuters/g, '').trim();
+
+  // Date
+  const dateMatch = html.match(/["']datePublished["']\s*:\s*["']([\d\-T:Z+]+)["']/) ||
+                    html.match(/<time[^>]+datetime=["']([\d\-T:Z+]+)["']/i);
+  const date = dateMatch ? dateMatch[1] : '';
+
+  // Authors
+  const authorMatches = html.match(/["']name["']\s*:\s*["']([^"']{3,60})["']/g) || [];
+  const authors = [...new Set(authorMatches.map(m => m.match(/["']name["']\s*:\s*["']([^"']+)["']/)[1])
+    .filter(a => a.length < 60 && !a.includes('{')))].slice(0, 3).join(', ');
+
+  // Images
+  const imgMatches = [...html.matchAll(/<img[^>]+src=["'](https?:\/\/[^"']+)["'][^>]*>/gi)];
+  const imgs = imgMatches.map(m => {
+    const altMatch = m[0].match(/alt=["']([^"']*)["']/i);
+    return { src: m[1], alt: altMatch ? altMatch[1] : '', width: 500 };
+  }).filter(i => !i.src.includes('logo') && !i.src.includes('icon') && i.src.length > 50).slice(0, 6);
+
+  // Article body - try structured extraction
+  let body = '';
+
+  // Try JSON-LD articleBody
+  const jsonLdMatch = html.match(/"articleBody"\s*:\s*"((?:[^"\\]|\\.)*)"/);
+  if (jsonLdMatch) {
+    body = jsonLdMatch[1].replace(/\\n/g, '\n').replace(/\\"/g, '"').replace(/\\\//g, '/');
+  }
+
+  if (!body) {
+    // Extract all paragraphs, clean HTML
+    const paraMatches = [...html.matchAll(/<p[^>]*>(.*?)<\/p>/gis)];
+    const noise = ['Sign up', 'Newsletter', 'Subscribe', 'Follow us', 'Read more', 'Click here',
+      'opens new tab', 'Thomson Reuters', 'All quotes delayed', 'Access unmatched'];
+    body = paraMatches.map(m => {
+      let text = m[1].replace(/<[^>]+>/g, '').replace(/&nbsp;/g, ' ')
+        .replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>').trim();
+      return text;
+    }).filter(t => t.length > 60 && !noise.some(n => t.includes(n))).join('\n\n');
+  }
+
+  return { title, date, authors, imgs, body: body.substring(0, 15000) };
+}
+
+function extractLinks(html, baseUrl, pattern) {
+  // Special case: The Conversation uses Next.js __NEXT_DATA__ JSON
+  const nextDataMatch = html.match(/<script id="__NEXT_DATA__" type="application\/json">([\s\S]*?)<\/script>/);
+  if (nextDataMatch) {
+    try {
+      const nextData = JSON.parse(nextDataMatch[1]);
+      const blocks = nextData?.props?.pageProps?.blocks || [];
+      const urls = [];
+      for (const block of blocks) {
+        for (const article of (block.blocks || [])) {
+          const url = article.url || '';
+          if (url && url.includes('theconversation.com') && /-\d+$/.test(url)) {
+            if (!urls.includes(url)) urls.push(url);
+          }
+        }
+      }
+      if (urls.length > 0) return urls.slice(0, 30);
+    } catch (e) {}
+  }
+
+  // Generic: find links by regex pattern
+  const regex = new RegExp(`href=["']((?:https?://)?[^"']*${pattern}[^"']*)["']`, 'gi');
+  const matches = [...html.matchAll(regex)];
+  const links = matches.map(m => {
+    let href = m[1];
+    if (href.startsWith('/')) {
+      const base = new URL(baseUrl);
+      href = `${base.protocol}//${base.host}${href}`;
+    }
+    return href;
+  }).filter(h => h.startsWith('http'));
+  return [...new Set(links)].slice(0, 30);
+}
+
+async function fetchPage(url) {
+  const html = await fetchHtml(url);
+  return extractArticle(html, url);
+}
+
+async function fetchHomeLinks(url, pattern) {
+  const html = await fetchHtml(url);
+  return extractLinks(html, url, pattern);
+}
+
+module.exports = { fetchPage, fetchHomeLinks };
+
+// CLI
+if (require.main === module) {
+  const url = process.argv[2];
+  if (!url) { console.error('Usage: node fetcher-direct.js <url>'); process.exit(1); }
+  fetchPage(url).then(data => {
+    process.stdout.write(JSON.stringify(data, null, 2));
+  }).catch(e => { console.error('Error:', e.message); process.exit(1); });
+}

scripts/save-weekly-thread.js

+#!/usr/bin/env node
+/**
+ * save-weekly-thread.js
+ * Save a weekly Discord thread ID to config.json so subsequent runs can reuse it.
+ * Usage: node save-weekly-thread.js <source_id> <week_key> <thread_id>
+ * Example: node save-weekly-thread.js the-conversation 2026-W09 1234567890123456789
+ */
+
+const fs = require('fs');
+const path = require('path');
+
+const SKILL_DIR = path.dirname(__dirname);
+const CONFIG_PATH = path.join(SKILL_DIR, 'config.json');
+
+const [sourceId, weekKey, threadId] = process.argv.slice(2);
+
+if (!sourceId || !weekKey || !threadId) {
+  console.error('Usage: node save-weekly-thread.js <source_id> <week_key> <thread_id>');
+  console.error('Example: node save-weekly-thread.js reuters 2026-W09 1234567890123456789');
+  process.exit(1);
+}
+
+const config = JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
+
+if (!config.weeklyThreads) config.weeklyThreads = {};
+if (!config.weeklyThreads[sourceId]) config.weeklyThreads[sourceId] = {};
+
+config.weeklyThreads[sourceId][weekKey] = threadId;
+
+fs.writeFileSync(CONFIG_PATH, JSON.stringify(config, null, 2));
+console.log(`✅ Saved: ${sourceId} / ${weekKey} → ${threadId}`);

scripts/source-manage.js

+#!/usr/bin/env node
+/**
+ * source-manage.js - Manage news sources
+ * Usage:
+ *   node source-manage.js list
+ *   node source-manage.js add --id reuters --name "路透社" --url https://... --method browser --channel 123 --pattern "..."
+ *   node source-manage.js edit <id> --field value
+ *   node source-manage.js enable <id>
+ *   node source-manage.js disable <id>
+ */
+
+const fs = require('fs');
+const path = require('path');
+
+const SKILL_DIR = path.dirname(__dirname);
+const CONFIG_PATH = path.join(SKILL_DIR, 'config.json');
+const TEMPLATE_PATH = path.join(SKILL_DIR, 'references', 'config-template.json');
+
+function loadConfig() {
+  if (!fs.existsSync(CONFIG_PATH)) {
+    const template = JSON.parse(fs.readFileSync(TEMPLATE_PATH, 'utf8'));
+    fs.writeFileSync(CONFIG_PATH, JSON.stringify(template, null, 2));
+  }
+  return JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
+}
+
+function saveConfig(config) {
+  fs.writeFileSync(CONFIG_PATH, JSON.stringify(config, null, 2));
+}
+
+function parseArgs(argv) {
+  const args = {};
+  for (let i = 0; i < argv.length; i++) {
+    if (argv[i].startsWith('--')) {
+      args[argv[i].slice(2)] = argv[i+1] || true;
+      i++;
+    } else {
+      args['_' + (args._count || 0)] = argv[i];
+      args._count = (args._count || 0) + 1;
+    }
+  }
+  return args;
+}
+
+const command = process.argv[2];
+const args = parseArgs(process.argv.slice(3));
+
+const config = loadConfig();
+
+switch (command) {
+  case 'list': {
+    console.log('\n📋 Configured Sources\n');
+    console.log('ID'.padEnd(20) + 'NAME'.padEnd(20) + 'METHOD'.padEnd(10) + 'CHANNEL'.padEnd(22) + 'STATUS');
+    console.log('─'.repeat(80));
+    for (const s of config.sources) {
+      const status = s.enabled ? '✅ enabled' : '❌ disabled';
+      console.log(
+        s.id.padEnd(20) +
+        s.name.padEnd(20) +
+        s.method.padEnd(10) +
+        (s.discordChannelId || '(not set)').padEnd(22) +
+        status
+      );
+    }
+    console.log();
+    break;
+  }
+
+  case 'add': {
+    const id = args.id || args._0;
+    if (!id) { console.error('--id required'); process.exit(1); }
+    if (config.sources.find(s => s.id === id)) {
+      console.error(`Source "${id}" already exists. Use "edit" to modify.`);
+      process.exit(1);
+    }
+
+    // Auto-detect URL pattern
+    let pattern = args.pattern;
+    if (!pattern && args.url) {
+      const urlObj = new URL(args.url);
+      const domain = urlObj.hostname.replace('www.', '').replace('.', '\\.');
+      if (args.url.includes('reuters')) {
+        pattern = `reuters\\.com/[a-z-]+/[a-z-]+/[\\w-]+-\\d{4}-\\d{2}-\\d{2}/`;
+      } else if (args.url.includes('theconversation')) {
+        pattern = `theconversation\\.com/[\\w-]+-\\d+$`;
+      } else {
+        pattern = `${domain}/[\\w/-]+-\\d{4,}`;
+      }
+    }
+
+    const newSource = {
+      id,
+      name: args.name || id,
+      homepageUrl: args.url || '',
+      method: args.method || 'direct',
+      articleUrlPattern: pattern || '',
+      vaultFolder: args.folder || args.name || id,
+      discordChannelId: args.channel || '',
+      enabled: args.enabled !== 'false'
+    };
+
+    config.sources.push(newSource);
+    saveConfig(config);
+
+    console.log(`\n✅ Added source "${newSource.name}" (id: ${id})`);
+    console.log(`   Method: ${newSource.method} | Channel: ${newSource.discordChannelId || '(not set)'}`);
+    console.log(`   URL Pattern: ${newSource.articleUrlPattern}`);
+    console.log(`   Vault Folder: ${newSource.vaultFolder}`);
+    console.log(`\n   Run: node harvest.js ${id} 5\n`);
+    break;
+  }
+
+  case 'edit': {
+    const id = args._0;
+    const source = config.sources.find(s => s.id === id);
+    if (!source) { console.error(`Source "${id}" not found`); process.exit(1); }
+
+    const editableFields = ['name', 'homepageUrl', 'method', 'articleUrlPattern', 'vaultFolder', 'discordChannelId', 'enabled'];
+    let changed = false;
+
+    for (const field of editableFields) {
+      if (args[field] !== undefined) {
+        const oldVal = source[field];
+        source[field] = args[field] === 'true' ? true : args[field] === 'false' ? false : args[field];
+        console.log(`   ${field}: ${oldVal} → ${source[field]}`);
+        changed = true;
+      }
+    }
+
+    if (!changed) {
+      console.log(`\nCurrent config for "${id}":`);
+      console.log(JSON.stringify(source, null, 2));
+      console.log('\nUsage: node source-manage.js edit <id> --field value');
+    } else {
+      saveConfig(config);
+      console.log(`\n✅ Updated source "${id}"`);
+    }
+    break;
+  }
+
+  case 'enable':
+  case 'disable': {
+    const id = args._0;
+    const source = config.sources.find(s => s.id === id);
+    if (!source) { console.error(`Source "${id}" not found`); process.exit(1); }
+    source.enabled = command === 'enable';
+    saveConfig(config);
+    console.log(`✅ Source "${id}" ${command}d`);
+    break;
+  }
+
+  default:
+    console.log(`
+Usage:
+  node source-manage.js list
+  node source-manage.js add --id <id> --name <name> --url <homepage> --method browser|direct --channel <discord_id>
+  node source-manage.js edit <id> --name "New Name" --channel 123456
+  node source-manage.js enable <id>
+  node source-manage.js disable <id>
+    `);
+}