Merge branch 'feat/source-sharing-bundles' into 'main'

feat: shareable source bundles (export/install)

See merge request !1

SKILL.md

@@ -21,6 +21,8 @@ Fetch latest articles from configured news sources, archive them to Obsidian vau
 | Scaffold a source recipe | `node scripts/source-manage.js helper <id>` |
 | Inspect a source's page structure | `node scripts/inspect-source.js <url> --scroll 3 [--pattern '<regex>'] [--write <id>]` |
 | Run recipe regression tests | `node scripts/recipe-test.js <source_id>` |
+| Export a source as a shareable bundle | `node scripts/source-manage.js export <id> [--out <path.nhsource.json>]` |
+| Install a source from a bundle | `node scripts/source-manage.js install <pkg.nhsource.json> [--force]` |
 | Ensure Chromium (browser sources) | `node scripts/ensure-chromium.js` (or `--check`) |
 
 > **All commands assume `cd <skill-root>` first** (the directory containing this `SKILL.md`). Scripts live in the `scripts/` subfolder — **always prefix with `scripts/`**, e.g. `node scripts/harvest.js`, never `node harvest.js` (that fails with `MODULE_NOT_FOUND`).
@@ -186,6 +188,31 @@ Full schema and authoring notes: `references/source-management.md` (Helper recip
 
 ---
 
+## Sharing sources (export / install)
+
+A recipe (`helpers/<id>.md` + optional `helpers/<id>.fixtures.json`) captures everything site-specific about a source — so a finished, verified recipe is portable. `source-manage.js` can pack a source into a single self-contained **`.nhsource.json`** bundle that another user installs in one command, with no manual recipe authoring on their end.
+
+```bash
+# Pack: config entry + recipe + fixtures → one shareable file
+node scripts/source-manage.js export <id> [--out <path.nhsource.json>]
+# Default output: ./<id>.nhsource.json in the cwd.
+
+# Install (on another machine / in another vault):
+node scripts/source-manage.js install <pkg.nhsource.json> [--force]
+```
+
+**What's in a bundle**: the source's config metadata (`id`/`name`/`homepageUrl`/`method`/`vaultFolder`) plus its `helpers/<id>.md` and `helpers/<id>.fixtures.json` (if present). It deliberately **excludes** `vaultPath` (machine-specific, global), `enabled` (the installer's choice — always set `true`), and `articleUrlPattern` (legacy-direct-only; a recipe source doesn't need it).
+
+**Install behavior**: writes the recipe + fixtures into `helpers/` and registers the source in `config.json` (enabled). The installed source is **immediately usable** — run `node scripts/harvest.js <id> 3 --preview` right after. If `helpers/<id>.fixtures.json` came with the bundle, `node scripts/recipe-test.js <id>` verifies the recipe still matches the live site.
+
+**Conflicts**: `install` refuses if `<id>` already exists in `config.json` or a helper file of the same name is present; `--force` overwrites both (printing what changed). This matches the `helper` command's refuse-with-force convention.
+
+**vaultPath caveat**: install works even before `vaultPath` is configured (the source is registered, preview works), but archiving needs a real vault path — install prints a warning if `vaultPath` is still the placeholder.
+
+> Bundles are plain JSON — open them in any editor to inspect before installing, or share via any channel (email, chat, git). No signing/checksum (personal-sharing scope); if you need integrity, verify the file's `exportedAt` + `source` fields by eye.
+
+---
+
 ## Workflow
 
 Two flows depending on intent: **archive** (fetch + save to Obsidian) or **preview** (read-only, just return article data).

bundles/nikkei-tech.nhsource.json

+{
+  "format": "news-harvester-source",
+  "version": 1,
+  "exportedAt": "2026-07-29T07:04:52.699Z",
+  "source": {
+    "id": "nikkei-tech",
+    "name": "日经亚洲科技",
+    "homepageUrl": "https://asia.nikkei.com/business/tech",
+    "method": "browser",
+    "vaultFolder": "日经亚洲/科技"
+  },
+  "files": {
+    "helpers/nikkei-tech.md": "---\nsource: nikkei-tech\nlistSelector: '[class*=\"StreamArticleCard\"], [class*=\"SpotlightArticleCard\"], [class*=\"SecondaryArticleCard\"]'\nlinkSelector: 'a[href]'\nurlPattern: 'asia\\.nikkei\\.com/(?:business|spotlight|economy|politics|editor-s-picks|location|opinion|features|life-arts)/[^/?#]+/.+'\nexcludeUrlPattern: '/(?:podcasts|static|infographics|member)/'\nscrollSteps: 3\nscrollWaitMs: 2000\nwaitMs: 15000\n# 详情页正文验证(见 fetcher-helper.js buildExtractExpr):放宽 urlPattern 后,\n# 靠 og:type / JSON-LD @type / 正文字符数在 fetchPage 阶段剔除分类页,而不是\n# 靠 URL 末段连字符数猜测。实测文章 og:type=article + 正文>3000字符;分类页\n# og:type=website + 正文 0,判别力极强。\nvalidateArticle: true\nbodyMinChars: 500\nbodySelectors:\n  - '[class*=\"ArticleBody\"] p'\n  - '[class*=\"article-body\"] p'\n  - 'article p'\nbodyMinParagraphs: 3\ndateSelector: 'time[datetime]'\nauthorSelector: '[class*=\"author\"] a, [rel=\"author\"]'\nimageSelector: 'img'\nimageMinWidth: 200\nexcludeImage:\n  - logo\n  - icon\n  - avatar\n  - profile\n  - placeholder\n  - outbrainimg.com\ntitleStripSuffix: ' - Nikkei Asia$'\nmaxBodyChars: 15000\n---\n# 日经亚洲科技 抓取备忘\n\n## 列表页链接收集(`/business/tech`)\n\n- **listSelector 用复合卡片选择器**:`[class*=\"StreamArticleCard\"], [class*=\"SpotlightArticleCard\"], [class*=\"SecondaryArticleCard\"]`。实测文章卡片分散在这三类容器里(StreamArticleCard 27 篇 + SpotlightArticleCard 3 篇 + SecondaryArticleCard 3 篇 = 33 篇)。旧规则只锁定 `StreamArticleCard`,丢了 6 篇在 `SpotlightArticleCard`/`SecondaryArticleCard` 里的真实文章(含 SK Hynix、Philippine women、humanoid robots、Kumamoto quake、Panasonic、KOSPI 等)。\n- **为什么不直接 `listSelector: null`(全页)**:实测全页扫描会被导航栏的 `/location/<country>` 国家下拉链接(`/location/east-asia/china` 等 ~30 条)在 `maxLinks: 30` 上限内塞满,导致真实文章一条都收不到。卡片容器范围是必须的,只要把三类 ArticleCard 容器都纳入即可覆盖全部文章且避开 nav。\n- **urlPattern 扩展顶层段白名单**。旧 pattern 只覆盖 `/business/`、`/spotlight/`;实测 `/economy/`、`/politics/`、`/editor-s-picks/`、`/location/`、`/opinion/`、`/features/`、`/life-arts/` 下都出现文章。`/[^/?#]+/.+` 要求至少 3 段路径(顶层/子段/slug),同时覆盖 2 段深文章(如 `/economy/<slug>`)和 3 段深文章(如 `/business/tech/semiconductors/<slug>`)。\n- **excludeUrlPattern** 剔除 `/podcasts/`、`/static/`、`/infographics/`、`/member/` 等非文章路径。\n- **删除\"末段 ≥5 连字符\"硬规则**。旧 pattern 用 `(?:-[^/?#]*){5,}$` 猜测\"长 slug=文章\",但分类页 slug 也可能很长(如 `artificial-intelligence`、`wealth-management`),短标题文章又会被误伤。改由详情页验证区分。卡片里仍会混入 ~18 条子分类页链接(如 `/business/tech/semiconductors`、`/spotlight/trump-administration`,作为卡片的 tag 链接出现),由 `validateArticle` 在 fetchPage 阶段剔除。\n- `scrollSteps: 3` 触发懒加载;Nikkei 用 `?page=N` 分页(非无限滚动),query strip 自动排除分页链接。\n\n## 详情页正文验证(取代 URL 猜测)\n\n`validateArticle: true` 让 `fetchPage` 在提取正文时额外采集判别信号,返回 `isArticle` 布尔;harvest 对 `isArticle === false` 的候选直接 skip。判别逻辑:\n\n```\nisArticle = (og:type === 'article' || JSON-LD @type ∈ {NewsArticle, Article})\n            && bodyChars >= 500\n            && bodyParagraphs >= 3\n```\n\n实测信号(2026-07-29):\n| 页面类型 | og:type | JSON-LD @type | 正文段落 | 正文字符 |\n|----------|---------|---------------|----------|----------|\n| 文章(SK Hynix Q2) | article | NewsArticle | 11 | 3671 |\n| 文章(Philippine women) | article | NewsArticle | 30 | 5647 |\n| 分类页(/business/tech/semiconductors) | website | Thing | 0 | 0 |\n\n正文验证发生在候选链接被收集**之后**(fetchPage 阶段),所以放宽 urlPattern 混入的少量分类页会在这一步被剔除,不会进入 registry 或落盘。\n\n## 其他\n\n- 正文优先 `[class*=\"ArticleBody\"]`(Nikkei 的 CSS Modules 正文容器)。\n- 部分文章有付费墙,正文可能不完整——bodyMinChars=500 + bodyMinParagraphs=3 的下限会过滤掉被付费墙截断到极短的页面;preview 验证后若 charCount 过低需排除。\n- 回归测试:`node scripts/recipe-test.js nikkei-tech`(固定文章 URL 必须 isArticle,固定分类页 URL 必须非 isArticle,列表页链接数 ≥10 且不含 exclude 路径)。\n",
+    "helpers/nikkei-tech.fixtures.json": "{\n  \"_comment\": \"Regression fixtures for the nikkei-tech recipe. Article URLs must fetch with isArticle=true (og:type=article + body ≥500 chars); section URLs must fetch with isArticle=false (og:type=website). URLs verified 2026-07-29 against https://asia.nikkei.com/business/tech. Update when Nikkei archives these — pick current articles from the same path depths (2-seg /economy/<slug>, 3-seg /business/tech/semiconductors/<slug>).\",\n  \"articles\": [\n    \"https://asia.nikkei.com/business/tech/semiconductors/sk-hynix-q2-profit-surges-but-misses-market-forecast-shares-slide\",\n    \"https://asia.nikkei.com/business/tech/semiconductors/ai-chip-boom-shifts-bottleneck-to-advanced-packaging-says-at-s-ceo\",\n    \"https://asia.nikkei.com/spotlight/society/for-philippine-women-in-taiwan-s-tech-factories-motherhood-is-impossible\",\n    \"https://asia.nikkei.com/spotlight/trump-administration/us-bans-new-chinese-humanoid-robots-to-protect-ai-buildout\",\n    \"https://asia.nikkei.com/economy/natural-disasters/major-japan-quake-traps-people-inside-kumamoto-shopping-mall-factory\",\n    \"https://asia.nikkei.com/business/markets/south-korea-s-kospi-plunges-11-on-china-chip-competition-fears\",\n    \"https://asia.nikkei.com/business/companies/panasonic-to-end-tv-production-in-malaysia-cutting-jobs\"\n  ],\n  \"sections\": [\n    \"https://asia.nikkei.com/business/tech\",\n    \"https://asia.nikkei.com/business/tech/semiconductors\",\n    \"https://asia.nikkei.com/business/markets\",\n    \"https://asia.nikkei.com/economy/natural-disasters\",\n    \"https://asia.nikkei.com/spotlight/society\"\n  ],\n  \"minLinks\": 10\n}\n"
+  }
+}
\ No newline at end of file

references/source-management.md

@@ -143,3 +143,52 @@ node scripts/source-manage.js helper <id> --show     # print path + exists/missi
 ```
 Recipes live in `helpers/` (committed to git, shared across machines) — unlike
 `config.json` which is machine-specific.
+
+---
+
+## Sharing sources (export / install)
+
+A finished recipe is portable: `source-manage.js` can pack a source (config
+entry + recipe + fixtures) into a single `.nhsource.json` bundle that another
+user installs in one command.
+
+```bash
+node scripts/source-manage.js export <id> [--out <path.nhsource.json>]
+node scripts/source-manage.js install <pkg.nhsource.json> [--force]
+```
+
+### Bundle format
+
+A `.nhsource.json` is plain JSON:
+
+| Field | Contents |
+|-------|----------|
+| `format` | `"news-harvester-source"` (install validates this) |
+| `version` | `1` |
+| `exportedAt` | ISO timestamp |
+| `source` | `{ id, name, homepageUrl, method, vaultFolder }` — the config entry, **minus** `vaultPath` (machine-global), `enabled` (installer's choice, always `true`), and `articleUrlPattern` (legacy-direct-only; a recipe source doesn't carry it) |
+| `files` | Map of relative path → file content. Includes `helpers/<id>.md` (the recipe) and `helpers/<id>.fixtures.json` (if present). |
+
+`export` warns (but still succeeds) if the source has no `helpers/<id>.md` —
+that exports a legacy source whose only portable part is the config entry.
+
+### Install behavior
+
+- Writes every file in `files` into `helpers/` (path traversal guarded — only
+  basenames under `helpers/` are written).
+- Registers the source in `config.json` as enabled. If `<id>` already exists,
+  the entry is overwritten in place; otherwise it's appended.
+- **Conflict handling** (matches the `helper` command's convention): refuses if
+  `<id>` is already in `config.json` *or* a helper file of the same name exists,
+  unless `--force` is passed. `--force` overwrites the config entry and all
+  helper files.
+- **vaultPath caveat**: install succeeds even when `config.vaultPath` is still
+  the placeholder (the source is registered, `--preview` works), but archiving
+  requires a real vault path — install prints a warning in that case.
+- After install, the source is immediately usable. The installer prints the
+  next steps: `recipe-test.js <id>` (if fixtures came with the bundle) and
+  `harvest.js <id> 3 --preview`.
+
+Bundles carry no signature/checksum (personal-sharing scope). To vet a bundle
+before installing, open it in an editor and check the `source` block + the
+recipe frontmatter in `files["helpers/<id>.md"]`.

scripts/source-manage.js

@@ -9,6 +9,8 @@
  *   node source-manage.js disable <id>
  *   node source-manage.js helper <id>          # scaffold/overwrite helpers/<id>.md
  *   node source-manage.js helper <id> --show    # print current helper.md path + status
+ *   node source-manage.js export <id> [--out <path>]   # pack source + recipe into a shareable .nhsource.json
+ *   node source-manage.js install <pkg.nhsource.json> [--force]   # install a shared bundle
  */
 
 const fs = require('fs');
@@ -224,6 +226,136 @@ maxBodyChars: 15000
     break;
   }
 
+  case 'export': {
+    // Pack a source + its recipe/fixtures into a single shareable .nhsource.json.
+    const id = args._0;
+    const source = config.sources.find(s => s.id === id);
+    if (!source) { console.error(`Source "${id}" not found. Run "list" to see configured sources.`); process.exit(1); }
+
+    const files = {};
+    const recipePath = path.join(HELPERS_DIR, `${id}.md`);
+    const fixturesPath = path.join(HELPERS_DIR, `${id}.fixtures.json`);
+    const included = [];
+    if (fs.existsSync(recipePath)) {
+      files[`helpers/${id}.md`] = fs.readFileSync(recipePath, 'utf8');
+      included.push(`helpers/${id}.md`);
+    } else {
+      console.error(`   ⚠️  No helpers/${id}.md found — exporting a legacy source (no recipe).`);
+    }
+    if (fs.existsSync(fixturesPath)) {
+      files[`helpers/${id}.fixtures.json`] = fs.readFileSync(fixturesPath, 'utf8');
+      included.push(`helpers/${id}.fixtures.json`);
+    }
+
+    const bundle = {
+      format: 'news-harvester-source',
+      version: 1,
+      exportedAt: new Date().toISOString(),
+      source: {
+        id: source.id,
+        name: source.name,
+        homepageUrl: source.homepageUrl,
+        method: source.method,
+        vaultFolder: source.vaultFolder
+      },
+      files
+    };
+    const outPath = path.resolve(args.out || path.join(process.cwd(), `${id}.nhsource.json`));
+    fs.writeFileSync(outPath, JSON.stringify(bundle, null, 2));
+
+    console.log(`\n✅ Exported source "${source.name}" (id: ${id})`);
+    console.log(`   Bundle: ${outPath}`);
+    if (included.length) {
+      console.log(`   Files:  ${included.join(', ')}`);
+    } else {
+      console.log(`   Files:  (none — legacy source, recipe not present)`);
+    }
+    console.log(`\n   Share the .nhsource.json file. Recipients install with:`);
+    console.log(`   node scripts/source-manage.js install ${path.basename(outPath)}\n`);
+    break;
+  }
+
+  case 'install': {
+    // Unpack a .nhsource.json: write recipe/fixtures into helpers/, add the
+    // source to config.json. Refuses on id conflict unless --force (matching
+    // the "helper" command's refuse-with-force convention).
+    const pkgPath = args._0;
+    if (!pkgPath) { console.error('Usage: node source-manage.js install <pkg.nhsource.json> [--force]'); process.exit(1); }
+    const absPkg = path.resolve(pkgPath);
+    if (!fs.existsSync(absPkg)) { console.error(`Bundle not found: ${absPkg}`); process.exit(1); }
+
+    let bundle;
+    try { bundle = JSON.parse(fs.readFileSync(absPkg, 'utf8')); }
+    catch (e) { console.error(`Failed to parse bundle: ${e.message}`); process.exit(1); }
+    if (bundle.format !== 'news-harvester-source') {
+      console.error(`Not a news-harvester source bundle (format="${bundle.format}").`);
+      process.exit(1);
+    }
+    const src = bundle.source || {};
+    const id = src.id;
+    if (!id) { console.error('Bundle is missing source.id'); process.exit(1); }
+
+    const existing = config.sources.find(s => s.id === id);
+    if (existing && args.force === undefined) {
+      console.error(`Source "${id}" already exists in config.json. Use --force to overwrite (config entry + helper files).`);
+      process.exit(1);
+    }
+
+    // Write helper files (refuse to overwrite an existing file unless --force,
+    // same convention as the id-conflict check above).
+    if (!fs.existsSync(HELPERS_DIR)) fs.mkdirSync(HELPERS_DIR, { recursive: true });
+    const written = [];
+    const files = bundle.files || {};
+    for (const [relPath, content] of Object.entries(files)) {
+      // Guard against path traversal: only allow writing into helpers/.
+      const dest = path.join(HELPERS_DIR, path.basename(relPath));
+      if (dest.indexOf(HELPERS_DIR) !== 0) {
+        console.error(`   ⚠️  Skipping unsafe path in bundle: ${relPath}`);
+        continue;
+      }
+      if (fs.existsSync(dest) && args.force === undefined && !existing) {
+        console.error(`helpers/${path.basename(dest)} already exists. Use --force to overwrite.`);
+        process.exit(1);
+      }
+      fs.writeFileSync(dest, content);
+      written.push(`helpers/${path.basename(dest)}`);
+    }
+
+    // Upsert the config entry. recipe sources don't carry articleUrlPattern
+    // (it's legacy-direct-only and ignored when a recipe exists).
+    const entry = {
+      id,
+      name: src.name || id,
+      homepageUrl: src.homepageUrl || '',
+      method: src.method || 'browser',
+      vaultFolder: src.vaultFolder || src.name || id,
+      enabled: true
+    };
+    if (existing) {
+      Object.assign(existing, entry);
+    } else {
+      config.sources.push(entry);
+    }
+    saveConfig(config);
+
+    console.log(`\n✅ Installed source "${entry.name}" (id: ${id})`);
+    console.log(`   Method: ${entry.method} | Vault folder: ${entry.vaultFolder}`);
+    if (written.length) console.log(`   Files:  ${written.join(', ')}`);
+
+    // Warn if vaultPath is still the placeholder — preview works, archive won't.
+    if (config.vaultPath === '/path/to/your/obsidian/vault' || !config.vaultPath) {
+      console.error(`\n   ⚠️  config.vaultPath is not set. Preview works, but archiving needs a real vault path —`);
+      console.error(`       set it in config.json before running a full harvest.`);
+    }
+
+    console.log(`\n   Next:`);
+    if (written.includes(`helpers/${id}.fixtures.json`)) {
+      console.log(`   node scripts/recipe-test.js ${id}            # verify the recipe still matches the live site`);
+    }
+    console.log(`   node scripts/harvest.js ${id} 3 --preview      # fetch latest articles (read-only)\n`);
+    break;
+  }
+
   default:
     console.log(`
 Usage:
@@ -233,9 +365,17 @@ Usage:
   node source-manage.js enable <id>
   node source-manage.js disable <id>
   node source-manage.js helper <id> [--force|--show]   # scaffold a helpers/<id>.md recipe
+  node source-manage.js export <id> [--out <path.nhsource.json>]   # pack a source + recipe into a shareable bundle
+  node source-manage.js install <pkg.nhsource.json> [--force]   # install a shared bundle (recipe + config entry)
 
 Helper recipes: a source opts into the spec-driven fetcher by having a
 helpers/<id>.md file. Scaffold one with "helper <id>", inspect the page with
 scripts/inspect-source.js, then fill in listSelector + urlPattern.
+
+Sharing sources: "export <id>" packs the source's config entry + its recipe
+(and fixtures, if any) into a single .nhsource.json. Share that file; the
+recipient runs "install <pkg>" to write the recipe into helpers/ and register
+the source in config.json — no manual recipe authoring needed. --force
+overwrites an existing source of the same id.
     `);
 }