Commit 89df597f authored by 谢宇轩's avatar 谢宇轩

feat: add wsj

parent 30f58c27
{
"format": "news-harvester-source",
"version": 1,
"exportedAt": "2026-09-08T06:30:08.451Z",
"source": {
"id": "wsj",
"name": "华尔街日报",
"homepageUrl": "https://www.wsj.com/news/latest-headlines?mod=wsjfooter",
"method": "browser",
"vaultFolder": "华尔街日报"
},
"files": {
"helpers/wsj.md": "---\nsource: wsj\nlistSelector: \"div.eqa17h20\"\nlinkSelector: 'a[href]'\nurlPattern: 'wsj\\.com/[a-z0-9-]+/[a-z0-9-]+/[a-z0-9-]+-[a-f0-9]{6,}'\nexcludeUrlPattern: 'news/author/'\nscrollSteps: 3\nscrollWaitMs: 2000\nwaitMs: 15000\n# ── 翻页(可选)── 站点把文章列表分多页时启用。两种模式互斥,最多配一个。\n# 配了翻页字段后还需在 config.json 该 source 下设 paginate: true 才生效。\n# nextSelector: 'a.next' # 点击\"下一页\"元素翻页\n# pageUrlTemplate: null # 或用 URL 模板 'https://site.com/news/page/{page}'\n# pageStart: 2 # pageUrlTemplate 起始页码(≥2)\n# maxPages: 5 # 最多翻页数(不含首页)\n# pageWaitMs: null # 翻页后等待(null 回退到 scrollWaitMs)\nbodySelectors:\n - '[data-testid^=\"paragraph\"]'\n - '[class*=\"articleBodyContent\"] p'\n - '[class*=\"article-body\"] p'\n - 'article p'\nbodyMinParagraphs: 3\ndateSelector: 'time[datetime]'\nauthorSelector: '[rel=\"author\"], [class*=\"author\"] a'\n# Scope to the article-body container (e.g. '[class*=\"ArticleBody\"] img, article img')\n# so bottom-of-page \"recommended articles\" thumbnails aren't downloaded into assets/.\nimageSelector: 'img'\nimageMinWidth: 200\nexcludeImage:\n - logo\n - icon\n - avatar\n - profile\n# excludeImageSelector drops imgs whose ancestor matches — use for\n# \"recommended/related articles\" thumbnails sharing the hero image's CDN host.\n# excludeImageSelector: '[class*=\"RelatedArticle\"], [class*=\"RecommendedArticle\"]'\ntitleStripSuffix: ''\nmaxBodyChars: 15000\n---\n# 华尔街日报 抓取备忘\n\n- inspect 自动选中的 `li.css-ahu2xl-LinkItem`(75条)实为页脚导航链接,手动改用真实文章流容器 `div.eqa17h20`(50条)。\n- urlPattern 收紧为 `分类/子分类/slug-十六进制ID` 形式,排除 `/news/author/` 作者页与栏目首页等非文章链接。\n- 首屏可能不全,scrollSteps=3 触发懒加载;反爬严重时调高 waitMs。\n- 验证: node scripts/preview.js --url https://www.wsj.com/news/latest-headlines?mod=wsjfooter --recipe helpers/wsj.md --count 3(链接为真实文章且 charCount>500 即通过)",
"helpers/wsj.fixtures.json": "{\n \"articles\": [\n \"https://www.wsj.com/pro/central-banking/ecb-set-to-raise-key-rate-again-signal-caution-on-further-moves-67dfa39e\",\n \"https://www.wsj.com/health/pharma/novartis-says-neuromuscular-drug-candidate-misses-goal-in-late-stage-trial-acc0a16e\",\n \"https://www.wsj.com/tech/ai/mistral-ai-exceeds-24-billion-valuation-after-samsung-led-investment-round-ea377b82\"\n ],\n \"sections\": [\n \"https://www.wsj.com/world?mod=wsjfooter\",\n \"https://www.wsj.com/business?mod=wsjfooter\",\n \"https://www.wsj.com/news/author/paul-hannon\"\n ],\n \"minLinks\": 20\n}\n"
}
}
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment