Commit edea36c0 authored by 谢宇轩's avatar 谢宇轩

feat: update reuters bundles

parent 5cdadaef
{
"format": "news-harvester-source",
"version": 1,
"exportedAt": "2026-07-30T04:12:38.724Z",
"exportedAt": "2026-08-05T06:40:26.452Z",
"source": {
"id": "reuters-business",
"name": "Reuters Business",
"name": "reuters business",
"homepageUrl": "https://www.reuters.com/business/",
"method": "browser",
"vaultFolder": "Reuters Business"
"vaultFolder": "路透社/商业"
},
"files": {
"helpers/reuters-business.md": "---\nsource: reuters-business\nlistSelector: 'div[data-testid=\"Title\"]'\nlinkSelector: 'a[href]'\nurlPattern: 'reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$'\nexcludeUrlPattern: 'reuters\\.com/(podcasts|newsletter)/'\nscrollSteps: 3\nscrollWaitMs: 2000\nwaitMs: 15000\nvalidateArticle: true\nbodyMinChars: 500\nbodySelectors:\n - '[data-testid^=\"paragraph\"]'\n - '[class*=\"articleBodyContent\"] p'\n - '[class*=\"article-body\"] p'\n - 'article p'\nbodyMinParagraphs: 3\ndateSelector: 'time[datetime]'\nauthorSelector: '[rel=\"author\"], [class*=\"author\"] a'\nimageSelector: 'img'\nimageMinWidth: 200\nexcludeImage:\n - logo\n - icon\n - avatar\n - profile\ntitleStripSuffix: ' \\| Reuters$'\nmaxBodyChars: 15000\n---\n# Reuters Business 抓取备忘\n\n## 列表页链接收集(`/business/`)\n\n- **listSelector: `div[data-testid=\"Title\"]`**。实测全页 20 篇文章链接全部在此容器内(单一容器覆盖全部,无分散问题)。该 data-testid 比 CSS Modules 类名稳定(hash 不变)。\n- **urlPattern: `reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$`**。Reuters 文章 URL 以 `-YYYY-MM-DD/` 结尾,日期后缀是极强的文章标识——分类页(`/business/`、`/business/finance/`)无日期后缀。放宽为全站匹配(不限 `/business/`),因为板块页会推荐其他分区的文章(如 `/world/asia-pacific/...`)。\n- **excludeUrlPattern** 排除 `/podcasts/`、`/newsletter/`(它们也带日期但非正文报道)。\n- `scrollSteps: 3` 触发懒加载拿全当天列表;首屏仅 ~10 篇。\n\n## 详情页正文验证\n\n`validateArticle: true` 让 fetchPage 用 `og:type` + JSON-LD `@type` + 正文长度剔除分类页。实测信号:\n- 文章:`og:type=article`、`@type=NewsArticle`、正文 4000+ 字符\n- 分类页 `/business/`:`og:type=website`、`@type=CollectionPage`、正文 0\n\nReuters 的日期后缀 URL 已能区分大部分文章/分类页,validateArticle 作为兜底防止非标准 URL 混入。\n\n## 其他\n\n- 反爬:headless 会被拦,需 `config.chromium.headless: false`。\n- 正文优先 `[data-testid^=\"paragraph\"]`,旧 `article p` 会被\"相关阅读\"\"订阅推广\"段落污染。\n"
......
{
"format": "news-harvester-source",
"version": 1,
"exportedAt": "2026-07-30T04:12:38.760Z",
"exportedAt": "2026-08-05T06:40:26.498Z",
"source": {
"id": "reuters-markets",
"name": "Reuters Markets",
"name": "reuters markets",
"homepageUrl": "https://www.reuters.com/markets/",
"method": "browser",
"vaultFolder": "Reuters Markets"
"vaultFolder": "路透社/市场"
},
"files": {
"helpers/reuters-markets.md": "---\nsource: reuters-markets\nlistSelector: '[data-testid=\"MediaStoryCard\"], [data-testid=\"BasicCard\"], [data-testid=\"HeroCard\"], [data-testid=\"HubCard\"], [data-testid=\"AuthorCard\"], [data-testid=\"OurColumnists\"]'\nlinkSelector: 'a[href]'\nurlPattern: 'reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$'\nexcludeUrlPattern: 'reuters\\.com/(podcasts|newsletter)/'\nscrollSteps: 3\nscrollWaitMs: 2000\nwaitMs: 15000\nvalidateArticle: true\nbodyMinChars: 500\nbodySelectors:\n - '[data-testid^=\"paragraph\"]'\n - '[class*=\"articleBodyContent\"] p'\n - '[class*=\"article-body\"] p'\n - 'article p'\nbodyMinParagraphs: 3\ndateSelector: 'time[datetime]'\nauthorSelector: '[rel=\"author\"], [class*=\"author\"] a'\nimageSelector: 'img'\nimageMinWidth: 200\nexcludeImage:\n - logo\n - icon\n - avatar\n - profile\ntitleStripSuffix: ' \\| Reuters$'\nmaxBodyChars: 15000\n---\n# Reuters Markets 抓取备忘\n\n## 列表页链接收集(`/markets/`)\n\n- **listSelector 用复合 `data-testid` 选择器**:`MediaStoryCard`、`BasicCard`、`HeroCard`、`HubCard`、`AuthorCard`、`OurColumnists` 六类卡片容器。实测全页 20 篇文章链接分散在这六类容器里(`MediaStoryCard` 18 + `BasicCard` 12 + `HeroCard`/`HubCard`/`AuthorCard`/`OurColumnists` 各 2-3,去重后 20)。\n- **为什么不用旧 `div.media-story-card-module__placement-container__1ZSB1`**:旧选择器依赖带 hash 的 CSS Modules 类名,hash 随构建变化 → 过期后只匹配 9 篇,漏掉 11 篇。`data-testid` 无 hash,稳定得多。Markets 页面布局与 Business/World 不同(`div[data-testid=\"Title\"]` 在此页 0 篇),必须用 Markets 专属的复合容器。\n- **urlPattern: `reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$`**。Reuters 文章 URL 以 `-YYYY-MM-DD/` 结尾,日期后缀是极强的文章标识——分类页(`/markets/`、`/markets/currencies/`)无日期后缀。放宽为全站匹配,因为 Markets 板块会推荐其他分区文章(如 `/world/asia-pacific/...`、`/business/energy/...`、`/commentary/...`)。\n- **excludeUrlPattern** 排除 `/podcasts/`、`/newsletter/`(它们也带日期但非正文报道)。\n- `scrollSteps: 3` 触发懒加载拿全当天列表。\n\n## 详情页正文验证\n\n`validateArticle: true` 让 fetchPage 用 `og:type` + JSON-LD `@type` + 正文长度剔除分类页。实测信号:\n- 文章:`og:type=article`、`@type=NewsArticle`、正文 4000+ 字符\n- 分类页 `/markets/`:`og:type=website`、正文 0\n\nReuters 的日期后缀 URL 已能区分大部分文章/分类页,validateArticle 作为兜底防止非标准 URL 混入。\n\n## 其他\n\n- 反爬:headless 会被拦,需 `config.chromium.headless: false`。\n- 正文优先 `[data-testid^=\"paragraph\"]`,旧 `article p` 会被\"相关阅读\"\"订阅推广\"段落污染。\n"
......
{
"format": "news-harvester-source",
"version": 1,
"exportedAt": "2026-07-30T04:12:38.797Z",
"exportedAt": "2026-08-05T06:40:26.534Z",
"source": {
"id": "reuters-world",
"name": "路透社 World",
"name": "reuters world",
"homepageUrl": "https://www.reuters.com/world/",
"method": "browser",
"vaultFolder": "路透社 World"
"vaultFolder": "路透社/世界"
},
"files": {
"helpers/reuters-world.md": "---\nsource: reuters-world\nlistSelector: 'div[data-testid=\"Title\"]'\nlinkSelector: 'a[href]'\nurlPattern: 'reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$'\nexcludeUrlPattern: 'reuters\\.com/(podcasts|newsletter)/'\nscrollSteps: 3\nscrollWaitMs: 2000\nwaitMs: 15000\nvalidateArticle: true\nbodyMinChars: 500\nbodySelectors:\n - '[data-testid^=\"paragraph\"]'\n - '[class*=\"articleBodyContent\"] p'\n - '[class*=\"article-body\"] p'\n - 'article p'\nbodyMinParagraphs: 3\ndateSelector: 'time[datetime]'\nauthorSelector: '[rel=\"author\"], [class*=\"author\"] a'\nimageSelector: 'img'\nimageMinWidth: 200\nexcludeImage:\n - logo\n - icon\n - avatar\n - profile\ntitleStripSuffix: ' \\| Reuters$'\nmaxBodyChars: 15000\n---\n# 路透社 World 抓取备忘\n\n## 列表页链接收集(`/world/`)\n\n- **listSelector: `div[data-testid=\"Title\"]`**。实测全页 19 篇文章链接全部在此容器内(单一容器覆盖全部,无分散问题)。该 data-testid 比 CSS Modules 类名稳定。\n- **urlPattern: `reuters\\.com/.+-\\d{4}-\\d{2}-\\d{2}/?$`**。Reuters 文章 URL 以 `-YYYY-MM-DD/` 结尾,日期后缀是极强的文章标识——分类页(`/world/`、`/world/asia-pacific/`)无日期后缀。放宽为全站匹配(不限 `/world/`),因为板块页会推荐其他分区的文章(如 `/business/environment/...`、`/legal/government/...`)。\n- **excludeUrlPattern** 排除 `/podcasts/`、`/newsletter/`(它们也带日期但非正文报道)。\n- `scrollSteps: 3` 触发懒加载拿全当天列表。\n\n## 详情页正文验证\n\n`validateArticle: true` 让 fetchPage 用 `og:type` + JSON-LD `@type` + 正文长度剔除分类页。实测信号:\n- 文章:`og:type=article`、`@type=NewsArticle`、正文 6000+ 字符\n- 分类页 `/world/`:`og:type=website`、正文 0\n\nReuters 的日期后缀 URL 已能区分大部分文章/分类页,validateArticle 作为兜底防止非标准 URL 混入。\n\n## 其他\n\n- 反爬:headless 会被拦,需 `config.chromium.headless: false`。\n- 正文优先 `[data-testid^=\"paragraph\"]`,旧 `article p` 会被\"相关阅读\"\"订阅推广\"段落污染。\n"
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment