Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Sign in
Toggle navigation
M
macos-hub-skills
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
郭阔
macos-hub-skills
Commits
89df597f
Commit
89df597f
authored
Sep 08, 2026
by
谢宇轩
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
feat: add wsj
parent
30f58c27
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
16 additions
and
0 deletions
+16
-0
wsj.nhsource.json
skills/news-harvester/bundles/wsj.nhsource.json
+16
-0
No files found.
skills/news-harvester/bundles/wsj.nhsource.json
0 → 100644
View file @
89df597f
{
"format"
:
"news-harvester-source"
,
"version"
:
1
,
"exportedAt"
:
"2026-09-08T06:30:08.451Z"
,
"source"
:
{
"id"
:
"wsj"
,
"name"
:
"华尔街日报"
,
"homepageUrl"
:
"https://www.wsj.com/news/latest-headlines?mod=wsjfooter"
,
"method"
:
"browser"
,
"vaultFolder"
:
"华尔街日报"
},
"files"
:
{
"helpers/wsj.md"
:
"---
\n
source: wsj
\n
listSelector:
\"
div.eqa17h20
\"\n
linkSelector: 'a[href]'
\n
urlPattern: 'wsj
\\
.com/[a-z0-9-]+/[a-z0-9-]+/[a-z0-9-]+-[a-f0-9]{6,}'
\n
excludeUrlPattern: 'news/author/'
\n
scrollSteps: 3
\n
scrollWaitMs: 2000
\n
waitMs: 15000
\n
# ── 翻页(可选)── 站点把文章列表分多页时启用。两种模式互斥,最多配一个。
\n
# 配了翻页字段后还需在 config.json 该 source 下设 paginate: true 才生效。
\n
# nextSelector: 'a.next' # 点击
\"
下一页
\"
元素翻页
\n
# pageUrlTemplate: null # 或用 URL 模板 'https://site.com/news/page/{page}'
\n
# pageStart: 2 # pageUrlTemplate 起始页码(≥2)
\n
# maxPages: 5 # 最多翻页数(不含首页)
\n
# pageWaitMs: null # 翻页后等待(null 回退到 scrollWaitMs)
\n
bodySelectors:
\n
- '[data-testid^=
\"
paragraph
\"
]'
\n
- '[class*=
\"
articleBodyContent
\"
] p'
\n
- '[class*=
\"
article-body
\"
] p'
\n
- 'article p'
\n
bodyMinParagraphs: 3
\n
dateSelector: 'time[datetime]'
\n
authorSelector: '[rel=
\"
author
\"
], [class*=
\"
author
\"
] a'
\n
# Scope to the article-body container (e.g. '[class*=
\"
ArticleBody
\"
] img, article img')
\n
# so bottom-of-page
\"
recommended articles
\"
thumbnails aren't downloaded into assets/.
\n
imageSelector: 'img'
\n
imageMinWidth: 200
\n
excludeImage:
\n
- logo
\n
- icon
\n
- avatar
\n
- profile
\n
# excludeImageSelector drops imgs whose ancestor matches — use for
\n
#
\"
recommended/related articles
\"
thumbnails sharing the hero image's CDN host.
\n
# excludeImageSelector: '[class*=
\"
RelatedArticle
\"
], [class*=
\"
RecommendedArticle
\"
]'
\n
titleStripSuffix: ''
\n
maxBodyChars: 15000
\n
---
\n
# 华尔街日报 抓取备忘
\n\n
- inspect 自动选中的 `li.css-ahu2xl-LinkItem`(75条)实为页脚导航链接,手动改用真实文章流容器 `div.eqa17h20`(50条)。
\n
- urlPattern 收紧为 `分类/子分类/slug-十六进制ID` 形式,排除 `/news/author/` 作者页与栏目首页等非文章链接。
\n
- 首屏可能不全,scrollSteps=3 触发懒加载;反爬严重时调高 waitMs。
\n
- 验证: node scripts/preview.js --url https://www.wsj.com/news/latest-headlines?mod=wsjfooter --recipe helpers/wsj.md --count 3(链接为真实文章且 charCount>500 即通过)"
,
"helpers/wsj.fixtures.json"
:
"{
\n
\"
articles
\"
: [
\n
\"
https://www.wsj.com/pro/central-banking/ecb-set-to-raise-key-rate-again-signal-caution-on-further-moves-67dfa39e
\"
,
\n
\"
https://www.wsj.com/health/pharma/novartis-says-neuromuscular-drug-candidate-misses-goal-in-late-stage-trial-acc0a16e
\"
,
\n
\"
https://www.wsj.com/tech/ai/mistral-ai-exceeds-24-billion-valuation-after-samsung-led-investment-round-ea377b82
\"\n
],
\n
\"
sections
\"
: [
\n
\"
https://www.wsj.com/world?mod=wsjfooter
\"
,
\n
\"
https://www.wsj.com/business?mod=wsjfooter
\"
,
\n
\"
https://www.wsj.com/news/author/paul-hannon
\"\n
],
\n
\"
minLinks
\"
: 20
\n
}
\n
"
}
}
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment