Commit 4685f05e authored by 谢宇轩's avatar 谢宇轩

feat: add news hub feed skills

parent 6a313771
# News Hub Skills
Two independent skills for the news harvesting pipeline:
Three independent skills for the news harvesting + consumption pipeline:
## news-source-analyzer (无状态分析器)
......@@ -10,6 +10,10 @@ Analyze news websites, author fetch recipes, and produce installable source bund
Harvest and archive news articles from configured sources to Obsidian vault with Chinese summaries + original text. Sources are installed from bundles produced by the analyzer. Manages vault path, push config, and source registry.
## news-hub-feed (阅读 / 标注)
Search and browse News Hub articles, read full article details (summary / original / translation), and manage key-value marks for annotation. A pure API consumer — no chromium, no vault, no bundle, zero npm dependencies. Used for rapid hot-topic discovery and post-hoc annotation.
## Data flow
```
......@@ -21,13 +25,17 @@ preview → verify article extraction ↓
↓ harvest → summaries → finalize → push
recipe-test → regression lock ↓
↓ Obsidian vault + News Hub
pack → bundle.nhsource.json
└───── manual/scripted transfer ─────→ harvester
pack → bundle.nhsource.json ↑
│ │ read + annotate
└───── manual/scripted transfer ─────→ harvester │
Feed (any machine) ───┘
─────────────────
search → article detail → marks
```
The `.nhsource.json` bundle is the only interface between the two skills. No shared runtime state, no shared config, no shared browser profile.
The `.nhsource.json` bundle is the only interface between the analyzer and harvester. The feed skill connects directly to the News Hub API — no shared runtime state, config, or browser profile with the other two.
## Shared scripts
Four CDP/Chromium infrastructure scripts (`ensure-chromium.js`, `cdp-client.js`, `fetcher-helper.js`, `block-check.js`) are duplicated in both skills as standalone copies. These are stable low-level CDP wrappers with low change frequency. When modifying one, sync the copy in the other skill.
Four CDP/Chromium infrastructure scripts (`ensure-chromium.js`, `cdp-client.js`, `fetcher-helper.js`, `block-check.js`) are duplicated in the analyzer and harvester skills as standalone copies. These are stable low-level CDP wrappers with low change frequency. When modifying one, sync the copy in the other skill. The feed skill does not use Chromium.
......@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) {
'--no-default-browser-check',
'--disable-extensions',
'--disable-gpu',
// Hide the navigator.webdriver=true flag that CDP sets by default.
// Anti-bot systems (PerimeterX, Cloudflare) check this on page load.
// Combined with the stealth injection in cdp-client.js openSession().
'--disable-blink-features=AutomationControlled',
// NOTE: --disable-blink-features=AutomationControlled was removed.
// It triggers an "unsupported command-line flag" warning banner on some
// Chromium builds (e.g. Arch Linux), which is itself a detectable artifact.
// navigator.webdriver suppression is handled entirely by the CDP injection
// in cdp-client.js openSession() via Page.addScriptToEvaluateOnNewDocument,
// which sets navigator.webdriver to undefined before any page script runs.
// That approach is flag-independent, more robust, and produces no banner.
// Let Chromium auto-detect Wayland vs X11 on Linux. Without this it
// defaults to the X11 backend and crashes with "Missing X server or
// $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless
......
---
name: news-hub-feed
description: Search, browse, and annotate News Hub articles via the News Hub API. Use this skill whenever the user wants to discover trending news, search articles by keyword/source/category/tags/marks, read an article's full detail (summary, original, translation), or add/remove key-value marks for later retrieval and analysis. A lightweight read + annotate client — no harvesting, no pushing, no browser. Triggers on phrases like "检索新闻", "搜索文章", "查看文章", "文章详情", "添加标记", "标注", "热点", "热门", "hot news", "trending", "search articles", "article detail", "marks", "discover trending", "标签列表", "按标记检索".
---
# News Hub Feed Skill
Search and browse articles stored in News Hub, read full article details (summary / original / translation), and manage key-value **marks** for annotation. This skill is a **pure API consumer** — it never harvests, pushes, or launches a browser. Its sole purpose is rapid hot-topic discovery and post-hoc annotation for later retrieval and analysis.
> **Skill root** = the directory containing this `SKILL.md`. Prefix every command with `cd "<skill-root>" &&`, or scripts fail with `MODULE_NOT_FOUND`.
## Quick Reference
| Task | Command |
|------|---------|
| Check connectivity + scopes | `node scripts/check.js` |
| Search articles (hot topics) | `node scripts/search.js --q <keyword> [--days N] [--source csv] [--category csv] [--tags csv] [--marks csv]` |
| Search with facets | `node scripts/search.js --q 利率 --days 7 --sort relevance` |
| Article detail | `node scripts/article.js <id> [--fields summary,original,translation]` |
| Add a mark | `node scripts/marks.js add <id> <key> [value]` |
| Add marks (batch) | `node scripts/marks.js add <id> --json '[{"key":"已读","value":"是"},{"key":"重要","value":"高"}]'` |
| Remove a mark | `node scripts/marks.js remove <id> <key>` |
| List article marks | `node scripts/marks.js list <id>` |
| List mark keys | `node scripts/marks.js keys` |
| List mark values | `node scripts/marks.js values <key>` |
| Search by marks | `node scripts/marks.js marked --marks 已读:是,重要:高` |
| List all tags (read-only) | `node scripts/tags.js` |
> **All commands must be prefixed with `cd "<skill-root>" &&`**. Scripts live in `scripts/` — always reference them as `node scripts/search.js`, never `node search.js`.
---
## First-Run Setup
**Pre-flight gate** — run this at the start of a task. If it prints `OK`, skip setup:
```bash
cd "<skill-root>" && node -e "const c=require('./config.json'); console.log(c.api&&c.api.enabled?'OK':'UNCONFIGURED')" 2>/dev/null || echo "UNCONFIGURED (no config.json)"
```
### Onboarding (only if UNCONFIGURED)
This skill only needs the `api` block in `config.json` (no chromium, no vault, no sources). The config is auto-created from `references/config-template.json` on first script run. To configure:
1. Edit `config.json` — fill in the `api` block:
```json
{
"api": {
"enabled": true,
"endpoint": "http://localhost:8080/api/v1",
"appId": "app_feed",
"appKeyId": "kh_feed_default",
"appSecret": "<your-secret>",
"defaultPageSize": 20,
"retry": { "maxAttempts": 5, "baseDelayMs": 1000 }
}
}
```
The App must hold these scopes: `articles:read`, `marks:read`, `marks:write`.
2. Verify connectivity:
```bash
node scripts/check.js
```
If you see `✓ 连接成功` and `✓ 必需 scope 齐全`, setup is complete.
---
## Configuration
| Field | Required? | Description |
|-------|-----------|-------------|
| `api.enabled` | | Must be `true` to make any API call |
| `api.endpoint` | | News Hub base URL, e.g. `http://localhost:8080/api/v1` |
| `api.appId` | | App public identifier (e.g. `app_feed`) |
| `api.appKeyId` | | Credential Key ID (e.g. `kh_feed_default`) |
| `api.appSecret` | | Secret plaintext (server verifies via bcrypt) |
| `api.defaultPageSize` | Optional | Default page size for search/marked (default `20`, max `100`) |
| `api.retry.maxAttempts` | Optional | Max retry attempts on 5xx/429/network errors (default `5`) |
| `api.retry.baseDelayMs` | Optional | Base delay for exponential backoff (default `1000`ms) |
> Auth uses three custom headers (`X-App-Id`, `X-App-Key-Id`, `X-App-Secret`) identical to the harvester's push config. You can reuse the same App credentials if it holds the required scopes, or configure a dedicated read/annotate App.
---
## Workflow
The core loop: **discover read annotate**.
### Flow A Discover hot topics (search.js)
```bash
# 最近 7 天全部文章,按发布时间降序
node scripts/search.js --days 7
# 关键词搜索 + 相关度排序
node scripts/search.js --q "央行 利率" --sort relevance
# 按分类 + 数据源过滤
node scripts/search.js --category 财经,中美关系 --source reuters-business,nikkei-markets --days 3
# 按已有标记过滤(只看已读的重要文章)
node scripts/search.js --marks 已读:是,重要:高 --days 30
```
Search output includes:
- Result list: ID / title / source / category / publish time / tags / marks
- Facet stats: category / source / tags / marks distributions (top 10 each)
- Pagination info
Use facets to spot trends — a high count in a category or tag signals a hot topic.
### Flow B — Read article detail (article.js)
```bash
# 全部内容(摘要 + 原文 + 译文)
node scripts/article.js 101
# 只看摘要
node scripts/article.js 101 --fields summary
# 只看译文
node scripts/article.js 101 --fields translation
```
Displays: title (ZH/EN), metadata (source/category/tags/authors/publish time/version/translation status), summary sections (overview/background/key quote/impact), full translation, original markdown, and asset list.
### Flow C — Annotate with marks (marks.js)
```bash
# 标记为已读
node scripts/marks.js add 101 已读 是
# 批量标注
node scripts/marks.js add 101 --json '[{"key":"已读","value":"是"},{"key":"重要","value":"高"},{"key":"分类","value":"跟进"}]'
# 删除标记
node scripts/marks.js remove 101 重要
# 查看某篇文章的所有标记
node scripts/marks.js list 101
# 查看当前 App 用过哪些标记键
node scripts/marks.js keys
# 查看某个键下用过哪些值
node scripts/marks.js values 重要
# 按标记检索(找出所有标记为"重要:高"的文章)
node scripts/marks.js marked --marks 重要:高 --page-size 50
```
Marks are **App-isolated** key-value pairs — only the current App's marks are visible. Use them to build a personal classification system: `已读`, `重要`, `分类`, `跟进`, `已分析`, etc.
---
## Marks vs Tags
| | Marks | Tags |
|---|------|------|
| **Writable** | ✅ After the fact via `marks.js` | ❌ Set at push time only |
| **Scope** | Per-App isolated | Global |
| **Search** | `search.js --marks` / `marks.js marked` | `search.js --tags` |
| **API** | `PUT/DELETE/GET /articles/:id/marks` | `GET /articles/tags` (read-only) |
Tags are useful for filtering during search but **cannot be modified** after an article is pushed. For post-hoc annotation and personal classification, **always use marks**.
---
## References
| File | When to read |
|------|--------------|
| News Hub API docs (external) | Full API specification — endpoint details, error codes, scope matrix, pagination conventions. Located in the News Hub repository at `docs/API.md`. |
{
"name": "news-hub-feed",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "news-hub-feed",
"version": "1.0.0",
"license": "ISC"
}
}
}
{
"name": "news-hub-feed",
"version": "1.0.0",
"description": "News Hub feed reader — search, detail, marks",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"type": "commonjs",
"dependencies": {}
}
{
"api": {
"enabled": false,
"endpoint": "http://localhost:8080/api/v1",
"appId": "",
"appKeyId": "",
"appSecret": "",
"defaultPageSize": 20,
"retry": {
"maxAttempts": 5,
"baseDelayMs": 1000
}
}
}
'use strict';
/**
* api-client.js — Shared News Hub API client for the news-hub-feed skill.
*
* Provides config loading, auth header construction, retrying fetch, and a
* high-level apiRequest() that every CLI script consumes. Pure Node globals
* (fetch / fs / path / crypto) — zero npm dependencies.
*
* Every CLI script does: const { loadConfig, requireApi, apiRequest, ... } = require('./api-client');
*/
const fs = require('fs');
const path = require('path');
// ---------------------------------------------------------------------------
// Paths
// ---------------------------------------------------------------------------
const SKILL_DIR = path.dirname(__dirname); // .../skills/news-hub-feed
const CONFIG_PATH = path.join(SKILL_DIR, 'config.json');
const TEMPLATE_PATH = path.join(SKILL_DIR, 'references', 'config-template.json');
// Scopes this skill needs for full functionality.
const REQUIRED_SCOPES = ['articles:read', 'marks:read', 'marks:write'];
// ---------------------------------------------------------------------------
// Config loading (self-bootstrapping)
// ---------------------------------------------------------------------------
/**
* Load config.json, auto-creating it from the template on first call.
* @returns {object} parsed config
*/
function loadConfig() {
if (!fs.existsSync(CONFIG_PATH)) {
const template = JSON.parse(fs.readFileSync(TEMPLATE_PATH, 'utf8'));
fs.writeFileSync(CONFIG_PATH, JSON.stringify(template, null, 2) + '\n');
}
return JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
}
/**
* Validate that the api block is enabled and credentials are present.
* Exits the process with a helpful message if not.
* @param {object} cfg — full config object
* @param {object} [opts] — { allowDisabled: false } skips the enabled check
* @returns {object} the cfg.api sub-object
*/
function requireApi(cfg, opts = {}) {
const api = cfg.api;
if (!api) {
console.error('✗ config.json 缺少 api 配置块。请参考 references/config-template.json。');
process.exit(1);
}
if (!opts.allowDisabled && api.enabled !== true) {
console.error('✗ api.enabled 为 false。请编辑 config.json 启用 API 连接。');
process.exit(1);
}
const missing = ['endpoint', 'appId', 'appKeyId', 'appSecret'].filter(
(k) => !api[k]
);
if (missing.length > 0) {
console.error(
`✗ api 配置不完整,缺少: ${missing.join(', ')}。请编辑 config.json 填入凭证。`
);
process.exit(1);
}
return api;
}
// ---------------------------------------------------------------------------
// HTTP helpers
// ---------------------------------------------------------------------------
/**
* Build the three custom auth headers.
* @param {object} api — cfg.api sub-object
* @returns {object} header map
*/
function authHeaders(api) {
return {
'X-App-Id': api.appId,
'X-App-Key-Id': api.appKeyId,
'X-App-Secret': api.appSecret,
};
}
/**
* Resolve a full URL for an API path.
* @param {object} api
* @param {string} apiPath — e.g. "/articles/search" (leading / required)
* @param {object} [query] — key→value (arrays become repeated keys; null/undefined skipped)
* @returns {string}
*/
function buildUrl(api, apiPath, query) {
const base = api.endpoint.replace(/\/$/, '');
let url = base + apiPath;
if (query && Object.keys(query).length > 0) {
const params = new URLSearchParams();
for (const [k, v] of Object.entries(query)) {
if (v === null || v === undefined || v === '') continue;
if (Array.isArray(v)) {
for (const item of v) params.append(k, String(item));
} else {
params.append(k, String(v));
}
}
const qs = params.toString();
if (qs) url += (url.includes('?') ? '&' : '?') + qs;
}
return url;
}
/**
* Retry-aware fetch. Retries on network errors, 5xx, and 429.
* @param {string} label — human-readable label for logging
* @param {string} url
* @param {object} fetchOpts — passed to fetch()
* @param {object} retryCfg — { maxAttempts, baseDelayMs }
* @returns {Promise<Response>}
*/
async function retryingFetch(label, url, fetchOpts, retryCfg) {
const max = (retryCfg && retryCfg.maxAttempts) || 5;
const baseDelay = (retryCfg && retryCfg.baseDelayMs) || 1000;
let lastErr;
let lastResp;
for (let attempt = 1; attempt <= max; attempt++) {
try {
const resp = await fetch(url, fetchOpts);
// Retry on 5xx and 429
if (resp.status >= 500 || resp.status === 429) {
if (attempt < max) {
const delay = baseDelay * Math.pow(2, attempt - 1);
console.error(
`↻ ${label}: HTTP ${resp.status}, retry ${attempt}/${max} in ${delay}ms`
);
await sleep(delay);
lastResp = resp;
continue;
}
return resp;
}
return resp;
} catch (err) {
lastErr = err;
if (attempt < max) {
const delay = baseDelay * Math.pow(2, attempt - 1);
console.error(
`↻ ${label}: ${err.message}, retry ${attempt}/${max} in ${delay}ms`
);
await sleep(delay);
continue;
}
throw err; // exhausted
}
}
if (lastResp) return lastResp;
throw lastErr; // unreachable in practice
}
/**
* High-level API request. Injects auth headers, Content-Type, makes the
* retrying call, parses JSON, and throws on non-2xx.
*
* @param {object} cfg — full config (for retry config)
* @param {string} method — GET | POST | PUT | DELETE
* @param {string} apiPath — e.g. "/articles/search"
* @param {object} [opts]
* @param {object} [opts.body] — JSON body (POST/PUT); sent as-is
* @param {object} [opts.query] — query params
* @param {boolean} [opts.rawResponse] — if true, returns { resp, json } without throwing
* @returns {Promise<object>} parsed JSON response body
*/
async function apiRequest(cfg, method, apiPath, opts = {}) {
const api = cfg.api;
const url = buildUrl(api, apiPath, opts.query);
const retryCfg = api.retry || {};
const headers = { ...authHeaders(api) };
let fetchOpts = { method, headers };
if (opts.body !== undefined) {
headers['Content-Type'] = 'application/json';
fetchOpts.body = JSON.stringify(opts.body);
}
const label = `${method} ${apiPath}`;
const resp = await retryingFetch(label, url, fetchOpts, retryCfg);
// Read body (may be empty for 204)
const text = await resp.text();
let json = null;
if (text) {
try {
json = JSON.parse(text);
} catch {
json = null;
}
}
if (opts.rawResponse) {
return { resp, json, status: resp.status };
}
if (!resp.ok) {
const errBody = json && json.error ? json.error : {};
const code = errBody.code || `HTTP_${resp.status}`;
const message = errBody.message || resp.statusText || 'Unknown error';
const field = errBody.field || '';
const e = new Error(`[${code}] ${message}${field ? ` (field: ${field})` : ''}`);
e.code = code;
e.status = resp.status;
e.field = field;
e.apiPath = apiPath;
throw e;
}
return json;
}
// ---------------------------------------------------------------------------
// Misc utilities
// ---------------------------------------------------------------------------
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/**
* Parse argv into a map of flags. Supports `--flag value`, `--flag=value`,
* and boolean `--flag`. Positional args go into `_`.
* @param {string[]} argv — process.argv.slice(2)
* @returns {{flags: object, positional: string[]}}
*/
function parseArgs(argv) {
const flags = {};
const positional = [];
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a.startsWith('--')) {
const eq = a.indexOf('=');
if (eq > -1) {
flags[a.slice(2, eq)] = a.slice(eq + 1);
} else if (i + 1 < argv.length && !argv[i + 1].startsWith('--')) {
flags[a.slice(2)] = argv[++i];
} else {
flags[a.slice(2)] = true;
}
} else {
positional.push(a);
}
}
return { flags, positional };
}
/**
* Split a CSV string into an array of trimmed non-empty strings.
* @param {string} csv
* @returns {string[]}
*/
function csvSplit(csv) {
if (!csv) return [];
if (Array.isArray(csv)) return csv;
return csv
.split(',')
.map((s) => s.trim())
.filter(Boolean);
}
/**
* Parse a "key:value,key2:value2" marks CSV into an array of {key, value}.
* A token without ":" is treated as key-only (empty value).
* @param {string} csv
* @returns {Array<{key: string, value: string}>}
*/
function parseMarksCsv(csv) {
if (!csv) return [];
return csv
.split(',')
.map((pair) => pair.trim())
.filter(Boolean)
.map((pair) => {
const idx = pair.indexOf(':');
if (idx === -1) return { key: pair, value: '' };
return { key: pair.slice(0, idx).trim(), value: pair.slice(idx + 1).trim() };
});
}
/**
* Format an RFC3339 timestamp for display (shortened).
* @param {string} ts
* @returns {string}
*/
function fmtTime(ts) {
if (!ts) return '';
// 2026-07-31T03:00:00Z → 2026-07-31 03:00
return ts.replace('T', ' ').replace(/:\d{2}Z?$/, '').replace(/Z$/, '');
}
/**
* Truncate a string to maxLen, adding ellipsis.
*/
function truncate(s, maxLen) {
if (!s) return '';
if (s.length <= maxLen) return s;
return s.slice(0, maxLen - 1) + '…';
}
module.exports = {
SKILL_DIR,
CONFIG_PATH,
TEMPLATE_PATH,
REQUIRED_SCOPES,
loadConfig,
requireApi,
authHeaders,
buildUrl,
retryingFetch,
apiRequest,
sleep,
parseArgs,
csvSplit,
parseMarksCsv,
fmtTime,
truncate,
};
'use strict';
/**
* article.js — Fetch and display a single article's full detail.
*
* Usage:
* node scripts/article.js <id> [--fields summary,original,translation]
* node scripts/article.js <id> [--json]
*
* --fields 控制返回字段组 (逗号分隔): summary, original, translation
* 缺省 = 全部
* --json 输出原始 JSON
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
fmtTime,
truncate,
} = require('./api-client');
const TRANSLATION_LABELS = { 0: '待翻译', 1: '已翻译', 2: '翻译失败' };
function printArticle(a) {
console.log();
console.log(`════════════════════════════════════════════════════════════`);
console.log(` #${a.articleId} ${a.titleZh || '(无中文标题)'}`);
if (a.titleEn) console.log(` EN: ${a.titleEn}`);
console.log(`════════════════════════════════════════════════════════════`);
console.log();
// Meta
console.log('── 元信息 ──');
console.log(` 来源 : ${a.source}`);
console.log(` 分类 : ${a.category || ''}`);
console.log(` 标签 : ${(a.tags || []).join(', ') || '(无)'}`);
console.log(` 作者 : ${a.authors || ''}`);
console.log(` 出版方 : ${a.publisher || ''}`);
console.log(` 发布时间 : ${fmtTime(a.publishedAt) || '(未知)'}`);
console.log(` 处理时间 : ${fmtTime(a.processedAt) || ''}`);
console.log(` 采集日期 : ${a.harvestDate || ''}`);
console.log(` 版本 : ${a.version ?? ''}`);
console.log(
` 翻译状态 : ${TRANSLATION_LABELS[a.translationStatus] ?? a.translationStatus}`
);
console.log(` URL : ${a.url || ''}`);
console.log();
// Summary sections
const hasSummary =
a.summaryMarkdown ||
a.summaryOverview ||
a.summaryBackground ||
a.summaryKeyQuote ||
a.summaryImpact;
if (hasSummary) {
console.log('── 摘要 ──');
if (a.summaryOverview) console.log(` [概述] ${a.summaryOverview}`);
if (a.summaryBackground) console.log(` [背景] ${a.summaryBackground}`);
if (a.summaryKeyQuote) console.log(` [关键引述] ${a.summaryKeyQuote}`);
if (a.summaryImpact) console.log(` [影响分析] ${a.summaryImpact}`);
if (a.summaryMarkdown) {
console.log();
console.log(indent(a.summaryMarkdown, ' '));
}
console.log();
}
// Translation
if (a.translation) {
console.log('── 译文 ──');
console.log(indent(a.translation, ' '));
console.log();
}
// Original
if (a.originalMarkdown) {
console.log('── 原文 ──');
console.log(indent(a.originalMarkdown, ' '));
console.log();
}
// Assets
if (a.assets && a.assets.length > 0) {
console.log(`── 配图 (${a.assets.length}) ──`);
a.assets.forEach((asset, i) => {
console.log(` [${i}] #${asset.assetId} (${asset.contentType || '?'}, ${formatBytes(asset.sizeBytes)})`);
if (asset.alt) console.log(` alt: ${asset.alt}`);
if (asset.originalSrc) console.log(` src: ${asset.originalSrc}`);
if (asset.url) console.log(` url: ${truncate(asset.url, 80)}`);
});
console.log();
}
}
function formatBytes(n) {
if (!n) return '?B';
if (n < 1024) return `${n}B`;
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)}KB`;
return `${(n / 1024 / 1024).toFixed(1)}MB`;
}
function indent(text, prefix) {
return text
.split('\n')
.map((line) => prefix + line)
.join('\n');
}
async function main() {
const { flags, positional } = parseArgs(process.argv.slice(2));
if (positional.length === 0) {
console.error('用法: node scripts/article.js <id> [--fields summary,original,translation] [--json]');
process.exit(1);
}
const id = positional[0];
if (!/^\d+$/.test(id)) {
console.error(`✗ 文章 ID 必须是正整数,得到: ${id}`);
process.exit(1);
}
const cfg = loadConfig();
requireApi(cfg);
const query = {};
if (flags.fields) query.fields = flags.fields;
const article = await apiRequest(cfg, 'GET', `/articles/${id}`, { query });
if (flags.json) {
process.stdout.write(JSON.stringify(article, null, 2) + '\n');
return;
}
printArticle(article);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* check.js — Connectivity + identity / scope verification.
*
* Usage:
* node scripts/check.js
*
* Calls GET /api/v1/me and prints the App identity + scopes, then verifies
* that all required scopes are present. This is the first-run pre-flight gate.
*/
const {
loadConfig,
requireApi,
apiRequest,
REQUIRED_SCOPES,
} = require('./api-client');
async function main() {
const cfg = loadConfig();
const api = requireApi(cfg);
console.log('正在连接 News Hub API …');
console.log(` endpoint: ${api.endpoint}`);
let me;
try {
me = await apiRequest(cfg, 'GET', '/me');
} catch (err) {
console.error(`\n✗ 连接失败: ${err.message}`);
if (err.code === 'UNAUTHENTICATED') {
console.error(' 请检查 config.json 中的 appId / appKeyId / appSecret 是否正确。');
}
process.exit(1);
}
console.log('\n✓ 连接成功\n');
console.log(` App ID : ${me.appId}`);
console.log(` Key ID : ${me.keyId}`);
console.log(` 名称 : ${me.name}`);
console.log(` Scopes : ${(me.scopes || []).join(', ') || '(无)'}`);
// Scope check
const have = new Set(me.scopes || []);
const missing = REQUIRED_SCOPES.filter((s) => !have.has(s));
if (missing.length > 0) {
console.error(`\n⚠️ 缺少必需 scope: ${missing.join(', ')}`);
console.error(' 本 skill 需要 articles:read + marks:read + marks:write。');
console.error(' 缺少 marks:write 将无法写入标记,缺少 articles:read 将无法检索文章。');
process.exit(1);
} else {
console.log('\n✓ 必需 scope 齐全');
}
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* marks.js — Mark (key-value annotation) management, sub-command style.
*
* Usage:
* node scripts/marks.js add <id> <key> [value] 写入单个标记
* node scripts/marks.js add <id> --json '<jsonArray>' 批量写入
* node scripts/marks.js remove <id> <key> 删除标记
* node scripts/marks.js list <id> 列出文章标记
* node scripts/marks.js keys 当前 App 的标记键列表
* node scripts/marks.js values <key> 指定键的值列表
* node scripts/marks.js marked [options] 按标记检索文章
*
* marked 选项:
* --marks <csv> key:value,key2:value2 (AND)
* --key <k> 单标记简写: 键
* --value <v> 单标记简写: 值 (key 非空 value 为空 → key-only 匹配)
* --page <N> 页码 (默认 1)
* --page-size <N> 每页条数 (默认 20)
* --json 输出原始 JSON
*/
const fs = require('fs');
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
parseMarksCsv,
fmtTime,
truncate,
} = require('./api-client');
const HELP = `\
用法:
node scripts/marks.js add <id> <key> [value] 写入单个标记
node scripts/marks.js add <id> --json '<jsonArray>' 批量写入
node scripts/marks.js remove <id> <key> 删除标记
node scripts/marks.js list <id> 列出文章标记
node scripts/marks.js keys 当前 App 的标记键列表
node scripts/marks.js values <key> 指定键的值列表
node scripts/marks.js marked [--marks csv|--key k --value v] [--page N] [--page-size N]
`;
// ---------------------------------------------------------------------------
// Sub-commands
// ---------------------------------------------------------------------------
async function cmdAdd(cfg, positional, flags) {
if (positional.length < 2) {
console.error('用法: node scripts/marks.js add <id> <key> [value]');
console.error(' node scripts/marks.js add <id> --json \'[...]\'');
process.exit(1);
}
const id = positional[0];
validateId(id);
let body;
if (flags.json) {
try {
body = JSON.parse(flags.json);
} catch {
console.error('✗ --json 的值不是合法 JSON');
process.exit(1);
}
} else {
const key = positional[1];
const value = positional[2] !== undefined ? positional[2] : '';
if (!key) {
console.error('✗ key 不能为空');
process.exit(1);
}
body = { key, value };
}
const result = await apiRequest(cfg, 'PUT', `/articles/${id}/marks`, { body });
if (Array.isArray(result.marks)) {
console.log(`✓ 已写入 ${result.marks.length} 个标记到文章 #${id}:`);
for (const m of result.marks) {
console.log(` ${m.key} = ${m.value} (updated: ${fmtTime(m.updatedAt)})`);
}
} else {
console.log(`✓ 已写入标记到文章 #${id}:`);
console.log(` ${result.key} = ${result.value} (updated: ${fmtTime(result.updatedAt)})`);
}
}
async function cmdRemove(cfg, positional) {
if (positional.length < 2) {
console.error('用法: node scripts/marks.js remove <id> <key>');
process.exit(1);
}
const id = positional[0];
const key = positional[1];
validateId(id);
if (!key) {
console.error('✗ key 不能为空');
process.exit(1);
}
await apiRequest(cfg, 'DELETE', `/articles/${id}/marks/${encodeURIComponent(key)}`);
console.log(`✓ 已删除文章 #${id} 的标记 "${key}"`);
}
async function cmdList(cfg, positional) {
if (positional.length < 1) {
console.error('用法: node scripts/marks.js list <id>');
process.exit(1);
}
const id = positional[0];
validateId(id);
const result = await apiRequest(cfg, 'GET', `/articles/${id}/marks`);
const marks = result.marks || [];
if (marks.length === 0) {
console.log(`文章 #${id} 无标记`);
return;
}
console.log(`文章 #${id} 的标记 (${marks.length}):`);
for (const m of marks) {
console.log(` ${m.key} = ${m.value} (updated: ${fmtTime(m.updatedAt)})`);
}
}
async function cmdKeys(cfg) {
const result = await apiRequest(cfg, 'GET', '/marks/keys');
const keys = Array.isArray(result) ? result : [];
if (keys.length === 0) {
console.log('当前 App 无标记键');
return;
}
console.log(`标记键列表 (${keys.length}):`);
for (const k of keys) {
console.log(` ${k.key} (${k.count} 篇)`);
}
}
async function cmdValues(cfg, positional) {
if (positional.length < 1) {
console.error('用法: node scripts/marks.js values <key>');
process.exit(1);
}
const key = positional[0];
const result = await apiRequest(cfg, 'GET', '/marks/values', {
query: { key },
});
const values = Array.isArray(result) ? result : [];
if (values.length === 0) {
console.log(`标记键 "${key}" 下无值`);
return;
}
console.log(`标记键 "${key}" 的值 (${values.length}):`);
for (const v of values) {
console.log(` ${v}`);
}
}
async function cmdMarked(cfg, flags) {
const query = {};
if (flags.marks) {
query.marks = flags.marks;
} else if (flags.key) {
query.key = flags.key;
if (flags.value) query.value = flags.value;
} else {
console.error('用法: node scripts/marks.js marked --marks <csv> | --key <k> [--value <v>]');
process.exit(1);
}
query.page = flags.page ? parseInt(flags.page, 10) : 1;
const pageSize = flags['page-size']
? parseInt(flags['page-size'], 10)
: (cfg.api && cfg.api.defaultPageSize) || 20;
query.pageSize = isNaN(pageSize) ? 20 : pageSize;
const resp = await apiRequest(cfg, 'GET', '/articles/marked', { query });
if (flags.json) {
process.stdout.write(JSON.stringify(resp, null, 2) + '\n');
return;
}
const { total, page, pageSize: ps, items } = resp;
console.log(`\n共 ${total} 篇 | 第 ${page} 页 | 每页 ${ps} 条\n`);
if (!items || items.length === 0) {
console.log('(无匹配结果)');
return;
}
for (const item of items) {
const title = truncate(item.titleZh || item.titleEn || '(无标题)', 50);
const src = truncate(item.source || '', 16);
const cat = truncate(item.category || '', 10);
const time = fmtTime(item.publishedAt);
console.log(` #${item.articleId} ${title}`);
console.log(` ${src} | ${cat} | ${time}`);
if (item.summaryOverview) {
console.log(` ${truncate(item.summaryOverview, 80)}`);
}
console.log();
}
}
// ---------------------------------------------------------------------------
// Utils
// ---------------------------------------------------------------------------
function validateId(id) {
if (!/^\d+$/.test(String(id))) {
console.error(`✗ 文章 ID 必须是正整数,得到: ${id}`);
process.exit(1);
}
}
// ---------------------------------------------------------------------------
// Main dispatch
// ---------------------------------------------------------------------------
async function main() {
const { flags, positional } = parseArgs(process.argv.slice(2));
const subcommand = positional[0];
const rest = positional.slice(1);
if (!subcommand || subcommand === 'help' || subcommand === '--help' || subcommand === '-h') {
console.log(HELP);
return;
}
const cfg = loadConfig();
requireApi(cfg);
switch (subcommand) {
case 'add':
await cmdAdd(cfg, rest, flags);
break;
case 'remove':
case 'rm':
await cmdRemove(cfg, rest);
break;
case 'list':
case 'ls':
await cmdList(cfg, rest);
break;
case 'keys':
await cmdKeys(cfg);
break;
case 'values':
await cmdValues(cfg, rest);
break;
case 'marked':
await cmdMarked(cfg, flags);
break;
default:
console.error(`✗ 未知子命令: ${subcommand}\n`);
console.log(HELP);
process.exit(1);
}
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* search.js — Article search & hot-topic discovery (main entry point).
*
* Usage:
* node scripts/search.js [options]
*
* Options:
* --q <keyword> 全文搜索关键词
* --source <csv> 数据源过滤 (OR)
* --category <csv> 分类过滤 (OR)
* --tags <csv> 标签过滤 (OR)
* --days <N> 便捷: publishedFrom = now - N 天
* --published-from <RFC3339> 发布时间下界
* --published-to <RFC3339> 发布时间上界
* --harvest-from <YYYYMMDD> 采集日期下界
* --harvest-to <YYYYMMDD> 采集日期上界
* --translation-status <csv> 翻译状态过滤 (0/1/2)
* --marks <csv> 标记过滤 key:value,key2:value2 (AND)
* --sort <mode> relevance | published_desc | published_asc
* --page <N> 页码 (默认 1)
* --page-size <N> 每页条数 (默认 20, 上限 100)
* --json 输出原始 JSON (不格式化)
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
csvSplit,
parseMarksCsv,
fmtTime,
truncate,
} = require('./api-client');
function buildRequestBody(flags, cfg) {
const body = {};
if (flags.q) body.q = flags.q;
if (flags.source) body.source = csvSplit(flags.source);
if (flags.category) body.category = csvSplit(flags.category);
if (flags.tags) body.tags = csvSplit(flags.tags);
if (flags['translation-status']) {
body.translationStatus = csvSplit(flags['translation-status']).map((s) =>
parseInt(s, 10)
);
}
// --days convenience: compute publishedFrom
if (flags.days) {
const n = parseInt(flags.days, 10);
if (!isNaN(n) && n > 0) {
const from = new Date(Date.now() - n * 24 * 60 * 60 * 1000);
body.publishedFrom = from.toISOString().replace(/\.\d{3}Z$/, 'Z');
}
}
if (flags['published-from']) body.publishedFrom = flags['published-from'];
if (flags['published-to']) body.publishedTo = flags['published-to'];
if (flags['harvest-from']) body.harvestFrom = flags['harvest-from'];
if (flags['harvest-to']) body.harvestTo = flags['harvest-to'];
if (flags.marks) {
body.marks = parseMarksCsv(flags.marks);
}
body.sort = flags.sort || 'published_desc';
const pageSize = flags['page-size']
? parseInt(flags['page-size'], 10)
: (cfg.api && cfg.api.defaultPageSize) || 20;
body.page = flags.page ? parseInt(flags.page, 10) : 1;
body.pageSize = isNaN(pageSize) ? 20 : pageSize;
return body;
}
function printResults(resp) {
const { total, page, pageSize, items, facets } = resp;
console.log(`\n共 ${total} 篇 | 第 ${page} 页 | 每页 ${pageSize} 条\n`);
if (!items || items.length === 0) {
console.log('(无匹配结果)');
return;
}
// Results table
for (const item of items) {
const id = item.articleId;
const title = truncate(item.titleZh || item.titleEn || '(无标题)', 50);
const src = truncate(item.source || '', 16);
const cat = truncate(item.category || '', 10);
const time = fmtTime(item.publishedAt);
const tags = (item.tags || []).slice(0, 3).join('/');
const marksStr = (item.marks || [])
.map((m) => `${m.key}:${m.value || ''}`)
.join(' ');
console.log(` #${id} ${title}`);
console.log(
` ${src} | ${cat} | ${time}${tags ? ` | ${tags}` : ''}${
marksStr ? ` | [${marksStr}]` : ''
}`
);
if (item.summaryOverview) {
console.log(` ${truncate(item.summaryOverview, 80)}`);
}
console.log();
}
// Facets
if (facets) {
console.log('── 分面统计 ──');
printFacet('分类', facets.category);
printFacet('数据源', facets.source);
printFacet('标签', facets.tags);
printFacet('标记', facets.marks);
}
}
function printFacet(label, map) {
if (!map || Object.keys(map).length === 0) return;
const entries = Object.entries(map).sort((a, b) => b[1] - a[1]);
const top = entries.slice(0, 10);
const parts = top.map(([k, v]) => `${k}(${v})`).join(' ');
console.log(` ${label}: ${parts}`);
}
async function main() {
const { flags } = parseArgs(process.argv.slice(2));
const cfg = loadConfig();
requireApi(cfg);
const body = buildRequestBody(flags, cfg);
if (flags.json) {
const resp = await apiRequest(cfg, 'POST', '/articles/search', { body });
process.stdout.write(JSON.stringify(resp, null, 2) + '\n');
return;
}
// Log the query for context
const parts = [];
if (body.q) parts.push(`关键词="${body.q}"`);
if (body.source && body.source.length) parts.push(`源=${body.source.join(',')}`);
if (body.category && body.category.length) parts.push(`分类=${body.category.join(',')}`);
if (body.tags && body.tags.length) parts.push(`标签=${body.tags.join(',')}`);
if (body.marks && body.marks.length) parts.push(`标记=${body.marks.map((m) => `${m.key}:${m.value}`).join(',')}`);
if (body.publishedFrom) parts.push(`from=${body.publishedFrom}`);
if (body.publishedTo) parts.push(`to=${body.publishedTo}`);
if (body.harvestFrom) parts.push(`harvest=${body.harvestFrom}~${body.harvestTo || ''}`);
console.log(`搜索: ${parts.join(' | ') || '(全部)'}`);
const resp = await apiRequest(cfg, 'POST', '/articles/search', { body });
printResults(resp);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* tags.js — List all tags used across articles (read-only).
*
* Usage:
* node scripts/tags.js [--json]
*
* Tags are set at push time and cannot be modified after the fact.
* Use marks (marks.js) for post-hoc annotation.
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
} = require('./api-client');
async function main() {
const { flags } = parseArgs(process.argv.slice(2));
const cfg = loadConfig();
requireApi(cfg);
const tags = await apiRequest(cfg, 'GET', '/articles/tags');
const list = Array.isArray(tags) ? tags : [];
if (flags.json) {
process.stdout.write(JSON.stringify(list, null, 2) + '\n');
return;
}
if (list.length === 0) {
console.log('系统中暂无标签');
return;
}
console.log(`\n标签列表 (${list.length}):\n`);
console.log(' 标签 文章数');
console.log(' ──────────────────────────────');
for (const t of list) {
console.log(` ${padRight(t.tag || '', 22)} ${t.count}`);
}
console.log();
console.log('提示: tags 在推送时设定,只读不可事后修改。');
console.log(' 如需事后标注,请使用 marks: node scripts/marks.js add <id> <key> [value]');
}
function padRight(s, len) {
s = String(s);
if (s.length >= len) return s;
return s + ' '.repeat(len - s.length);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
......@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) {
'--no-default-browser-check',
'--disable-extensions',
'--disable-gpu',
// Hide the navigator.webdriver=true flag that CDP sets by default.
// Anti-bot systems (PerimeterX, Cloudflare) check this on page load.
// Combined with the stealth injection in cdp-client.js openSession().
'--disable-blink-features=AutomationControlled',
// NOTE: --disable-blink-features=AutomationControlled was removed.
// It triggers an "unsupported command-line flag" warning banner on some
// Chromium builds (e.g. Arch Linux), which is itself a detectable artifact.
// navigator.webdriver suppression is handled entirely by the CDP injection
// in cdp-client.js openSession() via Page.addScriptToEvaluateOnNewDocument,
// which sets navigator.webdriver to undefined before any page script runs.
// That approach is flag-independent, more robust, and produces no banner.
// Let Chromium auto-detect Wayland vs X11 on Linux. Without this it
// defaults to the X11 backend and crashes with "Missing X server or
// $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment