Commit 4685f05e authored by 谢宇轩's avatar 谢宇轩

feat: add news hub feed skills

parent 6a313771
# News Hub Skills # News Hub Skills
Two independent skills for the news harvesting pipeline: Three independent skills for the news harvesting + consumption pipeline:
## news-source-analyzer (无状态分析器) ## news-source-analyzer (无状态分析器)
...@@ -10,6 +10,10 @@ Analyze news websites, author fetch recipes, and produce installable source bund ...@@ -10,6 +10,10 @@ Analyze news websites, author fetch recipes, and produce installable source bund
Harvest and archive news articles from configured sources to Obsidian vault with Chinese summaries + original text. Sources are installed from bundles produced by the analyzer. Manages vault path, push config, and source registry. Harvest and archive news articles from configured sources to Obsidian vault with Chinese summaries + original text. Sources are installed from bundles produced by the analyzer. Manages vault path, push config, and source registry.
## news-hub-feed (阅读 / 标注)
Search and browse News Hub articles, read full article details (summary / original / translation), and manage key-value marks for annotation. A pure API consumer — no chromium, no vault, no bundle, zero npm dependencies. Used for rapid hot-topic discovery and post-hoc annotation.
## Data flow ## Data flow
``` ```
...@@ -21,13 +25,17 @@ preview → verify article extraction ↓ ...@@ -21,13 +25,17 @@ preview → verify article extraction ↓
↓ harvest → summaries → finalize → push ↓ harvest → summaries → finalize → push
recipe-test → regression lock ↓ recipe-test → regression lock ↓
↓ Obsidian vault + News Hub ↓ Obsidian vault + News Hub
pack → bundle.nhsource.json pack → bundle.nhsource.json ↑
│ │ read + annotate
└───── manual/scripted transfer ─────→ harvester │
└───── manual/scripted transfer ─────→ harvester Feed (any machine) ───┘
─────────────────
search → article detail → marks
``` ```
The `.nhsource.json` bundle is the only interface between the two skills. No shared runtime state, no shared config, no shared browser profile. The `.nhsource.json` bundle is the only interface between the analyzer and harvester. The feed skill connects directly to the News Hub API — no shared runtime state, config, or browser profile with the other two.
## Shared scripts ## Shared scripts
Four CDP/Chromium infrastructure scripts (`ensure-chromium.js`, `cdp-client.js`, `fetcher-helper.js`, `block-check.js`) are duplicated in both skills as standalone copies. These are stable low-level CDP wrappers with low change frequency. When modifying one, sync the copy in the other skill. Four CDP/Chromium infrastructure scripts (`ensure-chromium.js`, `cdp-client.js`, `fetcher-helper.js`, `block-check.js`) are duplicated in the analyzer and harvester skills as standalone copies. These are stable low-level CDP wrappers with low change frequency. When modifying one, sync the copy in the other skill. The feed skill does not use Chromium.
...@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) { ...@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) {
'--no-default-browser-check', '--no-default-browser-check',
'--disable-extensions', '--disable-extensions',
'--disable-gpu', '--disable-gpu',
// Hide the navigator.webdriver=true flag that CDP sets by default. // NOTE: --disable-blink-features=AutomationControlled was removed.
// Anti-bot systems (PerimeterX, Cloudflare) check this on page load. // It triggers an "unsupported command-line flag" warning banner on some
// Combined with the stealth injection in cdp-client.js openSession(). // Chromium builds (e.g. Arch Linux), which is itself a detectable artifact.
'--disable-blink-features=AutomationControlled', // navigator.webdriver suppression is handled entirely by the CDP injection
// in cdp-client.js openSession() via Page.addScriptToEvaluateOnNewDocument,
// which sets navigator.webdriver to undefined before any page script runs.
// That approach is flag-independent, more robust, and produces no banner.
// Let Chromium auto-detect Wayland vs X11 on Linux. Without this it // Let Chromium auto-detect Wayland vs X11 on Linux. Without this it
// defaults to the X11 backend and crashes with "Missing X server or // defaults to the X11 backend and crashes with "Missing X server or
// $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless // $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless
......
---
name: news-hub-feed
description: Search, browse, and annotate News Hub articles via the News Hub API. Use this skill whenever the user wants to discover trending news, search articles by keyword/source/category/tags/marks, read an article's full detail (summary, original, translation), or add/remove key-value marks for later retrieval and analysis. A lightweight read + annotate client — no harvesting, no pushing, no browser. Triggers on phrases like "检索新闻", "搜索文章", "查看文章", "文章详情", "添加标记", "标注", "热点", "热门", "hot news", "trending", "search articles", "article detail", "marks", "discover trending", "标签列表", "按标记检索".
---
# News Hub Feed Skill
Search and browse articles stored in News Hub, read full article details (summary / original / translation), and manage key-value **marks** for annotation. This skill is a **pure API consumer** — it never harvests, pushes, or launches a browser. Its sole purpose is rapid hot-topic discovery and post-hoc annotation for later retrieval and analysis.
> **Skill root** = the directory containing this `SKILL.md`. Prefix every command with `cd "<skill-root>" &&`, or scripts fail with `MODULE_NOT_FOUND`.
## Quick Reference
| Task | Command |
|------|---------|
| Check connectivity + scopes | `node scripts/check.js` |
| Search articles (hot topics) | `node scripts/search.js --q <keyword> [--days N] [--source csv] [--category csv] [--tags csv] [--marks csv]` |
| Search with facets | `node scripts/search.js --q 利率 --days 7 --sort relevance` |
| Article detail | `node scripts/article.js <id> [--fields summary,original,translation]` |
| Add a mark | `node scripts/marks.js add <id> <key> [value]` |
| Add marks (batch) | `node scripts/marks.js add <id> --json '[{"key":"已读","value":"是"},{"key":"重要","value":"高"}]'` |
| Remove a mark | `node scripts/marks.js remove <id> <key>` |
| List article marks | `node scripts/marks.js list <id>` |
| List mark keys | `node scripts/marks.js keys` |
| List mark values | `node scripts/marks.js values <key>` |
| Search by marks | `node scripts/marks.js marked --marks 已读:是,重要:高` |
| List all tags (read-only) | `node scripts/tags.js` |
> **All commands must be prefixed with `cd "<skill-root>" &&`**. Scripts live in `scripts/` — always reference them as `node scripts/search.js`, never `node search.js`.
---
## First-Run Setup
**Pre-flight gate** — run this at the start of a task. If it prints `OK`, skip setup:
```bash
cd "<skill-root>" && node -e "const c=require('./config.json'); console.log(c.api&&c.api.enabled?'OK':'UNCONFIGURED')" 2>/dev/null || echo "UNCONFIGURED (no config.json)"
```
### Onboarding (only if UNCONFIGURED)
This skill only needs the `api` block in `config.json` (no chromium, no vault, no sources). The config is auto-created from `references/config-template.json` on first script run. To configure:
1. Edit `config.json` — fill in the `api` block:
```json
{
"api": {
"enabled": true,
"endpoint": "http://localhost:8080/api/v1",
"appId": "app_feed",
"appKeyId": "kh_feed_default",
"appSecret": "<your-secret>",
"defaultPageSize": 20,
"retry": { "maxAttempts": 5, "baseDelayMs": 1000 }
}
}
```
The App must hold these scopes: `articles:read`, `marks:read`, `marks:write`.
2. Verify connectivity:
```bash
node scripts/check.js
```
If you see `✓ 连接成功` and `✓ 必需 scope 齐全`, setup is complete.
---
## Configuration
| Field | Required? | Description |
|-------|-----------|-------------|
| `api.enabled` | | Must be `true` to make any API call |
| `api.endpoint` | | News Hub base URL, e.g. `http://localhost:8080/api/v1` |
| `api.appId` | | App public identifier (e.g. `app_feed`) |
| `api.appKeyId` | | Credential Key ID (e.g. `kh_feed_default`) |
| `api.appSecret` | | Secret plaintext (server verifies via bcrypt) |
| `api.defaultPageSize` | Optional | Default page size for search/marked (default `20`, max `100`) |
| `api.retry.maxAttempts` | Optional | Max retry attempts on 5xx/429/network errors (default `5`) |
| `api.retry.baseDelayMs` | Optional | Base delay for exponential backoff (default `1000`ms) |
> Auth uses three custom headers (`X-App-Id`, `X-App-Key-Id`, `X-App-Secret`) identical to the harvester's push config. You can reuse the same App credentials if it holds the required scopes, or configure a dedicated read/annotate App.
---
## Workflow
The core loop: **discover read annotate**.
### Flow A Discover hot topics (search.js)
```bash
# 最近 7 天全部文章,按发布时间降序
node scripts/search.js --days 7
# 关键词搜索 + 相关度排序
node scripts/search.js --q "央行 利率" --sort relevance
# 按分类 + 数据源过滤
node scripts/search.js --category 财经,中美关系 --source reuters-business,nikkei-markets --days 3
# 按已有标记过滤(只看已读的重要文章)
node scripts/search.js --marks 已读:是,重要:高 --days 30
```
Search output includes:
- Result list: ID / title / source / category / publish time / tags / marks
- Facet stats: category / source / tags / marks distributions (top 10 each)
- Pagination info
Use facets to spot trends — a high count in a category or tag signals a hot topic.
### Flow B — Read article detail (article.js)
```bash
# 全部内容(摘要 + 原文 + 译文)
node scripts/article.js 101
# 只看摘要
node scripts/article.js 101 --fields summary
# 只看译文
node scripts/article.js 101 --fields translation
```
Displays: title (ZH/EN), metadata (source/category/tags/authors/publish time/version/translation status), summary sections (overview/background/key quote/impact), full translation, original markdown, and asset list.
### Flow C — Annotate with marks (marks.js)
```bash
# 标记为已读
node scripts/marks.js add 101 已读 是
# 批量标注
node scripts/marks.js add 101 --json '[{"key":"已读","value":"是"},{"key":"重要","value":"高"},{"key":"分类","value":"跟进"}]'
# 删除标记
node scripts/marks.js remove 101 重要
# 查看某篇文章的所有标记
node scripts/marks.js list 101
# 查看当前 App 用过哪些标记键
node scripts/marks.js keys
# 查看某个键下用过哪些值
node scripts/marks.js values 重要
# 按标记检索(找出所有标记为"重要:高"的文章)
node scripts/marks.js marked --marks 重要:高 --page-size 50
```
Marks are **App-isolated** key-value pairs — only the current App's marks are visible. Use them to build a personal classification system: `已读`, `重要`, `分类`, `跟进`, `已分析`, etc.
---
## Marks vs Tags
| | Marks | Tags |
|---|------|------|
| **Writable** | ✅ After the fact via `marks.js` | ❌ Set at push time only |
| **Scope** | Per-App isolated | Global |
| **Search** | `search.js --marks` / `marks.js marked` | `search.js --tags` |
| **API** | `PUT/DELETE/GET /articles/:id/marks` | `GET /articles/tags` (read-only) |
Tags are useful for filtering during search but **cannot be modified** after an article is pushed. For post-hoc annotation and personal classification, **always use marks**.
---
## References
| File | When to read |
|------|--------------|
| News Hub API docs (external) | Full API specification — endpoint details, error codes, scope matrix, pagination conventions. Located in the News Hub repository at `docs/API.md`. |
{
"name": "news-hub-feed",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "news-hub-feed",
"version": "1.0.0",
"license": "ISC"
}
}
}
{
"name": "news-hub-feed",
"version": "1.0.0",
"description": "News Hub feed reader — search, detail, marks",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"type": "commonjs",
"dependencies": {}
}
{
"api": {
"enabled": false,
"endpoint": "http://localhost:8080/api/v1",
"appId": "",
"appKeyId": "",
"appSecret": "",
"defaultPageSize": 20,
"retry": {
"maxAttempts": 5,
"baseDelayMs": 1000
}
}
}
'use strict';
/**
* api-client.js — Shared News Hub API client for the news-hub-feed skill.
*
* Provides config loading, auth header construction, retrying fetch, and a
* high-level apiRequest() that every CLI script consumes. Pure Node globals
* (fetch / fs / path / crypto) — zero npm dependencies.
*
* Every CLI script does: const { loadConfig, requireApi, apiRequest, ... } = require('./api-client');
*/
const fs = require('fs');
const path = require('path');
// ---------------------------------------------------------------------------
// Paths
// ---------------------------------------------------------------------------
const SKILL_DIR = path.dirname(__dirname); // .../skills/news-hub-feed
const CONFIG_PATH = path.join(SKILL_DIR, 'config.json');
const TEMPLATE_PATH = path.join(SKILL_DIR, 'references', 'config-template.json');
// Scopes this skill needs for full functionality.
const REQUIRED_SCOPES = ['articles:read', 'marks:read', 'marks:write'];
// ---------------------------------------------------------------------------
// Config loading (self-bootstrapping)
// ---------------------------------------------------------------------------
/**
* Load config.json, auto-creating it from the template on first call.
* @returns {object} parsed config
*/
function loadConfig() {
if (!fs.existsSync(CONFIG_PATH)) {
const template = JSON.parse(fs.readFileSync(TEMPLATE_PATH, 'utf8'));
fs.writeFileSync(CONFIG_PATH, JSON.stringify(template, null, 2) + '\n');
}
return JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
}
/**
* Validate that the api block is enabled and credentials are present.
* Exits the process with a helpful message if not.
* @param {object} cfg — full config object
* @param {object} [opts] — { allowDisabled: false } skips the enabled check
* @returns {object} the cfg.api sub-object
*/
function requireApi(cfg, opts = {}) {
const api = cfg.api;
if (!api) {
console.error('✗ config.json 缺少 api 配置块。请参考 references/config-template.json。');
process.exit(1);
}
if (!opts.allowDisabled && api.enabled !== true) {
console.error('✗ api.enabled 为 false。请编辑 config.json 启用 API 连接。');
process.exit(1);
}
const missing = ['endpoint', 'appId', 'appKeyId', 'appSecret'].filter(
(k) => !api[k]
);
if (missing.length > 0) {
console.error(
`✗ api 配置不完整,缺少: ${missing.join(', ')}。请编辑 config.json 填入凭证。`
);
process.exit(1);
}
return api;
}
// ---------------------------------------------------------------------------
// HTTP helpers
// ---------------------------------------------------------------------------
/**
* Build the three custom auth headers.
* @param {object} api — cfg.api sub-object
* @returns {object} header map
*/
function authHeaders(api) {
return {
'X-App-Id': api.appId,
'X-App-Key-Id': api.appKeyId,
'X-App-Secret': api.appSecret,
};
}
/**
* Resolve a full URL for an API path.
* @param {object} api
* @param {string} apiPath — e.g. "/articles/search" (leading / required)
* @param {object} [query] — key→value (arrays become repeated keys; null/undefined skipped)
* @returns {string}
*/
function buildUrl(api, apiPath, query) {
const base = api.endpoint.replace(/\/$/, '');
let url = base + apiPath;
if (query && Object.keys(query).length > 0) {
const params = new URLSearchParams();
for (const [k, v] of Object.entries(query)) {
if (v === null || v === undefined || v === '') continue;
if (Array.isArray(v)) {
for (const item of v) params.append(k, String(item));
} else {
params.append(k, String(v));
}
}
const qs = params.toString();
if (qs) url += (url.includes('?') ? '&' : '?') + qs;
}
return url;
}
/**
* Retry-aware fetch. Retries on network errors, 5xx, and 429.
* @param {string} label — human-readable label for logging
* @param {string} url
* @param {object} fetchOpts — passed to fetch()
* @param {object} retryCfg — { maxAttempts, baseDelayMs }
* @returns {Promise<Response>}
*/
async function retryingFetch(label, url, fetchOpts, retryCfg) {
const max = (retryCfg && retryCfg.maxAttempts) || 5;
const baseDelay = (retryCfg && retryCfg.baseDelayMs) || 1000;
let lastErr;
let lastResp;
for (let attempt = 1; attempt <= max; attempt++) {
try {
const resp = await fetch(url, fetchOpts);
// Retry on 5xx and 429
if (resp.status >= 500 || resp.status === 429) {
if (attempt < max) {
const delay = baseDelay * Math.pow(2, attempt - 1);
console.error(
`↻ ${label}: HTTP ${resp.status}, retry ${attempt}/${max} in ${delay}ms`
);
await sleep(delay);
lastResp = resp;
continue;
}
return resp;
}
return resp;
} catch (err) {
lastErr = err;
if (attempt < max) {
const delay = baseDelay * Math.pow(2, attempt - 1);
console.error(
`↻ ${label}: ${err.message}, retry ${attempt}/${max} in ${delay}ms`
);
await sleep(delay);
continue;
}
throw err; // exhausted
}
}
if (lastResp) return lastResp;
throw lastErr; // unreachable in practice
}
/**
* High-level API request. Injects auth headers, Content-Type, makes the
* retrying call, parses JSON, and throws on non-2xx.
*
* @param {object} cfg — full config (for retry config)
* @param {string} method — GET | POST | PUT | DELETE
* @param {string} apiPath — e.g. "/articles/search"
* @param {object} [opts]
* @param {object} [opts.body] — JSON body (POST/PUT); sent as-is
* @param {object} [opts.query] — query params
* @param {boolean} [opts.rawResponse] — if true, returns { resp, json } without throwing
* @returns {Promise<object>} parsed JSON response body
*/
async function apiRequest(cfg, method, apiPath, opts = {}) {
const api = cfg.api;
const url = buildUrl(api, apiPath, opts.query);
const retryCfg = api.retry || {};
const headers = { ...authHeaders(api) };
let fetchOpts = { method, headers };
if (opts.body !== undefined) {
headers['Content-Type'] = 'application/json';
fetchOpts.body = JSON.stringify(opts.body);
}
const label = `${method} ${apiPath}`;
const resp = await retryingFetch(label, url, fetchOpts, retryCfg);
// Read body (may be empty for 204)
const text = await resp.text();
let json = null;
if (text) {
try {
json = JSON.parse(text);
} catch {
json = null;
}
}
if (opts.rawResponse) {
return { resp, json, status: resp.status };
}
if (!resp.ok) {
const errBody = json && json.error ? json.error : {};
const code = errBody.code || `HTTP_${resp.status}`;
const message = errBody.message || resp.statusText || 'Unknown error';
const field = errBody.field || '';
const e = new Error(`[${code}] ${message}${field ? ` (field: ${field})` : ''}`);
e.code = code;
e.status = resp.status;
e.field = field;
e.apiPath = apiPath;
throw e;
}
return json;
}
// ---------------------------------------------------------------------------
// Misc utilities
// ---------------------------------------------------------------------------
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/**
* Parse argv into a map of flags. Supports `--flag value`, `--flag=value`,
* and boolean `--flag`. Positional args go into `_`.
* @param {string[]} argv — process.argv.slice(2)
* @returns {{flags: object, positional: string[]}}
*/
function parseArgs(argv) {
const flags = {};
const positional = [];
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a.startsWith('--')) {
const eq = a.indexOf('=');
if (eq > -1) {
flags[a.slice(2, eq)] = a.slice(eq + 1);
} else if (i + 1 < argv.length && !argv[i + 1].startsWith('--')) {
flags[a.slice(2)] = argv[++i];
} else {
flags[a.slice(2)] = true;
}
} else {
positional.push(a);
}
}
return { flags, positional };
}
/**
* Split a CSV string into an array of trimmed non-empty strings.
* @param {string} csv
* @returns {string[]}
*/
function csvSplit(csv) {
if (!csv) return [];
if (Array.isArray(csv)) return csv;
return csv
.split(',')
.map((s) => s.trim())
.filter(Boolean);
}
/**
* Parse a "key:value,key2:value2" marks CSV into an array of {key, value}.
* A token without ":" is treated as key-only (empty value).
* @param {string} csv
* @returns {Array<{key: string, value: string}>}
*/
function parseMarksCsv(csv) {
if (!csv) return [];
return csv
.split(',')
.map((pair) => pair.trim())
.filter(Boolean)
.map((pair) => {
const idx = pair.indexOf(':');
if (idx === -1) return { key: pair, value: '' };
return { key: pair.slice(0, idx).trim(), value: pair.slice(idx + 1).trim() };
});
}
/**
* Format an RFC3339 timestamp for display (shortened).
* @param {string} ts
* @returns {string}
*/
function fmtTime(ts) {
if (!ts) return '';
// 2026-07-31T03:00:00Z → 2026-07-31 03:00
return ts.replace('T', ' ').replace(/:\d{2}Z?$/, '').replace(/Z$/, '');
}
/**
* Truncate a string to maxLen, adding ellipsis.
*/
function truncate(s, maxLen) {
if (!s) return '';
if (s.length <= maxLen) return s;
return s.slice(0, maxLen - 1) + '…';
}
module.exports = {
SKILL_DIR,
CONFIG_PATH,
TEMPLATE_PATH,
REQUIRED_SCOPES,
loadConfig,
requireApi,
authHeaders,
buildUrl,
retryingFetch,
apiRequest,
sleep,
parseArgs,
csvSplit,
parseMarksCsv,
fmtTime,
truncate,
};
'use strict';
/**
* article.js — Fetch and display a single article's full detail.
*
* Usage:
* node scripts/article.js <id> [--fields summary,original,translation]
* node scripts/article.js <id> [--json]
*
* --fields 控制返回字段组 (逗号分隔): summary, original, translation
* 缺省 = 全部
* --json 输出原始 JSON
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
fmtTime,
truncate,
} = require('./api-client');
const TRANSLATION_LABELS = { 0: '待翻译', 1: '已翻译', 2: '翻译失败' };
function printArticle(a) {
console.log();
console.log(`════════════════════════════════════════════════════════════`);
console.log(` #${a.articleId} ${a.titleZh || '(无中文标题)'}`);
if (a.titleEn) console.log(` EN: ${a.titleEn}`);
console.log(`════════════════════════════════════════════════════════════`);
console.log();
// Meta
console.log('── 元信息 ──');
console.log(` 来源 : ${a.source}`);
console.log(` 分类 : ${a.category || ''}`);
console.log(` 标签 : ${(a.tags || []).join(', ') || '(无)'}`);
console.log(` 作者 : ${a.authors || ''}`);
console.log(` 出版方 : ${a.publisher || ''}`);
console.log(` 发布时间 : ${fmtTime(a.publishedAt) || '(未知)'}`);
console.log(` 处理时间 : ${fmtTime(a.processedAt) || ''}`);
console.log(` 采集日期 : ${a.harvestDate || ''}`);
console.log(` 版本 : ${a.version ?? ''}`);
console.log(
` 翻译状态 : ${TRANSLATION_LABELS[a.translationStatus] ?? a.translationStatus}`
);
console.log(` URL : ${a.url || ''}`);
console.log();
// Summary sections
const hasSummary =
a.summaryMarkdown ||
a.summaryOverview ||
a.summaryBackground ||
a.summaryKeyQuote ||
a.summaryImpact;
if (hasSummary) {
console.log('── 摘要 ──');
if (a.summaryOverview) console.log(` [概述] ${a.summaryOverview}`);
if (a.summaryBackground) console.log(` [背景] ${a.summaryBackground}`);
if (a.summaryKeyQuote) console.log(` [关键引述] ${a.summaryKeyQuote}`);
if (a.summaryImpact) console.log(` [影响分析] ${a.summaryImpact}`);
if (a.summaryMarkdown) {
console.log();
console.log(indent(a.summaryMarkdown, ' '));
}
console.log();
}
// Translation
if (a.translation) {
console.log('── 译文 ──');
console.log(indent(a.translation, ' '));
console.log();
}
// Original
if (a.originalMarkdown) {
console.log('── 原文 ──');
console.log(indent(a.originalMarkdown, ' '));
console.log();
}
// Assets
if (a.assets && a.assets.length > 0) {
console.log(`── 配图 (${a.assets.length}) ──`);
a.assets.forEach((asset, i) => {
console.log(` [${i}] #${asset.assetId} (${asset.contentType || '?'}, ${formatBytes(asset.sizeBytes)})`);
if (asset.alt) console.log(` alt: ${asset.alt}`);
if (asset.originalSrc) console.log(` src: ${asset.originalSrc}`);
if (asset.url) console.log(` url: ${truncate(asset.url, 80)}`);
});
console.log();
}
}
function formatBytes(n) {
if (!n) return '?B';
if (n < 1024) return `${n}B`;
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)}KB`;
return `${(n / 1024 / 1024).toFixed(1)}MB`;
}
function indent(text, prefix) {
return text
.split('\n')
.map((line) => prefix + line)
.join('\n');
}
async function main() {
const { flags, positional } = parseArgs(process.argv.slice(2));
if (positional.length === 0) {
console.error('用法: node scripts/article.js <id> [--fields summary,original,translation] [--json]');
process.exit(1);
}
const id = positional[0];
if (!/^\d+$/.test(id)) {
console.error(`✗ 文章 ID 必须是正整数,得到: ${id}`);
process.exit(1);
}
const cfg = loadConfig();
requireApi(cfg);
const query = {};
if (flags.fields) query.fields = flags.fields;
const article = await apiRequest(cfg, 'GET', `/articles/${id}`, { query });
if (flags.json) {
process.stdout.write(JSON.stringify(article, null, 2) + '\n');
return;
}
printArticle(article);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* check.js — Connectivity + identity / scope verification.
*
* Usage:
* node scripts/check.js
*
* Calls GET /api/v1/me and prints the App identity + scopes, then verifies
* that all required scopes are present. This is the first-run pre-flight gate.
*/
const {
loadConfig,
requireApi,
apiRequest,
REQUIRED_SCOPES,
} = require('./api-client');
async function main() {
const cfg = loadConfig();
const api = requireApi(cfg);
console.log('正在连接 News Hub API …');
console.log(` endpoint: ${api.endpoint}`);
let me;
try {
me = await apiRequest(cfg, 'GET', '/me');
} catch (err) {
console.error(`\n✗ 连接失败: ${err.message}`);
if (err.code === 'UNAUTHENTICATED') {
console.error(' 请检查 config.json 中的 appId / appKeyId / appSecret 是否正确。');
}
process.exit(1);
}
console.log('\n✓ 连接成功\n');
console.log(` App ID : ${me.appId}`);
console.log(` Key ID : ${me.keyId}`);
console.log(` 名称 : ${me.name}`);
console.log(` Scopes : ${(me.scopes || []).join(', ') || '(无)'}`);
// Scope check
const have = new Set(me.scopes || []);
const missing = REQUIRED_SCOPES.filter((s) => !have.has(s));
if (missing.length > 0) {
console.error(`\n⚠️ 缺少必需 scope: ${missing.join(', ')}`);
console.error(' 本 skill 需要 articles:read + marks:read + marks:write。');
console.error(' 缺少 marks:write 将无法写入标记,缺少 articles:read 将无法检索文章。');
process.exit(1);
} else {
console.log('\n✓ 必需 scope 齐全');
}
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* marks.js — Mark (key-value annotation) management, sub-command style.
*
* Usage:
* node scripts/marks.js add <id> <key> [value] 写入单个标记
* node scripts/marks.js add <id> --json '<jsonArray>' 批量写入
* node scripts/marks.js remove <id> <key> 删除标记
* node scripts/marks.js list <id> 列出文章标记
* node scripts/marks.js keys 当前 App 的标记键列表
* node scripts/marks.js values <key> 指定键的值列表
* node scripts/marks.js marked [options] 按标记检索文章
*
* marked 选项:
* --marks <csv> key:value,key2:value2 (AND)
* --key <k> 单标记简写: 键
* --value <v> 单标记简写: 值 (key 非空 value 为空 → key-only 匹配)
* --page <N> 页码 (默认 1)
* --page-size <N> 每页条数 (默认 20)
* --json 输出原始 JSON
*/
const fs = require('fs');
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
parseMarksCsv,
fmtTime,
truncate,
} = require('./api-client');
const HELP = `\
用法:
node scripts/marks.js add <id> <key> [value] 写入单个标记
node scripts/marks.js add <id> --json '<jsonArray>' 批量写入
node scripts/marks.js remove <id> <key> 删除标记
node scripts/marks.js list <id> 列出文章标记
node scripts/marks.js keys 当前 App 的标记键列表
node scripts/marks.js values <key> 指定键的值列表
node scripts/marks.js marked [--marks csv|--key k --value v] [--page N] [--page-size N]
`;
// ---------------------------------------------------------------------------
// Sub-commands
// ---------------------------------------------------------------------------
async function cmdAdd(cfg, positional, flags) {
if (positional.length < 2) {
console.error('用法: node scripts/marks.js add <id> <key> [value]');
console.error(' node scripts/marks.js add <id> --json \'[...]\'');
process.exit(1);
}
const id = positional[0];
validateId(id);
let body;
if (flags.json) {
try {
body = JSON.parse(flags.json);
} catch {
console.error('✗ --json 的值不是合法 JSON');
process.exit(1);
}
} else {
const key = positional[1];
const value = positional[2] !== undefined ? positional[2] : '';
if (!key) {
console.error('✗ key 不能为空');
process.exit(1);
}
body = { key, value };
}
const result = await apiRequest(cfg, 'PUT', `/articles/${id}/marks`, { body });
if (Array.isArray(result.marks)) {
console.log(`✓ 已写入 ${result.marks.length} 个标记到文章 #${id}:`);
for (const m of result.marks) {
console.log(` ${m.key} = ${m.value} (updated: ${fmtTime(m.updatedAt)})`);
}
} else {
console.log(`✓ 已写入标记到文章 #${id}:`);
console.log(` ${result.key} = ${result.value} (updated: ${fmtTime(result.updatedAt)})`);
}
}
async function cmdRemove(cfg, positional) {
if (positional.length < 2) {
console.error('用法: node scripts/marks.js remove <id> <key>');
process.exit(1);
}
const id = positional[0];
const key = positional[1];
validateId(id);
if (!key) {
console.error('✗ key 不能为空');
process.exit(1);
}
await apiRequest(cfg, 'DELETE', `/articles/${id}/marks/${encodeURIComponent(key)}`);
console.log(`✓ 已删除文章 #${id} 的标记 "${key}"`);
}
async function cmdList(cfg, positional) {
if (positional.length < 1) {
console.error('用法: node scripts/marks.js list <id>');
process.exit(1);
}
const id = positional[0];
validateId(id);
const result = await apiRequest(cfg, 'GET', `/articles/${id}/marks`);
const marks = result.marks || [];
if (marks.length === 0) {
console.log(`文章 #${id} 无标记`);
return;
}
console.log(`文章 #${id} 的标记 (${marks.length}):`);
for (const m of marks) {
console.log(` ${m.key} = ${m.value} (updated: ${fmtTime(m.updatedAt)})`);
}
}
async function cmdKeys(cfg) {
const result = await apiRequest(cfg, 'GET', '/marks/keys');
const keys = Array.isArray(result) ? result : [];
if (keys.length === 0) {
console.log('当前 App 无标记键');
return;
}
console.log(`标记键列表 (${keys.length}):`);
for (const k of keys) {
console.log(` ${k.key} (${k.count} 篇)`);
}
}
async function cmdValues(cfg, positional) {
if (positional.length < 1) {
console.error('用法: node scripts/marks.js values <key>');
process.exit(1);
}
const key = positional[0];
const result = await apiRequest(cfg, 'GET', '/marks/values', {
query: { key },
});
const values = Array.isArray(result) ? result : [];
if (values.length === 0) {
console.log(`标记键 "${key}" 下无值`);
return;
}
console.log(`标记键 "${key}" 的值 (${values.length}):`);
for (const v of values) {
console.log(` ${v}`);
}
}
async function cmdMarked(cfg, flags) {
const query = {};
if (flags.marks) {
query.marks = flags.marks;
} else if (flags.key) {
query.key = flags.key;
if (flags.value) query.value = flags.value;
} else {
console.error('用法: node scripts/marks.js marked --marks <csv> | --key <k> [--value <v>]');
process.exit(1);
}
query.page = flags.page ? parseInt(flags.page, 10) : 1;
const pageSize = flags['page-size']
? parseInt(flags['page-size'], 10)
: (cfg.api && cfg.api.defaultPageSize) || 20;
query.pageSize = isNaN(pageSize) ? 20 : pageSize;
const resp = await apiRequest(cfg, 'GET', '/articles/marked', { query });
if (flags.json) {
process.stdout.write(JSON.stringify(resp, null, 2) + '\n');
return;
}
const { total, page, pageSize: ps, items } = resp;
console.log(`\n共 ${total} 篇 | 第 ${page} 页 | 每页 ${ps} 条\n`);
if (!items || items.length === 0) {
console.log('(无匹配结果)');
return;
}
for (const item of items) {
const title = truncate(item.titleZh || item.titleEn || '(无标题)', 50);
const src = truncate(item.source || '', 16);
const cat = truncate(item.category || '', 10);
const time = fmtTime(item.publishedAt);
console.log(` #${item.articleId} ${title}`);
console.log(` ${src} | ${cat} | ${time}`);
if (item.summaryOverview) {
console.log(` ${truncate(item.summaryOverview, 80)}`);
}
console.log();
}
}
// ---------------------------------------------------------------------------
// Utils
// ---------------------------------------------------------------------------
function validateId(id) {
if (!/^\d+$/.test(String(id))) {
console.error(`✗ 文章 ID 必须是正整数,得到: ${id}`);
process.exit(1);
}
}
// ---------------------------------------------------------------------------
// Main dispatch
// ---------------------------------------------------------------------------
async function main() {
const { flags, positional } = parseArgs(process.argv.slice(2));
const subcommand = positional[0];
const rest = positional.slice(1);
if (!subcommand || subcommand === 'help' || subcommand === '--help' || subcommand === '-h') {
console.log(HELP);
return;
}
const cfg = loadConfig();
requireApi(cfg);
switch (subcommand) {
case 'add':
await cmdAdd(cfg, rest, flags);
break;
case 'remove':
case 'rm':
await cmdRemove(cfg, rest);
break;
case 'list':
case 'ls':
await cmdList(cfg, rest);
break;
case 'keys':
await cmdKeys(cfg);
break;
case 'values':
await cmdValues(cfg, rest);
break;
case 'marked':
await cmdMarked(cfg, flags);
break;
default:
console.error(`✗ 未知子命令: ${subcommand}\n`);
console.log(HELP);
process.exit(1);
}
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* search.js — Article search & hot-topic discovery (main entry point).
*
* Usage:
* node scripts/search.js [options]
*
* Options:
* --q <keyword> 全文搜索关键词
* --source <csv> 数据源过滤 (OR)
* --category <csv> 分类过滤 (OR)
* --tags <csv> 标签过滤 (OR)
* --days <N> 便捷: publishedFrom = now - N 天
* --published-from <RFC3339> 发布时间下界
* --published-to <RFC3339> 发布时间上界
* --harvest-from <YYYYMMDD> 采集日期下界
* --harvest-to <YYYYMMDD> 采集日期上界
* --translation-status <csv> 翻译状态过滤 (0/1/2)
* --marks <csv> 标记过滤 key:value,key2:value2 (AND)
* --sort <mode> relevance | published_desc | published_asc
* --page <N> 页码 (默认 1)
* --page-size <N> 每页条数 (默认 20, 上限 100)
* --json 输出原始 JSON (不格式化)
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
csvSplit,
parseMarksCsv,
fmtTime,
truncate,
} = require('./api-client');
function buildRequestBody(flags, cfg) {
const body = {};
if (flags.q) body.q = flags.q;
if (flags.source) body.source = csvSplit(flags.source);
if (flags.category) body.category = csvSplit(flags.category);
if (flags.tags) body.tags = csvSplit(flags.tags);
if (flags['translation-status']) {
body.translationStatus = csvSplit(flags['translation-status']).map((s) =>
parseInt(s, 10)
);
}
// --days convenience: compute publishedFrom
if (flags.days) {
const n = parseInt(flags.days, 10);
if (!isNaN(n) && n > 0) {
const from = new Date(Date.now() - n * 24 * 60 * 60 * 1000);
body.publishedFrom = from.toISOString().replace(/\.\d{3}Z$/, 'Z');
}
}
if (flags['published-from']) body.publishedFrom = flags['published-from'];
if (flags['published-to']) body.publishedTo = flags['published-to'];
if (flags['harvest-from']) body.harvestFrom = flags['harvest-from'];
if (flags['harvest-to']) body.harvestTo = flags['harvest-to'];
if (flags.marks) {
body.marks = parseMarksCsv(flags.marks);
}
body.sort = flags.sort || 'published_desc';
const pageSize = flags['page-size']
? parseInt(flags['page-size'], 10)
: (cfg.api && cfg.api.defaultPageSize) || 20;
body.page = flags.page ? parseInt(flags.page, 10) : 1;
body.pageSize = isNaN(pageSize) ? 20 : pageSize;
return body;
}
function printResults(resp) {
const { total, page, pageSize, items, facets } = resp;
console.log(`\n共 ${total} 篇 | 第 ${page} 页 | 每页 ${pageSize} 条\n`);
if (!items || items.length === 0) {
console.log('(无匹配结果)');
return;
}
// Results table
for (const item of items) {
const id = item.articleId;
const title = truncate(item.titleZh || item.titleEn || '(无标题)', 50);
const src = truncate(item.source || '', 16);
const cat = truncate(item.category || '', 10);
const time = fmtTime(item.publishedAt);
const tags = (item.tags || []).slice(0, 3).join('/');
const marksStr = (item.marks || [])
.map((m) => `${m.key}:${m.value || ''}`)
.join(' ');
console.log(` #${id} ${title}`);
console.log(
` ${src} | ${cat} | ${time}${tags ? ` | ${tags}` : ''}${
marksStr ? ` | [${marksStr}]` : ''
}`
);
if (item.summaryOverview) {
console.log(` ${truncate(item.summaryOverview, 80)}`);
}
console.log();
}
// Facets
if (facets) {
console.log('── 分面统计 ──');
printFacet('分类', facets.category);
printFacet('数据源', facets.source);
printFacet('标签', facets.tags);
printFacet('标记', facets.marks);
}
}
function printFacet(label, map) {
if (!map || Object.keys(map).length === 0) return;
const entries = Object.entries(map).sort((a, b) => b[1] - a[1]);
const top = entries.slice(0, 10);
const parts = top.map(([k, v]) => `${k}(${v})`).join(' ');
console.log(` ${label}: ${parts}`);
}
async function main() {
const { flags } = parseArgs(process.argv.slice(2));
const cfg = loadConfig();
requireApi(cfg);
const body = buildRequestBody(flags, cfg);
if (flags.json) {
const resp = await apiRequest(cfg, 'POST', '/articles/search', { body });
process.stdout.write(JSON.stringify(resp, null, 2) + '\n');
return;
}
// Log the query for context
const parts = [];
if (body.q) parts.push(`关键词="${body.q}"`);
if (body.source && body.source.length) parts.push(`源=${body.source.join(',')}`);
if (body.category && body.category.length) parts.push(`分类=${body.category.join(',')}`);
if (body.tags && body.tags.length) parts.push(`标签=${body.tags.join(',')}`);
if (body.marks && body.marks.length) parts.push(`标记=${body.marks.map((m) => `${m.key}:${m.value}`).join(',')}`);
if (body.publishedFrom) parts.push(`from=${body.publishedFrom}`);
if (body.publishedTo) parts.push(`to=${body.publishedTo}`);
if (body.harvestFrom) parts.push(`harvest=${body.harvestFrom}~${body.harvestTo || ''}`);
console.log(`搜索: ${parts.join(' | ') || '(全部)'}`);
const resp = await apiRequest(cfg, 'POST', '/articles/search', { body });
printResults(resp);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
'use strict';
/**
* tags.js — List all tags used across articles (read-only).
*
* Usage:
* node scripts/tags.js [--json]
*
* Tags are set at push time and cannot be modified after the fact.
* Use marks (marks.js) for post-hoc annotation.
*/
const {
loadConfig,
requireApi,
apiRequest,
parseArgs,
} = require('./api-client');
async function main() {
const { flags } = parseArgs(process.argv.slice(2));
const cfg = loadConfig();
requireApi(cfg);
const tags = await apiRequest(cfg, 'GET', '/articles/tags');
const list = Array.isArray(tags) ? tags : [];
if (flags.json) {
process.stdout.write(JSON.stringify(list, null, 2) + '\n');
return;
}
if (list.length === 0) {
console.log('系统中暂无标签');
return;
}
console.log(`\n标签列表 (${list.length}):\n`);
console.log(' 标签 文章数');
console.log(' ──────────────────────────────');
for (const t of list) {
console.log(` ${padRight(t.tag || '', 22)} ${t.count}`);
}
console.log();
console.log('提示: tags 在推送时设定,只读不可事后修改。');
console.log(' 如需事后标注,请使用 marks: node scripts/marks.js add <id> <key> [value]');
}
function padRight(s, len) {
s = String(s);
if (s.length >= len) return s;
return s + ' '.repeat(len - s.length);
}
main().catch((err) => {
console.error(`\n✗ ${err.message}`);
process.exit(1);
});
...@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) { ...@@ -189,10 +189,13 @@ function spawnBrowser(bin, opts) {
'--no-default-browser-check', '--no-default-browser-check',
'--disable-extensions', '--disable-extensions',
'--disable-gpu', '--disable-gpu',
// Hide the navigator.webdriver=true flag that CDP sets by default. // NOTE: --disable-blink-features=AutomationControlled was removed.
// Anti-bot systems (PerimeterX, Cloudflare) check this on page load. // It triggers an "unsupported command-line flag" warning banner on some
// Combined with the stealth injection in cdp-client.js openSession(). // Chromium builds (e.g. Arch Linux), which is itself a detectable artifact.
'--disable-blink-features=AutomationControlled', // navigator.webdriver suppression is handled entirely by the CDP injection
// in cdp-client.js openSession() via Page.addScriptToEvaluateOnNewDocument,
// which sets navigator.webdriver to undefined before any page script runs.
// That approach is flag-independent, more robust, and produces no banner.
// Let Chromium auto-detect Wayland vs X11 on Linux. Without this it // Let Chromium auto-detect Wayland vs X11 on Linux. Without this it
// defaults to the X11 backend and crashes with "Missing X server or // defaults to the X11 backend and crashes with "Missing X server or
// $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless // $DISPLAY" on Wayland-only sessions (common on Arch/GNOME/Sway). Harmless
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment