All Bash scripts live in `bin/`: `harvest.sh` performs one source fetch, `daily-digest.sh` builds the daily report, and `provider.sh` is a shared library; `add-source.sh` and `modify-source.sh` manage per-source timers and conf, while `migrate.sh` and `deploy.sh` manage installation and deployment (repo-level tools, not copied into the runtime directory). Keep systemd unit and timer definitions in `systemd/`, and keep only redacted configuration examples in `config/`. The optional Flask administration UI lives in `web/app.py`; its Jinja templates belong in `web/templates/`.
The deployed layout matters: runtime scripts (everything in `bin/` except `deploy.sh`/`migrate.sh`) and the web app are copied to `~/Documents/News-Harvester/bin/` and `~/Documents/News-Harvester/web/`, while systemd units and live configuration reside under `/etc/systemd/system/` and `/etc/news-harvester/`.
## Build, Test, and Development Commands
There is no build step or automated test suite. Before committing script changes, run:
```bash
bash -n bin/*.sh
python3 -m py_compile web/app.py
```
On an Arch Linux server, install the optional panel dependency with `sudo pacman -S python-flask`. For an end-to-end smoke test after installation, run `sudo systemctl start news-harvester@nikkei-world.service` and inspect its journal. Use `./bin/deploy.sh <host> --no-systemd` only when the change does not require updating units.
## Coding Style & Naming Conventions
Write Bash for Bash (`#!/bin/bash`), use `set -euo pipefail` for mutating scripts, quote expansions, and expose installation paths through the existing `${VAR:-default}` pattern. `status.sh` intentionally tolerates read-only command failures; preserve that behavior. Use lowercase, hyphenated source names such as `nikkei-world`; the management scripts accept only letters, digits, `_`, and `-`.
Use four spaces and `snake_case` in Python. Keep Flask route handlers small, validate source/provider input before invoking subprocesses, and place UI changes in the matching template. Preserve existing systemd naming: `news-harvester@<source>.timer` pairs with the template service.
## Testing & Change Validation
No coverage threshold is configured. Run the syntax checks above for every change. For timer or configuration changes, validate with `systemctl daemon-reload` and `systemctl list-timers`; for UI changes, start the panel in a configured server environment and exercise the changed route. Record the command and result in the pull request.
## Commits, Pull Requests & Security
Existing history uses concise Conventional Commit-style subjects, e.g. `feat: add provider support`; use `fix:`, `feat:`, or `docs:` with an imperative summary. Keep pull requests focused, explain any deployment-path or systemd impact, link relevant issues, and include screenshots for template/UI changes.
Never commit live provider JSON, SMTP credentials, passwords, logs, or generated output. Add sanitized `*.example` files instead and keep secret files permission-restricted (typically `600`).
((if (.result|type)=="object" then [.result.failures[]?.type] else [] end) | unique) as $types |
((if (.result|type)=="object" then [.result.failures[]?.message] else [] end) | map(select(length>0)) | join("；")) as $messages |
(if .status=="PARSE_ERROR" or ($types|index("INVALID_OUTPUT")) then "OUTPUT_ERROR"
elif (($types|index("LOGIN_REQUIRED")) or ($types|index("PAYWALL")) or ($types|index("CAPTCHA")) or ($types|index("BODY_TOO_SHORT")) or ($types|index("CONTENT_INCOMPLETE"))) then "ACCESS_STATE"
elif ($types|index("ARCHIVE_FAILED")) then "ARCHIVE_ERROR"
elif ($types|index("PUSH_FAILED")) then "PUSH_ERROR"
elif (($types|index("RUNTIME_ERROR")) and (.max_budget_usd>0) and (.cost >= .max_budget_usd*0.95)) then "BUDGET_EXCEEDED"
elif ($types|index("RUNTIME_ERROR")) then "PROVIDER_ERROR"
{% set archive = article.archive if article.archive is defined else {} %}
<tr>
<td>{% if article.url.startswith('https://') or article.url.startswith('http://') %}<aclass="source-link"href="{{ article.url }}"target="_blank"rel="noopener noreferrer">{{ article.title or article.url }}</a>{% else %}{{ article.title }}{% endif %}
<pstyle="color:#888;font-size:12px;margin:6px 0 0">归档：{{ article.archive_path or '未归档' }}</p></td>
<td><spanclass="badge {{ 'ok' if article.status == 'SUCCESS' else 'fail' }}">{{ '成功' if article.status == 'SUCCESS' else '失败' }}</span></td>
<td>{% if archive.get('body_length') is not none %}{{ archive.body_length }} 字符{% elif article.body_length is defined %}{{ article.body_length }} 字符{% else %}—{% endif %}</td>
<td>{{ article.failure_type or '—' }}{% if article.failure_reason %}<br>{{ article.failure_reason }}{% endif %}{% for evidence in article.quality_evidence %}{% if evidence != article.failure_reason %}<br>{{ evidence }}{% endif %}{% endfor %}</td>
