- 30 Oct, 2026 7 commits
-
-
郭阔 authored
1440 宽下逐源表过宽把右侧栏顶出屏幕(用户看到'不展示')。去掉左右 flex 分栏改为纵向堆叠,失败占比列固定 45% 宽。latest 顶部分页文案移除。
-
郭阔 authored
-
郭阔 authored
原先每源只显示最新一场,补跑+定时连跑时前一场被顶下,用户误以为覆盖。 现同源多场并列(按运行时间倒序,上限80张),卡片带运行类型(续采/修复); 文章明细提取拆为 _articles_from_data 纯函数供多场复用。
-
郭阔 authored
-
郭阔 authored
替换 flex 拼柱:细柱大缝、刻度挤、数字悬空等问题。SVG viewBox 坐标系内 预算几何,5 条基线+刻度右对齐,柱顶总量标签,日期标签随柱,悬浮整组高亮 并显示站点明细;配色降饱和;图例移至标题行右侧带 7 天合计。
-
郭阔 authored
align-items:flex-end 使柱体塌缩为内容高度,分段 height:% 无参照全为 0。 改为 stretch+column-reverse,高度按向上取整的轴最大值归一; 补 25/50/75/100% 网格线与数值刻度,顶部留白与刻度对齐。
-
郭阔 authored
- KPI 卡:今日文章/今日场次(成/部/败)/7天文章/7天场次/7天成本/当前运行 - 每日产出堆叠柱状图(按站点着色,悬浮看站点明细)+图例 - 逐源表现表(今日/7天文章、场次✓
⚠ ✗、最近运行状态) - 失败原因 Top(7天,按类型计数+比例条) - 数据一次遍历运行 JSON 聚合,5 分钟自动刷新;导航置于栏目列表之后
-
- 29 Oct, 2026 3 commits
-
-
郭阔 authored
-
郭阔 authored
- 顶部一行主操作:
▶ 手动触发 /⚙ 编辑配置 /⏸ 停用(⏵启用按技能状态切换) / ←返回列表; 原来散在页面底部的次要操作上移,不再上下找按钮 - 续采/采集修复两个表单(各带一个 provider 下拉)合并为'⚙ 高级恢复'折叠区: 单表单单选模式+共享 provider,去除重复控件;说明文字内置 - 头部徽标修复 WARNING 黄色(原误显红) -
郭阔 authored
bloomberg-technology 03:00 实例:glm 在交付完成后 api_error(exit=1), attempt3 磁盘重建 20 篇成功,但归一化仍按非零进程退出码判 RUNTIME_ERROR 失败。重建路径现将 RC 置 0——成果已磁盘核验,进程尾错不再构成业务失败。 该记录已重判为 SUCCESS(20/20 已推送)。
-
- 28 Oct, 2026 10 commits
-
-
郭阔 authored
点徽标弹浮层(成功数量+标题 / 失败数量+原因),Esc/点遮罩/✕ 关闭, 不再把表格向下撑开;数据块随页面渲染,无需请求。排程编辑抽屉保持不变。
-
郭阔 authored
- 徽标(绿/黄/红)带 ▾ 可点击,行下展开:成功 N 篇(标题链接原文)+ 失败 M 篇(标题+[失败类型] 原因) - 抽屉互斥(排程/详情同时只开一个);数据复用 _latest_run_articles 助手 (最近抓取页共用);修复 helper 误插装饰器与函数之间导致的 500
-
郭阔 authored
盲区:取尽+部分交付(如 reuters-technology 4成/5非文章拒绝)被取尽豁免整体 判 SUCCESS 绿色。现只要 succeeded>0 且 failed>0 → WARNING(部分成功),与是否 取尽无关;纯'取尽+0交付+全非文章'仍为正常成功。 回归矩阵 5 用例全过;今日 world(17/1)、business(17/3)、technology(4/5) 三场已重判为黄色并同步 last.log。
-
郭阔 authored
实测 reuters-technology 15:30 与 markets 12:00 两场:归档/摘要/finalize 全部 完成,但会话以'避免未授权的外部发布'为由跳过自动推送(模型主观拒判)。 现改为:运行结束时有交付而 day 目录 syncstate 未在本次窗口内更新 → 脚本直接补推(push.js 幂等);摘要未齐等推送前置不满足时留待下次自动重检。 本次 8 篇已手工补推并经服务器 unchanged 确认在库。
-
郭阔 authored
runtime 类失败(结构化交付失败)时,采集成果其实已全部落盘(registry+归档): 新增 reconstruct-result.py 按运行窗口内的 registry 条目重建 structured_output (归档文件逐篇核验),注入原信封——秒级完成、零 token,跳过备用 provider 重跑。 窗口内无交付(business 失败/真没干成活)自动落回原 attempt3 重跑路径。 实测:ft 14:25 窗口重建 40 篇核验通过;空窗口 exit 3 正确回退。
-
郭阔 authored
-
郭阔 authored
- 抽屉新增"手动时间"输入框(HH:MM 逗号分隔,回填当前值),走 /schedule-time - 服务端硬校验:与其他源任一时间或自身多次间隔不足全局最小间隔(24h 回环绕算) → 拒绝保存并列出全部冲突明细(与谁、差多少),不弹确认框 - 实测:02:10 提交被拦并列出与 business 02:00/markets 02:30 冲突,排程未变; 无冲突时间正常保存
-
郭阔 authored
- bin/wall-audit.sh <site> [--compare]:Cookie 清单(datadome 到期)/登录态/ 墙状态/拦截请求网络证据/出口 IP/系统代理/VPN 网卡,快照存 logs/<site>.wall-audit/ - keepalive 撞墙(exit 3)自动抓一份现场快照 - 首份基线:datadome cookie 有效期 365 天(非隔夜过期),出口 IP 与昨日截图不同
-
郭阔 authored
- harvest.sh 运行前检查 .wall-<port>:熔断期内同站源直接跳过(不碰浏览器/ 不耗 API),last.log 记'
⏸ 站点熔断';12h 过期自动解除 - 触发条件:本次 FAILED 且 0 交付 + 反爬特征(≥3 篇 EMPTY_BODY/CAPTCHA/ LOGIN_REQUIRED,或 0 尝试整场拒绝) - keepalive.sh 每小时确认站点正常(wall=false)即删除熔断标记,自动恢复采集 - 调度中心熔断状态显示灰色徽标(区别于红失败) - 实测:昨夜 reuters-world 墙记录判定触发;模拟熔断标记后 nikkei-world 1 秒跳过、零副作用 -
郭阔 authored
-
- 27 Oct, 2026 11 commits
-
-
郭阔 authored
时间表页面本身就展示全部排程,flash 里重复贴排程计划+应用日志冗长; 成功路径只报"已重排,详见下表",失败路径才附输出末尾供排查 (intent/gap/plan-apply 三处一致)
-
郭阔 authored
- harvest-result.py: 要求N篇仅交付M篇且未取尽时,只要有文章成功入库(succeeded>0) → status=WARNING/exit 0,消息进 warnings;交付 0 篇仍判 FAILED - harvest.sh 报告:WARNING →
⚠ ️ 成功(部分交付)(邮件仍按成功分支发) - 面板全站支持黄色徽标:调度中心上次运行/源详情结果状态与历史/运行详情 - 已重判今日 ft 14:30 记录(14/14 交付误判失败→WARNING)并同步修正 last.log -
郭阔 authored
- 导航「添加栏目」后新增 /guide:markdown 实时渲染仓库文档(单一事实源,文档更新自动同步) - web-venv 补装 markdown(阿里镜像);缺库时降级显示原文并提示
-
郭阔 authored
-
郭阔 authored
launchd 最小 PATH 不含 /opt/homebrew/bin,keepalive 每小时触发但 4 次全部 exit 127(node: command not found)。对齐 run-web.sh 的 PATH 导出后, launchctl kickstart 实测:访问成功、无墙、实例正常回收。
-
郭阔 authored
- 导航「站点登录」后新增入口 /latest:每源一张可折叠卡片, 摘要行=源名+运行时间+✓n/✗m+已取尽标记,展开为文章标题(链接原文)+成功/失败徽标 - 数据取自各源最新运行 JSON 的业务结果(.result.articles),按运行时间倒序
-
郭阔 authored
- 抽屉改为分行布局:当前定时(chips)/采集频率/第N次逐行+实际时间列, 不再一行平铺;加收起按钮 - 回显:schedule_board 将实际定时按时段归类(与规划器同口径)对应到各次意图; 未配置时段意图的源按实际时间归类生成回显(所见即所排)
-
郭阔 authored
- 操作列由平铺的频率+6时段编辑器改为 ✎排程/
▶ 采集/⏸ 停用(⏵启用)/✕ 一行小按钮, 意图编辑器移到点击后行下方展开的抽屉,表格不再被撑宽 -▶ 采集=立即采集一次(/run),⏸ /⏵=调度与技能总开关(/toggle) - 时间轴上的 06:00 交付线竖线与图例说明移除(规划仍按交付线排,仅不再展示) -
郭阔 authored
- 队列目录 .queue → .queue.port-<port>:同站(同浏览器同账号)互斥, 异站并行(如 nikkei 9315 与 reuters 9408 同时跑,日常即两条流水线) - 站点端口在排队锁之前解析,锁粒度与 runlock 一致 - reuters-world 09:27 实跑验证重试链:attempt1 交付17篇+结构化交付失败 → attempt2 续会话确认 all_seen → 合并17篇并推送成功
-
郭阔 authored
昨夜 nikkei-markets(GLM 无工具会话)与 reuters×3(DataDome 墙)失败时, 模型都通过 structured_output 如实上报 status=FAILED/0交付——但 attempt_needs_retry 只认运行错误(is_error/结构化缺失),业务失败被当'质量失败'直接放弃,attempt2/3 一次没跑。 - attempt_needs_retry 新增 business 判据:status=FAILED 或 0 交付; 取尽豁免优先(all_seen/全非文章页虽自报 FAILED 但业务上是成功,不重跑) - business 失败不走 --resume 续会话(会话确实没干成活,续接只会复述失败), attempt2 同 provider 整场重跑,attempt3 仍切备用 provider - 实测矩阵:四失败信封→重试;成功交付→不重试;取尽→不重试
-
郭阔 authored
- bin/keepalive.sh <site>:解析站点端口/profile,采集进行中自动跳过, 状态写 logs/<site>.keepalive.json - LaunchAgent com.newsharvester.keepalive-reuters(StartInterval 3600,装于本机); 面板调度列表(platform_layer.list_schedules / nh_schedule_list)排除 keepalive*, 不显示为数据源 - 背景:DataDome 按风险评分拦 reuters(出口为数据中心 IP),保温访问维持 cookie 活性降低凌晨采集撞墙概率
-
- 26 Oct, 2026 9 commits
-
-
郭阔 authored
freq dropdown now spawns 第1次..第6次 window selectors (凌晨批次/上午/ 下午/晚间), each run placed independently — e.g. reuters-world 2×/day as overnight+afternoon → 02:00 + 12:00. Sources without explicit windows keep auto semantics (first run in the deadline batch + extras evenly across day_window); legacy string windows normalized. JS toggles run selectors with the freq value; intent persists as a runs list in schedule-plan.json.
-
郭阔 authored
schedule-intent/schedule-gap concatenated str / str (default_conf_dir returns str) → 500 on save; schedule_board already wrapped its copy.
-
郭阔 authored
- freq>=2 with a preferred window now means: first run in the overnight deadline batch + remaining runs spread in the window (reuters-world 2×/day+morning → 04:00 overnight + 07:00 morning) - global min gap is a dropdown on the schedule board (10..120 min), saved via /schedule-gap and immediately re-planned - applied: gap 30min, reuters-world 04:00/07:00
-
郭阔 authored
调度中心 rows now maintain 每日次数 + 时段偏好(自动晨间批次/上午/下午/ 晚间) via dropdowns; POST /schedule-intent persists to schedule-plan.json and immediately re-plans all times. Planner gains windows support: a source with a preferred window gets all its runs evenly placed inside 06:30-11:30 / 12:00-17:30 / 18:00-21:30, outside the morning batch; global gap/deadline math stays automatic. Applied: reuters-technology → morning (06:30), freeing an overnight slot (everything shifts 40min later, more slack before the 06:00 deadline).
-
郭阔 authored
- harvest.sh: global queue lock (runlocks/.queue, mkdir-atomic, stale-PID steal, 180min wait cap) — a source whose time arrives while another is harvesting now waits (phase 排队中) and auto-starts when the previous run exits; runlock is created before queueing so the panel shows queued sources; manual panel trigger no longer refused (it queues) - bin/plan_schedules.py + schedule-plan.json: frequency-driven planner — deadline/safety/min-gap/day-window/avoid all configurable; morning slots laid backward from deadline-safety with exact gaps (wrapping to previous evening when the roster exceeds the night), freq>1 sources spread evenly across the day window, global gap resolution; --dry-run supported - 调度中心:
⚙ 按计划重排全部排程 button (POST /schedule-plan-apply) - applied: 12 sources — 3×4/day (wsj/ft/digitimes-feed latest morning slots) + 9×1/day, 40-min spacing, last start 05:30 vs 06:00 deadline -
郭阔 authored
A fresh attempt2 re-fetched every article (minutes + dollars) just to re-deliver a result whose data was already archived and pushed. attempt2 now extracts session_id from the failed envelope and runs claude --resume with a short re-deliver prompt (no re-collection, no browser); heartbeat is not reset in resume mode. attempt3 (fallback provider) stays a fresh full run — cross-provider resume not trusted.
-
郭阔 authored
When attempt1 delivers data but fails structured reporting, the final attempt fast-exits all_seen with 0 articles — the record then showed nothing delivered, leaving the operator to guess. After the attempt loop, merge earlier attempts' delivered articles (URL-deduped, archive_path required) into the final envelope, recompute counts, and record each attempt's contribution in warnings. Single verdict, full visibility. Unit-tested: 0+3+2(1 dup) → 4 articles, per-attempt notes.
-
郭阔 authored
digitimes updates registry/index incrementally while still fetching, so a mid-fetch registry write was misread as 推送/收尾 and overrode the agent's own correct 采集中 heartbeat. Disk inference has now been wrong in both directions (stale when the agent forgets to flip,超前 on incremental writers) — trust the agent's heartbeat whenever present; disk evidence and neutral 运行中 remain fallbacks only.
-
郭阔 authored
On auto-retry, attempt2 resets the heartbeat to 采集中 and refetches, but attempt1's completion files (registry etc.) are newer than the run start — they overrode the fresh heartbeat and showed 推送/收尾 while the browser was actively fetching. Disk evidence must beat the heartbeat's own mtime to override; attempt resets naturally re-anchor the baseline.
-