{
 "batch": "deep-27",
 "setup": "Hermes installed from source per RUN.md: git clone --depth 1 of github.com/NousResearch/hermes-agent, `uv sync --locked --python 3.14` (uv installed into test/scratch; Python 3.14.10 placed in test/scratch because the sandbox cannot read the default uv Python dir). `hermes plugins --help` lists install, validate and doctor. Each plugin fetched at its pinned sha from field.json and run with tryit.py (Nous's fake_llm_provider scripted model): empty plan first, then a 4-5 call plan including a bad input. Plugin tools are deferred behind tool_search/tool_call in this Hermes, so added_by_plugin is empty for all six; calling the tool names directly ran them. Plugin tests were run in separate venvs (Py 3.12 and 3.14) with Hermes off PYTHONPATH. This sandbox's /tmp is read-only, which explains crystallize's 17 test failures.",
 "plugins": [
  {
   "pr": 153647,
   "name": "riah-swipe",
   "validate": "passed (hermes plugins validate --json exit 0, every check ok) for folder hermes-plugin/",
   "inside": "Enabled and doctor OK: 4 tools (swipe_publish, swipe_status, swipe_results, swipe_wait), 2 hooks (pre_llm_call, pre_gateway_dispatch), /swipe command, `hermes swipe` CLI. swipe_publish with two text cards returned a deck id and a one-time 127.0.0.1:8137 pairing link; swipe_status and swipe_results returned the deck with 2 undecided cards; empty cards was rejected by schema (minItems); deck_id '../../etc/passwd' returned a clean 'invalid deck id' error.",
   "tests": "unittest (its documented runner): 307 ran, OK, 57 skipped on the second run; the first run had 1 failure that did not repeat (flaky)",
   "useful": 4,
   "reliable": 5,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 26,
   "rank": 1,
   "best": "A real human-in-the-loop review surface (local PWA board with pairing-cookie auth, phone LAN mode, Telegram inline Keep/Cut/Redo) that feeds verdicts back so Hermes regenerates what you sent back.",
   "worst": "Value depends on the user generating batches of options; the board is a local web server whose phone mode needs LAN setup, and one test was flaky.",
   "blocked": null
  },
  {
   "pr": 152243,
   "name": "morph",
   "validate": "passed (validate exit 0, all checks ok)",
   "inside": "NOT enabled: `hermes plugins enable morph` was refused in both runs ('venv: uv lock exited 2: Failed to generate package metadata for morph-hermes==1.0.0 ... Distribution not found'), because the repo root ships a buildable pyproject.toml (setuptools build-system, name morph-hermes, py-modules = []). Doctor alone reported 7 tools and 3 hooks registered, but inside the chat morph_discover and morph_workspace_state came back 'Tool does not exist' and the conversation ended on 'Model generated invalid tool call: morph_plan_workspace'. No call worked.",
   "tests": "pytest: 281 passed, 0 failed (run from a folder named morph-hermes; run from a folder named 'morph', collection fails with a circular import because the root __init__.py shadows the inner morph package)",
   "useful": 3,
   "reliable": 2,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 22,
   "rank": 4,
   "best": "Serious engineering: propose-then-human-approve writes with exact before/after, re-read verification, a sha256 hash-chained audit log, an SSRF netguard, tokens via Hermes secret_scope, a dashboard and 281 passing tests.",
   "worst": "Hermes refused to enable it at the pinned commit (its dev pyproject.toml breaks plugin dependency locking), so no tool ran; by reading, a dashboard request with no session is treated as the operator, so an agent with a terminal could approve its own write.",
   "blocked": null
  },
  {
   "pr": 154067,
   "name": "hermes-design-studio",
   "validate": "passed (validate exit 0, all checks ok)",
   "inside": "Enabled and doctor OK: 7 tools, a skill, and a Desktop page (desktop/plugin.js). design_studio_catalog listed the built-in shaders; design_studio_project new created a project (revision 0); design_studio_recipe list returned []; apply with project_id 'xyz' was rejected by schema pattern; design_studio_preview returned a clean error: 'No renderer response or cached preview. Open Hermes Desktop with the plugin enabled.' Rendering, export and shader checks need Desktop, so they were not run.",
   "tests": "pytest on Python 3.14: 354 passed, 1 failed (needs Hermes's plugins module); on 3.12 all 355 error out at collection because annotations rely on 3.14 deferred evaluation (method named list shadows the builtin)",
   "useful": 3,
   "reliable": 3,
   "polish": 3,
   "creative": 4,
   "beyond": 4,
   "total": 21,
   "rank": 6,
   "best": "A real non-destructive shader stack in Desktop. The agent edits the same revision-checked project as the UI and can author GLSL drafts, but only a human can approve them.",
   "worst": "Without an open Desktop renderer the agent can do little. The repo is buried in 232 evidence files and P1-P3 contract reports, and its README still says it is unpublished and that validation passed 'with CAUTION, not SAFE'.",
   "blocked": null
  },
  {
   "pr": 152544,
   "name": "crystallize",
   "validate": "passed (validate exit 0, all checks ok)",
   "inside": "Enabled and doctor OK: 4 tools, 8 hooks, llm_request and tool_execution middleware, a Desktop page and a dashboard API. crystal_list returned no routines; crystal_run on a bad id returned 'No routine with id ... Do the task yourself.'; crystal_full_result on a bad id returned a clean error; crystal_cheap refused with 'Crystallize Chat is off for this chat' (opt-in gate). No call crashed. The savings themselves need real traffic and a paid helper model, so they were not exercised.",
   "tests": "pytest on Py 3.14: 1438 passed, 17 failed, 2 skipped. All 17 failures come from runner.py hard-coding /tmp, which is read-only in this sandbox (9 direct OSErrors and 8 asserts on 'routine could not finish (OSError)').",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 3,
   "best": "It changes how Hermes itself spends tokens, through request and tool-execution middleware, result shrinking, re-read projection and cron skip-if-unchanged. Its huge test suite and a Desktop panel make it a plugin-only capability.",
   "worst": "Read in the code: crystal_run executes nothing (guarded dispatch is hard-coded off), some hooks are on by default (stream buffering, stripping its own tools), cron tool results are stored unredacted, and its runner hard-codes /tmp.",
   "blocked": null
  },
  {
   "pr": 154078,
   "name": "harness-story",
   "validate": "passed (validate exit 0, all checks ok)",
   "inside": "Enabled and doctor OK: 5 tools, 12 hooks, /story command, Desktop tab. Hermes warned about the unknown capability 'filesystem:write'. harness_story_generate (latest, use_llm false) built a story of the live session; inspect and render on it worked and wrote a JSON plus an animated HTML viewer; a bad session id gave a clean 'not found in state.db' error; generate with the default use_llm=true fell back to deterministic text because its default endpoint was unreachable.",
   "tests": "pytest: 5 passed, 1 skipped (thin for 12 hooks and 5 tools)",
   "useful": 3,
   "reliable": 3,
   "polish": 3,
   "creative": 4,
   "beyond": 4,
   "total": 21,
   "rank": 5,
   "best": "It turns the hooks and state.db into a readable causal story of a session, with a context-window breakdown and an HTML player that runs fully offline.",
   "worst": "The narrative LLM defaults to the author's LAN address http://192.168.1.12:8080 rather than Hermes's own model, so by default session transcripts go to whatever host answers there; render also writes output_html to any path the agent passes.",
   "blocked": null
  },
  {
   "pr": 153113,
   "name": "standing-orders",
   "validate": "passed (validate exit 0, all checks ok; installs 5 pinned python deps incl. opencv-headless and onnxruntime)",
   "inside": "Enabled (dependencies resolved) and doctor OK as a platform plugin: 8 tools, a skill, CLI, slash commands, Desktop page. so_status reported the camera as 'not started' because the gateway had not started the camera engine; so_list_orders returned []; so_create_order without a camera returned a clear 'no reference frame ... or pass reference_frame_path' error; so_arm on a bad id returned 'no such order'. No webcam here, so arming and firing were judged only by reading the code and the 170 tests.",
   "tests": "pytest: 170 passed, 2 skipped",
   "useful": 3,
   "reliable": 4,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 26,
   "rank": 2,
   "best": "Plain-English camera triggers: a local cascade (change gate, embedder, rules) wakes the agent only on uncertain frames, within an hourly vision budget, with backtest, corrections and a clip as evidence. It is new for agents and a real platform that only a plugin could add.",
   "worst": "It needs a webcam and a running gateway, so it serves a narrower audience. Arming 'only after the user approved' rests on the tool description rather than an enforced gate.",
   "blocked": null
  }
 ]
}