{
 "batch": "deep-29",
 "setup": "Hermes cloned from github.com/NousResearch/hermes-agent (depth 1) and built with uv sync --locked --python 3.14 (uv and Python installed under test/scratch); each plugin run with tryit.py (validate, enable, doctor, then an in-process AIAgent chat against tests/fakes/fake_llm_provider.py). This Hermes defers plugin tools behind tool_search/tool_call, so plugin tools never appear in the first offered list ('added_by_plugin' is empty for all six); planned calls were made through tool_call by exact name and ran. Own tests run with the Hermes venv python (pytest/unittest), Hermes source off PYTHONPATH. Code also read in full by helper readers.",
 "plugins": [
  {
   "pr": 150346,
   "name": "construct",
   "validate": "passed (hermes plugins validate --install-deps: every check ok, exit 0)",
   "inside": "Loaded and enabled; doctor OK: 12 tools, 1 hook (plus /construct, a skill, Desktop page). construct_start {} listed the example menu; construct_start example=braun-sk4-knob made project 'knob' and reported missing browser/ffmpeg/freecad cleanly; bad project '../etc' refused with a clear id error; construct_command ['author'] refused (network model step). Plain tool_search call failed on my wrong args (queries must be an array).",
   "tests": "Hermes-plugin suites (test_hermes_plugin, test_hermes_netlogo): 48 ran, OK, 8 skipped; full 451-test suite not finished inside a 10-minute limit here (not judged a failure).",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 4,
   "total": 25,
   "rank": 4,
   "best": "A research-to-checked-3D pipeline (hologram in chat, joints, print, CAD/STEP, films) with a Desktop page, tight child-process sandboxing and refusals that held on every bad input I tried.",
   "worst": "Very large and niche, and by default it runs a headless browser, ffmpeg, FreeCAD or Blender on the machine without asking unless confirm_runs is switched on.",
   "blocked": null
  },
  {
   "pr": 152214,
   "name": "passkey-gate",
   "validate": "passed (hermes plugins validate --install-deps: every check ok, exit 0)",
   "inside": "Loaded and enabled; doctor OK: 3 tools (gate_challenge, gate_verify, gate_status), 3 hooks incl. pre_tool_call enforcement. gate_status reported origin not configured with next step; gate_challenge minted a challenge bound to the exact terminal command (url null plus a clear hint to run setup); 'grant access' refused (under 40 characters); gate_verify wait_seconds='abc' stopped by schema validation.",
   "tests": "pytest: 324 passed, 30 failed of 354; the failures I inspected are the plugin's own pending-challenge caps (10 per requester / 100 global) tripping because challenge state carries over between tests, so a suite-isolation fault, not crypto failures.",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 27,
   "rank": 1,
   "best": "Real WebAuthn approval of the exact action and tool call (py_webauthn verify, pinned key, counter checks, one-time consume) enforced in Hermes's pre_tool_call hook, which is exactly the human-approval gate Nous asks for.",
   "worst": "Full use needs Tailscale or your own HTTPS plus a phone, the README admits regex patterns can be routed around and direct config edits bypass its weakening checks, and 30 of its own tests fail from shared state.",
   "blocked": null
  },
  {
   "pr": 153861,
   "name": "summon",
   "validate": "passed (hermes plugins validate --install-deps: every check ok, exit 0)",
   "inside": "Loaded and enabled as a platform plugin (kind: platform, label X); doctor OK with 0 tools and 0 hooks, so no tool to call: its gateway adapter needs an X account or the author's relay key, which I lack, so the platform path was judged by reading.",
   "tests": "unittest: 32 ran, OK (relay e2e needs wrangler, not run).",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 3,
   "best": "A real X platform adapter that lets the owner tag their bot under any post and get their own Hermes (model, memory) answering in thread, with 'learn this' saving a thread as a skill the user approves.",
   "worst": "Replies post publicly with no approval step, trial mode routes threads and stores bot OAuth tokens on the author's own Cloudflare relay, a non-numeric daily budget crashes start-up, and the README's 'never @mentions anyone' is only enforced on the relay path.",
   "blocked": null
  },
  {
   "pr": 150873,
   "name": "skill-court",
   "validate": "passed (all checks ok, exit 0) with one warning: private core imports (catalog rule 9) in appeal.py: agent.skill_utils.parse_frontmatter and tools.skills_guard.",
   "inside": "Loaded and enabled; doctor OK: 0 tools, 5 hooks, llm_request middleware. skills_list ran through its transform hook normally; driven by CLI: 'hermes court list' (no skills quarantined), 'hermes court report' (no trials yet), 'hermes court trial' with no names answered 'Name skills to try, or pass --all.' Real trials need Docker and provider spend, not run.",
   "tests": "pytest: 130 passed, 0 failed.",
   "useful": 3,
   "reliable": 3,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 25,
   "rank": 5,
   "best": "Measures whether each agent-written skill actually helps by replaying tasks with and without it (sandbox check or blind pairwise judge) and hides harmful skills from the model through middleware and hooks, a change to Hermes's own skill system only a plugin can make.",
   "worst": "Leans on private Hermes internals the catalog check warns about, trial profiles are clones of the user's profile including keys, and auto mode spends and quarantines in the background without asking.",
   "blocked": null
  },
  {
   "pr": 150840,
   "name": "kata",
   "validate": "passed (hermes plugins validate --install-deps: every check ok, exit 0)",
   "inside": "Loaded and enabled; doctor OK: 1 tool (kata_list), 6 hooks, /kata and hermes kata CLI, dashboard tab. kata_list returned no katas with the approval note; an unknown kata tool was refused by Hermes; after the chat 'hermes kata list' showed Kata had observed the 3 tool calls, so its recording hooks work.",
   "tests": "pytest: 61 passed, 0 failed (README cites 50 and 61 in different places).",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 26,
   "rank": 2,
   "best": "Mines tool routines the agent repeats and, only after the user approves, compiles each into a new deterministic Hermes tool with argument re-classification, no retries of side-effect steps and quoted shell templating.",
   "worst": "Once approved, local-write routines run without asking each time, its read-only shell classifier has gaps (rg --pre, git diff --output), and it needs three conversations of history before it does anything visible.",
   "blocked": null
  },
  {
   "pr": 151968,
   "name": "tickover",
   "validate": "passed (all checks ok, exit 0; 6 requirements from pyproject).",
   "inside": "As shipped, 'hermes plugins enable tickover' was refused twice: uv lock could not build its pyproject member hermes-tickover ('Distribution not found' in Hermes's plugin-sources workspace), so its tools were not callable in that run; doctor alone passed (10 tools, 3 hooks). In a copy without pyproject.toml (deps from plugin.yaml) it enabled and tickover_selftest passed all 4 checks, a missing file and a non-allowlisted URL were refused clearly, tickover_fleet answered.",
   "tests": "pytest: 168 passed, 9 failed, 1 skipped; the failures I inspected are its 120-character source limit tripping on this machine's long temp paths.",
   "useful": 3,
   "reliable": 2,
   "polish": 4,
   "creative": 5,
   "beyond": 4,
   "total": 22,
   "rank": 6,
   "best": "Gives Hermes ears for real hardware: hear a clip, verify an action by before/after sound, enroll a machine and catch drift, with strong SSRF and cron-injection protection and a selftest that needs no recordings.",
   "worst": "Hermes refused to enable it as shipped because its pyproject dependency member failed to build, its first real use downloads an 87 MB model, and watch/check can schedule cron jobs and message a repairer without per-call approval.",
   "blocked": null
  }
 ]
}