Autopilot

Live

Signature capability

A path/host/endpoint trading firewall proven to hold at two independent levels: 187 red-team bypass attacks each closing a discovered hole at the unit boundary, plus a 77-case safety-through-loop suite proving that even when the model itself is tricked into emitting an order, zero effectors fire and subprocess is never invoked.

What it is

Autopilot is a Python (3.11+, Click) CLI agent for macOS built as a perceive-reason-act loop. It perceives the screen by fusing the PyObjC Accessibility (AXUIElement) tree, Retina-aware screencapture, and Vision-framework OCR into a single ScreenModel, then reasons by shelling out to the `claude -p` CLI (subscription OAuth, no API key carried) as a stream-json driver. It acts through a tiered surface router that prefers no-touch executors (background Accessibility, AppleScript, headless harnesses) over moving the cursor, only falling back to vision-based clicks last. It self-heals with bounded retries, logs every step to replayable JSONL, and compiles successful runs into deterministic AX/AppleScript skills that replay with no model and no vision. Autonomous lanes add cron triggers, wake-word voice, and a fleet conductor that fans a goal across the Tailscale mesh. Every effector, surface, loop, and replayed skill routes through one authorization gate that is hard-blocked from live trading.

Highlights

Trading firewall is the keystone: a 296-line gate + 797-line rule registry that classifies by path/host/endpoint (not intent), fails closed, and uses a single-use phrase-exact override token a casual 'please' cannot unlock
Firewall proven end-to-end, not just at the unit boundary: 187 parametrized red-team attacks (order place/cancel, key access, writes to capital.json/.env/SHARED_STATE, base64|sh and subshell obfuscation) plus a 77-case safety-through-loop suite where the decisive test runs the real Gate and proves an order to quantpod is HARD_BLOCKed and never reaches ssh
~99.5% coverage with the safety module at 100% line and branch (562 stmts, 0 missed) against a 95% strict bar; ~2,200 test functions across safety, core, perceive, act, skills, cli, fleet including an 81-case Hypothesis property suite proving the classifier is total
Two-lane design: a direct computer-use lane and an autonomous CLI lane with checkpoint/resume, an in-repo 5-field cron matcher (no croniter dependency), file/time/process/health triggers, and a poll-thread scheduler with in-flight overlap guards
Surface router never touches the foreground when a no-touch surface serves; H2 self-improvement compiles audit logs into deterministic replayable skills, and vision-only runs on unharnessed apps emit a cli-anything harness stub
Fleet conductor decomposes a goal into per-host subgoals fail-closed (no blind fan-out) and gates every SSH hop through the same M1 firewall, adding no new authorization path

Tech · Python 3.11+, Click, PyObjC (AppKit/Quartz/Vision/ApplicationServices), macOS Accessibility API, Vision-framework OCR, claude -p CLI (subscription OAuth), MCP server, Tailscale SSH mesh, pytest + pytest-cov + Hypothesis, pvporcupine + Whisper (optional voice)