INDEPENDENT PROJECT · EXECUTION 0.1.8

Agent GUI Runtime

A released Windows execution runtime for MCP-compatible agents, with visual grounding, evidence-bound actions, result verification, and confirmed updates with rollback.

Project context

Project status / type: INDEPENDENT PROJECT · EXECUTION 0.1.8
Date context: January 2026 — Present
Role: Independent developer

Problem and constraints

GUI agents need to distinguish intended actions, dispatched input and verified results.
Runtime maintenance must preserve task ownership and recover from interrupted updates.

Personal contribution

Designed and implemented the Python/FastAPI runtime, authenticated local HTTP/MCP interfaces and structured execution receipts.
Implemented screenshot grounding, Decision checks and explicit main-agent review that continues the original job without replaying completed input.
Added an image-capable current-agent click adapter and a confirmed updater with installation locks, retained backups and recovery journals.

Architecture and workflow

PLAN → CAPTURE → GROUND TARGET → REVIEW → ACT → CHECK RESULT → CONTINUE / CALLER REVIEW
The API route uses configurable visual services. Current-agent mode delegates basic-click grounding and verification to the connected image-capable agent.
Stage an exact release, confirm its digest, acquire maintenance authority, apply and health-check; retain recovery state on failure.

Implementation decisions

Bind input authority to fresh screenshots and one original command; unknown input is never replayed.
Keep read-only next-step grounding separate from permission to dispatch the next input.
Release Runtime and the optional Amadeus HTTP MCP + Skill extension independently; the extension remains 0.0.4.

Technology map

Runtime & interface

Python
FastAPI
MCP

Visual grounding & review

Computer vision
OpenAI API

Verification & observability

pytest
Structured receipts
Diagnostic logging

Verification and evidence

Execution 0.1.8 is released with exact-target update confirmation, task-aware maintenance locks and rollback.
Current-agent mode completed 20 live 7-Zip steps; installed 0.1.7 separately completed a reviewed Calculator workflow.
The 0.1.8 development regression group recorded 502 passes and two skips; overlapping follow-up checks are reported separately.
0.1.8: 502 development regression checks passed with two skips; temporary install, update, rollback, process-interruption recovery and uninstall checks are documented.
0.1.7: 20 live 7-Zip steps on one connection, plus a separate installed Calculator C → 1 → + → 2 → = workflow; cancellation and cleanup controls passed.
0.1.6: five-step direct Runtime and four-step Amadeus/Pi recovery workflows completed after explicit main-agent review.
pytest regression checks and before/after screenshot receipts support diagnosis without equating input dispatch with task success.

Limitations and unfinished work

Scoped checks do not establish arbitrary-task, unfamiliar-application, or unattended reliability.
A model can incorrectly accept or reject evidence; secondary review is not an accuracy guarantee.
Current-agent mode excludes text entry, keys, scrolling, task plans and administrator mode; those capabilities belong to other supported routes.
The installer is unsigned. Physical power-loss recovery and clean-machine dependency installation were not newly certified.
Learning is paused. Amadeus Host update PR #261 is pending merge and is not installed by Runtime 0.1.8.

Lessons and next steps

Keep execution facts, permissions and verification separate.
Release evidence should identify the exact version, route and tested workflow.