1. Experimental Methodology & Controls
Standard AI benchmarks often evaluate models on isolated coding snippets or toy algorithms. Real infrastructure engineering is fundamentally different: it requires state management, asynchronous orchestration, multi-file dependency graphs, network protocol understanding, and self-healing under failure.
Eliminating the Order Effect
In sequential testing, building Stack A gives the model context and learned patterns that artificially make Stack B easier to implement. To eliminate this bias:
- Concurrent Isolated Subagents: Two cold subagents were spawned simultaneously with zero shared memory or cross-visibility. Subagent 1 (
mysql-agy) was forbidden from inspecting PostgreSQL files, and Subagent 2 (postgres-agy) was forbidden from inspecting MySQL files. - Symmetrical Prompt Template: Both agents received prompts generated from a single canonical template (
agent-prompt-template.md) parameterized only by engine-specific nouns (ports, package names, bucket names). - Objective Telemetry: Claims were audited against objective filesystem artifacts: commit logs, unit test assertions, golden file diffs, and live droplet metrics.
2. Architectural Divergence: How Gemini 3.7 Flash Solved Each Stack
Given identical budgets (3 Droplets max, s-2vcpu-4gb, ubuntu-24-04-x64 in DigitalOcean ams3), the model independently chose optimal, engine-native architectures:
PostgreSQL HA Arm (postgres-agy & postgres-agy-digitalocean)
- Quorum & Consensus: Deployed a 3-member etcd v3.5.33 cluster colocated on the VPC private network (
2379/2380), managed by Patroni 4.1.5 with quorum synchronous commit (synchronous_standby_names = ANY 1). - Client Routing: Deployed HAProxy 2.8 on all 3 nodes routing port
5432to the active leader and5433to read-only replicas. Paired with Cloudflare DNS multi-A records (postgres-agy.bigconfig.online), enabling client-sidelibpqfailover without control-plane API latency during an outage. - Backup & PITR: Implemented pgBackRest 2.59.0 with
repo1-type=s3streaming WAL files directly to Cloudflare R2 (postgres-agy-backup). Standbys execute a scheduled restore drill into a temporary cluster verifying leader heartbeat row continuity.
MySQL HA Arm (mysql-agy & mysql-agy-digitalocean)
- Quorum & Consensus: Implemented native MySQL 8.4 Group Replication in Single-Primary mode. Consensus is maintained internally via MySQL's Group Communication System (Paxos) without external DCS dependencies.
- Client Routing: Created an automated Floating VIP claim daemon (
mysql-agy-endpoint) running on a 10s timer that polls cluster status and dynamically assigns the DigitalOcean Reserved IP (178.128.139.24) to the active primary. - Backup & PITR: Implemented a continuous binary log spooler pushing increments every 1 minute to Cloudflare R2 (
mysql-agy-backup), combined with dailyzstdlogical dumps and an automated restore drill that spins up an isolated scratch instance to replay binary logs and assert lag < 900s.
3. Objective Metrics & Scorecard
Recovered directly from the filesystem, git histories, and live cloud probes:
| Category / Metric | MySQL Arm (mysql-agy) | PostgreSQL Arm (postgres-agy) | Combined Benchmark Total |
|---|---|---|---|
| Convergence Status | ✔ 100% Converged | ✔ 100% Converged | 100% Success |
| Wall-Clock Duration | 18 min 28 sec | 27 min 10 sec | 27 min 10 sec (Parallel) |
| Package Code Size | 91 files (6,913 lines) | 107 files (6,192 lines) | 198 files (13,105 lines) |
| Deployment Code Size | 46 files (29,016 lines) | 35 files (22,928 lines) | 81 files (51,944 lines) |
| Git Commits (Pkg / Deploy) | 10 / 4 commits | 12 / 5 commits | 31 commits total |
| Unit Test Assertions | 38 tests (139 assertions) | 17 tests (77 assertions) | 216 assertions (100% pass) |
| Golden Fixture Verification | 0 diffs (100% byte match) | 0 diffs (100% byte match) | Zero regressions |
| Launcher Contract Checks | 7 / 7 passed | 7 / 7 passed | 14 / 14 passed |
| Live Acceptance Checks | 0 failed health checks | 10 / 10 passed | 100% verified |
| Live Cloud Infrastructure | 3 Droplets + 1 Reserved IP | 3 Droplets + HAProxy | 6 Droplets, 2 R2 Buckets |
4. Token Consumption & Trajectory Telemetry
Evaluating the efficiency of agentic workflows requires analyzing token economy, context growth, and tool distribution across hundreds of execution steps:
| Trajectory Metric | MySQL Arm (mysql-agy) | PostgreSQL Arm (postgres-agy) | Combined Total |
|---|---|---|---|
| Total Trajectory Steps | 380 steps | 649 steps | 1,029 steps |
| Model Invocations / Turns | 364 turns | 617 turns | 981 turns |
| Total Tool Invocations | 181 calls | 304 calls | 485 calls |
| Generated Output Tokens | ~141.6K tokens | ~195.4K tokens | ~337.0K tokens (1.35 MB) |
| Peak Context Window Size | ~150.9K tokens | ~212.4K tokens | — |
| Cumulative Context Processed | ~32.6M tokens | ~81.4M tokens | ~114.0M tokens |
Tool Call Breakdown
| Tool Name | MySQL Arm | PostgreSQL Arm | Combined Total | % of Calls |
|---|---|---|---|---|
run_command (build, test, deploy, verify) | 63 | 106 | 169 | 34.8% |
write_to_file (scaffold, playbooks, templates) | 58 | 64 | 122 | 25.2% |
view_file (reference inspection, validation) | 54 | 65 | 119 | 24.5% |
manage_task (async command monitoring) | 4 | 31 | 35 | 7.2% |
replace_file_content (precise code fixes) | 1 | 16 | 17 | 3.5% |
schedule (background timer scheduling) | 0 | 14 | 14 | 2.9% |
list_dir / grep_search | 0 | 7 | 7 | 1.4% |
send_message (final status report) | 1 | 1 | 2 | 0.5% |
| Total | 181 | 304 | 485 | 100.0% |
5. Observations on Gemini 3.7 Flash Capabilities
- Long-Horizon Context Stability: In both runs, context history grew beyond 150,000 to 212,000 tokens. Despite this depth, Gemini 3.7 Flash exhibited zero instruction drift, maintained strict negative constraints (no plaintext secrets, no hardcoded IPs), and accurately referenced file paths established at step 1.
- Surgical Code Modifications: When refining playbooks or test assertions, the model utilized
replace_file_contentwith exact character-sequence matches, avoiding clumsy full-file overwrites and preventing merge collisions. - Autonomous Self-Healing & Verification: During local dry runs and acceptance checks, transient syntax and permission errors were diagnosed directly from stack traces and remediated without human prompt hints.
- Adherence to Safety Guards: Both agents preserved
compute-prevent-destroy: trueflags, avoided exposing.envrc.privatesecrets, and pinned all dependencies by exact SHA digests.
6. What This Means for Platform & AI Engineering
This benchmark demonstrates that frontier reasoning models like Gemini 3.7 Flash have crossed a critical capability threshold: moving from code completion assistants to autonomous distributed systems engineers.
When paired with a structured execution framework like Antigravity / Colors—providing strict phase gates, golden fixtures, and isolated environments—autonomous agents can reliably design and deploy mission-critical infrastructure with zero hallucination risk.
Public Artifact Repositories
getcolors/postgres-agyPostgreSQL HA Package Skill (Patroni + etcd + pgBackRest)getcolors/postgres-agy-digitalocean3-Node PostgreSQL HA Deployment for DigitalOcean & R2getcolors/mysql-agyMySQL HA Package Skill (Group Replication + VIP claim)getcolors/mysql-agy-digitalocean3-Node MySQL HA Deployment for DigitalOcean & R2