Portfolio — Case Study
AI Trading · Prediction MarketsAutonomous AI trading on the world's largest decentralised prediction market.
Live on Polymarket since 17 July 2026, trading real USDC on Polygon. 167,240 markets scored all-time, 404 whale wallets tracked, 30-minute scan cycles running continuously. Gemini Flash 2.5 evaluates each shortlisted market and returns a 9-field structured JSON decision — conviction, fair value, edge in basis points, reasoning, bear case, resolution risk, and position size — before the hardcoded 9-rule risk gate decides whether to execute. Every decision is explainable, every trade auditable, and the risk gate cannot be overridden by the LLM.
Gemini
Flash 2.5
AI Decision Layer
9
Hard Risk Rules
16
Audit Log Triggers
Live
Trading Status
The Challenge
Prediction markets price real-world events — elections, sports results, economic releases — but the prices are set by human sentiment and often diverge significantly from actual probability. The bot needed to identify genuine mispricing: where market price deviates from calculated expected value, execute positions within strict risk parameters, and operate in a fully auditable way where every single decision could be explained and reviewed.
Gemini Flash 2.5 was chosen as the AI decision layer for its ability to produce reliable structured JSON reasoning across complex probability scenarios at the scan frequency required. Every decision carries nine explicit fields — conviction score, estimated fair value, edge in basis points, reasoning, bear case, resolution risk, and more — so any trade can be opened and understood by a human reviewer at any time. Unstructured or black-box output was not acceptable.
The audit requirement was non-negotiable: every signal evaluated, every AI decision, and every trade placed had to be permanently and immutably recorded. Append-only SQLite triggers ensure the log cannot be altered after writing, providing a complete and tamper-resistant decision trail.
What Was Delivered
Technology Stack
AI
Trading
Data Signals
Backend
Interface
Risk
Observability
Key Features
The AI evaluates each shortlisted market and returns a structured JSON object with nine required fields: decision (BUY, SELL, or PASS), outcome (YES or NO), conviction (0.0–1.0), fair_value (estimated true probability, used for Brier score calibration), edge_bps (expected edge in basis points), reasoning (trade thesis), bear_case (strongest argument against), resolution_risk (LOW, MEDIUM, or HIGH), and size_usd (always clamped by the risk gate — the LLM suggestion is never trusted directly). If conviction falls below 0.70 or resolution_risk is HIGH, the LLM auto-sets its own decision to PASS. This is an LLM-layer override, not a risk gate check.
Nine hard-coded rules checked in this exact order before every order submission: (1) kill switch active, (2) daily loss exceeds the configured percentage of current bankroll — triggers full halt and cancels all open orders, (3) market price below 5% or above 95%, (4) resolves within 6 hours, (5) position size exceeds the configured percentage of bankroll (first-week cap also enforced here), (6) more than 10 concurrent positions open, (7) total exposure exceeds configured percentage of bankroll, (8) exit-side book depth below the minimum liquidity threshold, (9) fill would move price more than 2% from best. All thresholds are read from config at runtime — no hardcoded dollar amounts. Any single failure vetoes the trade and writes to risk_events with the full state snapshot.
smart_money.py tracks on-chain wallet concentration using Polymarket's leaderboard and top-holder data. Each wallet is weighted by its realised PnL on a log scale. Conviction is now measured against each wallet's own historical median position size: a whale taking an unusually large stake relative to their own baseline registers as a stronger signal than the same absolute size from a whale known for large positions. The whale_wallets table records each tracked wallet, its PnL, sample size, conviction score, and last refresh timestamp.
mispricing.py combines four sub-scores into a single directional mispricing signal. Cross-market basket arbitrage (70% weight) detects price deviations within neg-risk groups — mutually-exclusive outcomes whose probabilities must sum to 1.00 after the full CLOB fee and on-chain gas cost are deducted. This is the highest-trust edge source: arithmetic certainty, not sentiment. Stoikov microprice (15%) computes directional book pressure. Stale price (15%) measures hours since the last recorded trade. Book microstructure (0%) is retained but zeroed. combine.py merges both signals with a 0.4 floor; either at zero collapses the composite to zero.
16 SQLite BEFORE DELETE and BEFORE UPDATE triggers enforce append-only writes at the database level — not by application discipline. All 11 tables are covered: scans (every scored market, both signal scores, composite, component breakdown JSON), decisions (full LLM request and response, parsed decision, prompt version), orders (every order in shadow and live mode, idempotency key), fills, positions, resolutions (outcomes and realised PnL), risk_events (every veto and halt with state snapshot), exits, rotations, whale_wallets, and markets.
The bot has no web interface. It is operated entirely through a Python CLI: polymkt shadow runs the full decision cycle and logs intended trades without placing orders; polymkt live activates real order submission; polymkt report generates a session summary with Brier scores; polymkt kill cancels all open orders, writes a HALT record, and locks the system until an operator runs polymkt resume. Shadow and live records share the same 11 audit tables, distinguished only by the mode column. Transitioning from shadow to live is a single command — no code change required.
Three separate Polymarket APIs, each with its own HTTP client and independent rate-limit bucket at 5 req/s, burst 10. Gamma handles market discovery. CLOB handles order book data and order execution. Data handles on-chain holder distribution and leaderboard PnL. Because each client manages its own backpressure independently, a slow or throttled response from one API cannot block the others.
Every order gets a SHA-256 idempotency key — a hash of market ID, token ID, side, rounded price, and size — stored as orders.client_key. Before any order is submitted, the key is checked against the database. A duplicate key means the process crashed and restarted mid-submission; the order is not resubmitted, preventing double-fills. (decision_id was removed from the key after it caused the same intended trade appearing in two consecutive scan cycles to generate different keys, resulting in duplicate orders — the key must be stable across cycles for the same trade.) The kill switch (polymkt kill) cancels all open orders and writes a HALT record. The system refuses to enter live mode until an operator explicitly runs polymkt resume — no confirmation prompt, no automatic recovery.
297 automated tests — all passing — cover every critical path: all 9 risk gate limits at breach, pass, and boundary conditions; idempotency key collision handling; LLM response parsing for all 9 output fields; signal scoring edge cases; WebSocket event handling for all message types; resolution detection; and log redaction — verifying that private keys never appear in any log output. Calibration is measured by Brier score — mean((fair_value − outcome)²) across resolved positions — computed separately for shadow and live modes. A naive always-guess-50% baseline scores 0.25; lower is better. Win rate is not the primary metric.
When the bot opens a position, it subscribes to live price ticks for that market via a persistent WebSocket connection to the Polymarket CLOB (wss://ws-subscriptions-clob.polymarket.com/ws/market). Price updates arrive in real time — no REST polling for held positions. When a position closes, the subscription is cancelled automatically. This eliminates both latency and API cost for any market the bot is actively holding. Background recv tasks handle price_change, book, best_bid_ask, and market_resolved events with automatic reconnection on drop.
Once a position is open, a convergence take-profit rule runs every cycle. The position exits when either: (a) the remaining edge — fair value minus current market price — falls below 5¢, or (b) 90% of the original edge at entry has been captured. The previous 30% flat take-profit rule was retired because it was mathematically unachievable on high-probability markets — a market priced at 85¢ can never reach 110¢. Every exit records the rule that fired and whether the price came from the WebSocket cache or a REST fallback.
Wallets on the smart-money watchlist must have a minimum of 20 closed positions before they count toward any signal. Wallets with a known insufficient sample size are excluded at both watchlist refresh and scoring time — preventing a single lucky trade from inflating a wallet's credibility score and generating false conviction. The whale_wallets table records each wallet's closed position count alongside its PnL, so the sample-size floor is enforced at the data layer, not just at scoring time.
Every exit cycle computes remaining_edge = fair_value - market_price. The position closes when remaining edge falls below 5¢ (fair value is essentially captured) or when 90% of the original edge at entry has already been captured. The 5¢ absolute floor replaces the previous percentage-of-original calculation, which could produce an impossibly small threshold on already-converged positions. Exit orders carry an idempotency key, a pre-written pending row, and book-depth slippage estimation before the order is placed.
When a better opportunity appears while a position is held, the rotation rule fires if candidate_edge - incumbent_remaining_edge > round_trip_cost * 2.0. The threshold accounts for both the cost of exiting the current position and entering the new one, so rotation only happens when the new market is genuinely superior — not merely marginally better. Every rotation is logged to a dedicated rotations table with the candidate market, the edge delta, and the round-trip cost at time of decision.
If only one watchlist whale has taken a position, the smart_money_score is capped below the admission floor regardless of that wallet's individual conviction. At least two independent whale sources must be present before the signal can contribute to an entry decision. This prevents a single large wallet — even a historically accurate one — from unilaterally triggering a trade. The cap is applied at scoring time so it cannot be circumvented by adjusting individual wallet weights.
A signal scoring 0.6 or above on its own dimension can be admitted if at least one other independent signal provides 0.2 or more corroboration — even if no single dimension clears the full threshold alone. This allows a very strong primary signal to proceed with light confirmation rather than being blocked entirely by signal isolation. The exception is logged with the corroborating score so it can be audited separately from standard multi-signal admissions.
The fill-chain matcher identifies whether a trade was executed against a market-maker (quoting both sides simultaneously) or a directional participant. Classification is written to the fills table and is currently logged, not enforced — building the dataset to discover whether market-maker fills have different predictive value to directional fills. The infrastructure is in place to enforce a discount or bypass rule once the sample reaches statistical significance.
After each order, the fill-chain matcher queries recent CLOB trade history and attempts to match the system's fills to their counterparties. Matched fills are annotated with counterparty classification and stored against the order record. This feeds the market-maker dataset used by the classification engine and gives the review layer a richer picture of who the system is trading against on each market.
A structured observation snapshot is recorded after each trading decision — signals at the time, AI conviction, gate pass/fail, outcome, and Brier score delta. polymkt observe generates reports that surface calibration drift: markets where the model is systematically over- or under-confident, signal pairs that predict well together, and gate rules that fire disproportionately on winning or losing trades. Production data feeds directly back into signal weight tuning.
August 2026
The original system made auditable, rules-bound trading decisions. What has shipped since launch closes the loop: production data now feeds directly back into signal calibration, risk parameters are proportional rather than fixed, and three new analytical layers — basket arbitrage, fill-chain matching, and market-maker classification — give the review layer a materially richer picture of what the system is trading against and why.
The proportional risk gate was the most significant structural change: every limit previously expressed as a hardcoded dollar amount is now a percentage of current bankroll, read from config at runtime. The gate logic did not change; the brittleness did.
Multi-Model AI Workflow
Most AI systems commit to one model and route everything through it. This system assigns each job to the model best suited for that job — then has a different model check the output. No single model audits its own decisions.
Per-cycle trading decisions. Chosen for cost, speed, and reliable structured JSON output. Evaluates each shortlisted market every 30-minute scan cycle and returns a 9-field decision: conviction, fair value, edge in basis points, reasoning, bear case, resolution risk, and position size. Cheap enough to run at scan frequency; structured enough that output can be machine-validated before the risk gate sees it.
Batch review of Gemini's decisions, run out-of-band via polymkt review. Audits conviction accuracy, calibration drift, and systematic bias across resolved markets — things Gemini can't self-detect. Opus sees not just the trades taken but the trades considered and refused, giving it a complete picture of decision quality over time.
Architecture and code-safety review. This week it caught a live bug in the slippage gate that the primary reviewer missed: the CLOB /book API returns order-book levels in worst-first order, so bids[0] was silently reading the worst bid. The MAX_SLIPPAGE gate was passing every trade at 0% slippage because it walked the book backwards. A class-of-bug fix — one correction at the client boundary, not N fixes across every consumer.
One model per job, cross-checked by another. The architecture is model-agnostic by design — each layer can be swapped independently without affecting the others.
System Architecture
Market data enters from three independent Polymarket API surfaces, each with its own rate-limited HTTP client at 5 requests per second, burst 10. Gamma discovers active markets. CLOB provides order book depth for microstructure analysis and handles order execution. Data supplies on-chain holder distribution and leaderboard PnL for the smart money signal. Because each client has its own rate-limit bucket, backpressure on one API surface cannot starve the others. Every 30 minutes — configurable via the --interval CLI flag — the scan loop fetches fresh data and passes it to the signal engine.
Two independent signals score each market. smart_money.py weights on-chain wallet concentration by log-scale realised PnL. mispricing.py combines four sub-scores — cross-market arithmetic (70%), Stoikov microprice (15%), stale price (15%), and book microstructure (0%, zeroed) — into a single directional score. combine.py merges them into a weighted composite; both signals must independently clear a 0.4 floor and agree on direction — YES or NO — before the market is shortlisted. Either signal at zero collapses the composite to zero. Shortlisted markets go to Gemini Flash 2.5, which returns a 9-field JSON decision. That decision then passes through the 9-rule hard risk gate — checked synchronously, in order — before any order is submitted. Before submission, a SHA-256 idempotency key derived from market ID, token ID, side, rounded price, and size is checked against the orders table to prevent double-fills on process restart.
Decisions that pass the risk gate proceed to the trade executor: in live mode, an order is submitted via the CLOB API; in shadow mode, the intended trade is logged with mode=shadow and zero capital is at risk. Every step across the pipeline writes immutable records to the 11-table SQLite audit schema, enforced by 16 BEFORE DELETE / BEFORE UPDATE triggers at the database level. Brier score — mean((fair_value − outcome)²) across resolved positions — is the primary calibration metric, computed separately for shadow and live modes. A naive always-guess-50% baseline scores 0.25; lower is better. The system has no web interface.
Polymarket APIs (3 independent rate-limited clients: 5 req/s, burst 10)
├── Gamma API → market discovery, active market list
├── CLOB API → order book depth + order execution
└── Data API → on-chain holders + leaderboard PnL
│
▼ 30-min scan interval (--interval CLI flag, configurable)
Signal Engine
├── smart_money.py → wallet concentration, log-scale PnL weighting
├── mispricing.py → neg-risk arb + book microstructure + stale price
└── combine.py → weighted composite; both signals must clear 0.4
│ and agree on direction (YES or NO)
▼
Gemini Flash 2.5 (Decision Layer)
│ Output: { decision, outcome, conviction, fair_value,
│ edge_bps, reasoning, bear_case,
│ resolution_risk, size_usd }
│ Auto-PASS if conviction < 0.70 or resolution_risk = HIGH
▼
9-Rule Hard Risk Gate (checked in order, not configurable)
│ 1. Kill switch active? 6. Max concurrent positions (10)
│ 2. Daily loss ≥ loss_pct × bankroll? 7. Total exposure > exposure_pct × bankroll?
│ 3. Price out of 5–95%? 8. Exit liquidity < min_exit_liquidity_usd?
│ 4. Resolves in < 6 hours? 9. Slippage > 2%?
│ 5. Position size > pos_pct × bankroll?
│ FAIL → trade vetoed, written to risk_events
▼
Trade Executor
│ SHA-256 idempotency key checked before every submission
│ Live: CLOB API order │ Shadow: log only (mode = shadow)
▼
SQLite Audit (11 tables, 16 append-only BEFORE DELETE / BEFORE UPDATE triggers)
├── scans / decisions / orders / fills / positions
└── resolutions / risk_events / exits / rotations
whale_wallets / marketsPolymarket APIs
3 rate-limited clients — Gamma, CLOB, Data API
Signal Engine
Smart-money wallets + order-book mispricing; both signals must clear 0.4 and agree on direction
Gemini Flash 2.5
9-field JSON decision: conviction, fair value, edge bps, reasoning, bear case, resolution risk, size
9-Rule Hard Risk Gate
Kill switch · proportional daily loss · price range · expiry window · proportional size · concurrent positions · proportional exposure · liquidity · slippage
Trade Executor
SHA-256 idempotency key checked before every CLOB API submission
SQLite Audit
11 tables · 16 append-only triggers · every decision and trade recorded
Development Discipline
Every session begins the same way. Every change to execution code goes through the same quality gate. Every restart to live mode passes the same 10-item checklist. The rules encode decisions that were already made — so they don't need to be re-litigated each time a different session starts or a different AI is involved.
SOUL.md (non-negotiable, loaded at session start)
└── Capital preservation principles
No position > max_pos_pct of bankroll
Daily loss halt at daily_loss_pct × bankroll
Shadow soak before every live restart
.claude/rules/ (7 mandatory rule files)
├── api-surfaces.md → rate limits, client boundaries
├── risk-gate.md → gate rules, override prohibition
├── signal-layer.md → scoring, floor, dominant exception
├── execution-safety.md → idempotency, order hygiene
├── storage.md → append-only, migration protocol
├── docs.md → documentation requirements
└── testing.md → coverage gates per module
Session quality-gate sequence:
/architect → design + trade-offs before any code
/implement → write against the agreed design
/verify → tests, types, risk-gate coverage
/review → Opus batch-reviews decision output
/deploy-check → 10-item PASS/FAIL before live restart
/wrap-up → session summary + open items logged
Touching execution/ or storage schema?
→ /review required before commit
Restarting in live mode?
→ /deploy-check must show all 10 items PASSDesign Pattern in Practice
A real example from July 2026: a class-of-bug fix caught during architecture review that affected every order the system had ever evaluated.
The Bug
Polymarket's /book API returns order-book levels in worst-first order — the opposite of the intuitive convention. Consumers reading bids[0] as best bid were silently reading the worst bid. The MAX_SLIPPAGE gate was passing every trade at 0% slippage because it walked the book backwards from the wrong end.
The Fix
Normalise once at the ClobReadClient boundary: reverse both sides on the way in. Downstream code reads bids[0] and gets the best bid, as expected. The non-standard API contract disappears at the entry point and does not propagate to every consumer.
The Architectural Pattern
Non-obvious external API contracts belong in the client wrapper, not pushed to every consumer. One normalisation fix instead of N consumer fixes. Every future reader gets the intuitive convention without needing to know the API was non-standard.
# BEFORE: API quirk pushed to every consumer
def get_book(self, token_id: str) -> dict:
resp = self._get(f"/book?token_id={token_id}")
return resp # bids[0] = WORST bid (non-intuitive)
# Every consumer silently reads wrong end:
best_bid = book["bids"][0] # worst bid
best_ask = book["asks"][-1] # worst ask
# MAX_SLIPPAGE gate: always 0% — walked backwards
# AFTER: normalise once at the ClobReadClient boundary
def get_book(self, token_id: str) -> dict:
resp = self._get(f"/book?token_id={token_id}")
# Polymarket returns worst-first; invert on the way in
resp["bids"] = list(reversed(resp.get("bids", [])))
resp["asks"] = list(reversed(resp.get("asks", [])))
return resp
# Every consumer now reads correctly:
best_bid = book["bids"][0] # best bid
best_ask = book["asks"][0] # best ask
# MAX_SLIPPAGE gate: real slippage calculated
# One fix — not N consumer fixesAudit Trail Architecture
Nine append-only tables
orders, fills, risk_events, decisions, scans, exits, rotations, resolutions, positions_history — none of these tables are ever updated or deleted. Corrections arrive as new rows with a supersedes_id foreign key pointing to the row being corrected. Forward-only migrations under store/migrations/NNN_*.sql. The audit trail cannot be rewritten from the application layer, and 16 SQLite BEFORE DELETE / BEFORE UPDATE triggers enforce this at the database level.
orders fills risk_events
decisions scans exits
rotations resolutions positions_history
All 9 tables: 16 BEFORE DELETE / BEFORE UPDATE
triggers prevent mutation at the database level.
Corrections → new row with supersedes_id FK
Migrations → store/migrations/NNN_*.sql (forward-only)
Deletions → not possible from the application layerPreflight veto reasons
Every trade that fails the risk gate is written to risk_events with the rule that fired and the full state snapshot at the time. This is what makes polymkt review useful: Opus sees not just the trades taken but the trades considered and refused — and can flag calibration issues in either direction.
Signal Design & Success Metrics
AND floor + dominant-signal exception
Both signals must independently clear 0.4 — or one must clear 0.6 while the other provides corroboration ≥ 0.2. Prevents solo-signal trades without discarding real edge when one signal is exceptionally strong.
Single-whale cap at 0.399
One wallet is anecdote, not concentration. A single whale's score is capped just below the 0.4 admission floor — so a solo whale can never carry a trade alone, regardless of position size or historical accuracy.
PnL log-weight curve
$1k = 0.0 (noise floor), $10k ≈ 0.33, $100k ≈ 0.67, $1M = 1.0. Only real size counts. Small positions are aggressively discounted to prevent low-conviction accounts from moving the signal.
Cross-market arithmetic on neg-risk groups
The highest-trust mispricing source. Yes prices of mutually-exclusive outcomes in a neg-risk group must sum to approximately 1.0 after fees. Deviations from that sum are executable edge — not sentiment, not approximation.
A model that is 60% right at 50% conviction is well-calibrated. A model that is 70% right at 95% conviction is not.
polymkt report computes Brier score — mean((fair_value − outcome)²) — on every resolved market. Lower is better. A naive always-guess-50% baseline scores 0.25. Computed separately for shadow and live modes so calibration can be tracked across the transition. Optimising for win rate produces overconfident decisions; optimising for Brier score produces accurate ones.
Operational Safety
Every safety feature is code-enforced. None rely on the operator doing the right thing at runtime.
First-week $5 position cap
Code-enforced by LIVE_MODE_START_DATE in the environment. Not an honor system — the risk gate rejects any position size > $5 until 7 live trading days have elapsed.
Geoblock check on every boot
GET /api/geoblock on every startup. Refuses --live mode if the operator's current IP is blocked. Runs before any live capital is committed.
Kill switch
polymkt kill cancels all open orders with no confirmation prompt, then sets a persistent flag that blocks live restart until the operator manually runs polymkt resume.
Private key log redaction
The root logger filters the 0x[a-fA-F0-9]{64} pattern before every log write. Covered by a dedicated test. Private keys cannot appear in logs, even inside error traces.
Watchdog.ps1 process manager
An external PowerShell process manager owns the bot lifecycle. The trading process can be restarted without operator presence — watchdog handles crash recovery and log rotation.
WAL-mode SQLite
Write-ahead logging allows the live trading process and read-only inspection commands (polymkt status, polymkt cancel) to run concurrently without database lock contention.
Shadow soak gate before every deployment
≥6h clean --shadow run of the exact code being deployed · risk-gate test suite green · kill switch tested this session · fresh polymkt.db backup taken today · signature_type and funder verified at startup · geoblock check passing from the current IP · zero 0x[a-fA-F0-9]{64} matches in the last 7 days of logs
What This Means In Practice
100%
Decisions Auditable
30min
Default Scan Interval
9
Risk Rules Always Enforced
0
Black-Box Decisions