Portfolio — Case Study
AI Trading · Prediction MarketsAutonomous AI trading on the world's largest decentralised prediction market.
An autonomous prediction market trading bot built on Polymarket — the world’s largest decentralised prediction market. Gemini Flash 2.5 acts as the AI decision layer: evaluating market positions, producing 9-field structured JSON reasoning, and submitting orders via Polymarket’s CLOB API. Every decision is explainable, every trade auditable, and the risk gate is hardcoded — the LLM cannot override it.
Gemini
Flash 2.5
AI Decision Layer
9
Hard Risk Rules
16
Audit Log Triggers
Shadow
Safe Testing System
The Challenge
Prediction markets price real-world events — elections, sports results, economic releases — but the prices are set by human sentiment and often diverge significantly from actual probability. The bot needed to identify genuine mispricing: where market price deviates from calculated expected value, execute positions within strict risk parameters, and operate in a fully auditable way where every single decision could be explained and reviewed.
Gemini Flash 2.5 was chosen as the AI decision layer for its ability to produce reliable structured JSON reasoning across complex probability scenarios at the scan frequency required. Every decision carries nine explicit fields — conviction score, estimated fair value, edge in basis points, reasoning, bear case, resolution risk, and more — so any trade can be opened and understood by a human reviewer at any time. Unstructured or black-box output was not acceptable.
The audit requirement was non-negotiable: every signal evaluated, every Claude decision, and every trade placed had to be permanently and immutably recorded. Append-only SQLite triggers ensure the log cannot be altered after writing, providing a complete and tamper-resistant decision trail.
What Was Delivered
Technology Stack
AI
Trading
Data Signals
Backend
Interface
Risk
Key Features
The AI evaluates each shortlisted market and returns a structured JSON object with nine required fields: decision (BUY, SELL, or PASS), outcome (YES or NO), conviction (0.0–1.0), fair_value (estimated true probability, used for Brier score calibration), edge_bps (expected edge in basis points), reasoning (trade thesis), bear_case (strongest argument against), resolution_risk (LOW, MEDIUM, or HIGH), and size_usd (always clamped by the risk gate — the LLM suggestion is never trusted directly). If conviction falls below 0.70 or resolution_risk is HIGH, the LLM auto-sets its own decision to PASS. This is an LLM-layer override, not a risk gate check.
Nine hard-coded rules checked in this exact order before every order submission: (1) kill switch active, (2) daily loss ≥ $50 — triggers full halt and cancels all open orders, (3) market price below 5% or above 95%, (4) resolves within 6 hours, (5) position size exceeds $25 (hardcoded $5 cap for the first 7 live trading days), (6) more than 10 concurrent positions open, (7) total exposure exceeds $250, (8) exit-side book depth below $1,000, (9) fill would move price more than 2% from best. Any single failure vetoes the trade and writes to risk_events with the full state snapshot. The 0.70 conviction threshold is an LLM-layer self-override — it is not part of this gate.
smart_money.py tracks on-chain wallet concentration using Polymarket’s leaderboard and top-holder data. Each wallet is weighted by its realised PnL on a log scale — preventing a single high-PnL outlier from dominating the signal. When significant wallet concentration moves in a direction, the score rises and informs combine.py. The whale_wallets table records each tracked wallet, its PnL, sample size, and last refresh timestamp.
mispricing.py combines three independent sub-scores into a single mispricing signal. Cross-market arithmetic detects price deviations within neg-risk groups — binary markets whose probabilities must sum to 1.00. Book microstructure scores bid/ask spread, order depth, and book asymmetry. Stale price measures hours since the last recorded trade. combine.py merges both signals into a weighted composite; either signal at zero collapses the composite to zero and the market is skipped.
16 SQLite BEFORE DELETE and BEFORE UPDATE triggers enforce append-only writes at the database level — not by application discipline. All 11 tables are covered: scans (every scored market, both signal scores, composite, component breakdown JSON), decisions (full LLM request and response, parsed decision, prompt version), orders (every order in shadow and live mode, idempotency key), fills, positions, resolutions (outcomes and realised PnL), risk_events (every veto and halt with state snapshot), exits, rotations, whale_wallets, and markets.
The bot has no web interface. It is operated entirely through a Python CLI: polymkt shadow runs the full decision cycle and logs intended trades without placing orders; polymkt live activates real order submission; polymkt report generates a session summary with Brier scores; polymkt kill cancels all open orders, writes a HALT record, and locks the system until an operator runs polymkt resume. Shadow and live records share the same 11 audit tables, distinguished only by the mode column. Transitioning from shadow to live is a single command — no code change required.
Three separate Polymarket APIs, each with its own HTTP client and independent rate-limit bucket at 5 req/s, burst 10. Gamma handles market discovery. CLOB handles order book data and order execution. Data handles on-chain holder distribution and leaderboard PnL. Because each client manages its own backpressure independently, a slow or throttled response from one API cannot block the others.
Every order gets a SHA-256 idempotency key — a hash of market ID, side, rounded price, size, and decision ID — stored as orders.client_key. Before any order is submitted, the key is checked against the database. A duplicate key means the process crashed and restarted mid-submission; the order is not resubmitted, preventing double-fills. The kill switch (polymkt kill) cancels all open orders and writes a HALT record. The system refuses to enter live mode until an operator explicitly runs polymkt resume — no confirmation prompt, no automatic recovery.
199 automated tests cover every critical path: all 9 risk gate limits at breach, pass, and boundary conditions; idempotency key collision handling; LLM response parsing for all 9 output fields; signal scoring edge cases; resolution detection; and log redaction — verifying that private keys never appear in any log output. Calibration is measured by Brier score — mean((fair_value − outcome)²) across resolved positions — computed separately for shadow and live modes. A naive always-guess-50% baseline scores 0.25; lower is better. Win rate is not the primary metric.
System Architecture
Market data enters from three independent Polymarket API surfaces, each with its own rate-limited HTTP client at 5 requests per second, burst 10. Gamma discovers active markets. CLOB provides order book depth for microstructure analysis and handles order execution. Data supplies on-chain holder distribution and leaderboard PnL for the smart money signal. Because each client has its own rate-limit bucket, backpressure on one API surface cannot starve the others. Every 30 minutes — configurable via the --interval CLI flag — the scan loop fetches fresh data and passes it to the signal engine.
Two independent signals score each market. smart_money.py weights on-chain wallet concentration by log-scale realised PnL. mispricing.py combines neg-risk group arithmetic, book microstructure, and stale price detection into a single score. combine.py merges them into a weighted composite; both signals must independently clear a 0.4 floor and agree on direction — YES or NO — before the market is shortlisted. Either signal at zero collapses the composite to zero. Shortlisted markets go to Gemini Flash 2.5, which returns a 9-field JSON decision. That decision then passes through the 9-rule hard risk gate — checked synchronously, in order — before any order is submitted. Before submission, a SHA-256 idempotency key derived from market ID, side, rounded price, size, and decision ID is checked against the orders table to prevent double-fills on process restart.
Decisions that pass the risk gate proceed to the trade executor: in live mode, an order is submitted via the CLOB API; in shadow mode, the intended trade is logged with mode=shadow and zero capital is at risk. Every step across the pipeline writes immutable records to the 11-table SQLite audit schema, enforced by 16 BEFORE DELETE / BEFORE UPDATE triggers at the database level. Brier score — mean((fair_value − outcome)²) across resolved positions — is the primary calibration metric, computed separately for shadow and live modes. A naive always-guess-50% baseline scores 0.25; lower is better. The system has no web interface.
Polymarket APIs (3 independent rate-limited clients: 5 req/s, burst 10)
├── Gamma API → market discovery, active market list
├── CLOB API → order book depth + order execution
└── Data API → on-chain holders + leaderboard PnL
│
▼ 30-min scan interval (--interval CLI flag, configurable)
Signal Engine
├── smart_money.py → wallet concentration, log-scale PnL weighting
├── mispricing.py → neg-risk arb + book microstructure + stale price
└── combine.py → weighted composite; both signals must clear 0.4
│ and agree on direction (YES or NO)
▼
Gemini Flash 2.5 (Decision Layer)
│ Output: { decision, outcome, conviction, fair_value,
│ edge_bps, reasoning, bear_case,
│ resolution_risk, size_usd }
│ Auto-PASS if conviction < 0.70 or resolution_risk = HIGH
▼
9-Rule Hard Risk Gate (checked in order, not configurable)
│ 1. Kill switch active? 6. Max concurrent positions (10)
│ 2. Daily loss ≥ $50? 7. Total exposure > $250?
│ 3. Price out of 5–95%? 8. Exit liquidity < $1,000?
│ 4. Resolves in < 6 hours? 9. Slippage > 2%?
│ 5. Position size > $25?
│ FAIL → trade vetoed, written to risk_events
▼
Trade Executor
│ SHA-256 idempotency key checked before every submission
│ Live: CLOB API order │ Shadow: log only (mode = shadow)
▼
SQLite Audit (11 tables, 16 append-only BEFORE DELETE / BEFORE UPDATE triggers)
├── scans / decisions / orders / fills / positions
└── resolutions / risk_events / exits / rotations
whale_wallets / marketsWhat This Means In Practice
100%
Decisions Auditable
30min
Default Scan Interval
9
Risk Rules Always Enforced
0
Black-Box Decisions