{ "what_if": "Stock is free: forgive negative edge on YES bid fills at 5 cents or below", "blocked": null, "python_version": "3.10.12", "base_mean_edge": 109.27408535278173, "base_unseen": { "mean_edge": 98.27031805983209, "retail_edge": 126.02025543371683, "arb_edge": -27.74993737388321 }, "diff": { "mean": 10.600204159728674, "standard_error": 0.47489436001810703, "simulations": 4000, "seed_start": 50000, "retail": 0.0, "arb": 10.600204159729392 }, "by_spread": [ { "spread": 1, "runs": 1016, "diff": 16.15857656548119, "standard_error": 1.4810119967139241 }, { "spread": 2, "runs": 998, "diff": 11.223948532129667, "standard_error": 0.9254338233516672 }, { "spread": 3, "runs": 1017, "diff": 8.328110261261065, "standard_error": 0.527909940587626 }, { "spread": 4, "runs": 969, "diff": 6.514462412402368, "standard_error": 0.40289349071506947 } ], "how": "Cloned https://github.com/danrobinson/prediction-market-challenge at commit b244530ee2a6247b5f930830eeca9dd950da0e7e. Extracted JSON code verbatim; SHA256 baef955ec10728839f7c3357016ec9eb0550e7ec689bbd5171b1cdd5c9e37273. In scratch copy orderbook_pm_challenge/market.py, PredictionMarket.record_participant_fills, immediately after computing the original fill edge and before adding it to total_edge and retail_edge/arb_edge, inserted exactly: if fill.side is Side.BUY and fill.price_ticks <= 5 and edge < 0.0: edge = 0.0. Positive and zero edge are retained. Includes all participant bids at those ticks, including stock bids, ordinary quotes and tiny probes, as required; excludes every SELL and bids above tick 5. Ran PYTHONHASHSEED=0 python3 -m orderbook_pm_challenge run ../base.py --simulations 600 --seed-start 1000 --workers 4 --json for calibration, and the same command with --simulations 4000 --seed-start 50000 in both original and patched clones. The supplied base artifact records Python 3.12.3; this experiment runs Python 3.10.12. Reference-score discrepancies are reported rather than hidden. Python changed float summation in 3.12; this is a possible contributor to the drift, not an isolated causal diagnosis. PYTHONHASHSEED=0 was fixed equally for original and patched processes (including training/attempt), to prevent hash-order floating summation differences in collateral from contaminating pairing. No sandbox, dependency installs, latent-process/config/flow/book/collateral changes, or strategy changes in the what-if. Means use the simulator BatchResult convention sum(values)/n. Differences pair identical seeds; standard error is statistics.stdev(per-seed differences)/sqrt(n), using sample variance (n-1). Spread groups use the sampled config competitor_spread_ticks, not strategy estimates. All numeric JSON values are unrounded Python floats.", "clean": "Only recorded edge is forgiven: no cash subsidy, free shares, relaxed collateral, order priority, probability signal, modified fills, or inventory advantage is supplied. Stats are not part of StepState and cannot feed back into the strategy. All 4000 paired runs match on every non-edge SimulationResult field. This is loss forgiveness, not setting purchase price to zero (which would also boost profitable purchases). It is a fixed-base ceiling for removing this particular acquisition cost while retaining its current downstream cover benefits. Per-step scorer rounding to 10 decimals is retained, so paired total and component differences can differ at floating/rounding precision.", "realizable": "A real strategy can reduce this loss by avoiding or shrinking low-price replenishment bids when existing YES covers foreseeable retail sales, and by resting bids below its probability estimate to acquire stock on small crossings rather than paying above fair value. The base already uses crossing bids; the incremental mechanism tested here caps combined cheap bid targets by projected cover demand. It cannot forgive an executed negative-edge purchase, see the next jump, or identify the future aggressor. Cutting such bids risks losing profitable retail bid fills and bid-fill information; shrinking stock also leaves fewer covered asks, so selling either stops or needs nearly $1 collateral per uncovered YES, reducing capacity elsewhere. With near-zero unchanged fair value, a purchase and resale at the same tick have approximately zero combined edge; forgiving the purchase while retaining the sale grants a subsidy, so removing both trades does not realize that apparent gain. The accounting ceiling retains all downstream cover benefits and therefore is not the expected gain of the real mechanism. See attempt for the frozen unseen test with those opportunity costs included. No positive realizable gain is established by this attempt: the frozen stock cap changes unseen edge by -7.335753105350152 (paired standard error 0.5921357294213615). The loss-forgiveness ceiling remains an accounting opportunity, not a demonstrated profitable redesign.", "attempt": { "mechanism": "At phat < 0.05, before original order management, compute target = 0.4 * min(steps_remaining, horizon) / (max(planned_ask_tick, 1) / 100), stock_room = max(0, target - current_yes_inventory). Cap qam at stock_room when amt <= 5, then cap qb at remaining stock_room when bid_t <= 5, then cap q1 at the remainder. Preserve the base tiny probes, posterior, collateral accounting, and existing order management/hysteresis. This changes bid quantity targets only, in the original simulator; existing order-size hysteresis can leave quantities up to its original retention ratio.", "tuning": { "simulations": 600, "seed_start": 4000, "seed_end": 4599, "horizons_tested": [ 50, 200, 1000 ], "baseline_mean_edge": 100.61720680106001, "candidates": { "stock_cap_50": { "mean_edge": 77.84093679246887, "diff": { "mean": -22.776270008591155, "standard_error": 2.7875326952395807, "simulations": 600, "seed_start": 4000, "retail": -19.230722690041176, "arb": -3.5455473185486652 } }, "stock_cap_200": { "mean_edge": 84.86057113909268, "diff": { "mean": -15.756635661967321, "standard_error": 2.023887976833277, "simulations": 600, "seed_start": 4000, "retail": -9.624314277328834, "arb": -6.132321384626836 } }, "stock_cap_1000": { "mean_edge": 92.78343130417436, "diff": { "mean": -7.833775496885666, "standard_error": 1.0903452616605493, "simulations": 600, "seed_start": 4000, "retail": -5.68178341429083, "arb": -2.151992082598168 } } }, "selection_rule": "Highest training mean edge among the three capped candidates, even if below baseline; no selection using unseen results.", "selected_horizon": 1000 }, "unseen": { "mean_edge": 90.93456495448173, "retail_edge": 120.3198404298684, "arb_edge": -29.38527547538158 }, "diff": { "mean": -7.335753105350152, "standard_error": 0.5921357294213615, "simulations": 4000, "seed_start": 50000, "retail": -5.700415003848648, "arb": -1.6353381014983754 }, "by_spread": [ { "spread": 1, "runs": 1016, "diff": -11.567850906789163, "standard_error": 1.7099985386727572 }, { "spread": 2, "runs": 998, "diff": -7.678892337310322, "standard_error": 0.9726595244926893 }, { "spread": 3, "runs": 1017, "diff": -5.329446163977386, "standard_error": 1.0047350188287385 }, { "spread": 4, "runs": 969, "diff": -4.650665220538809, "standard_error": 0.7858947441650846 } ], "failed_runs": 0, "interpretation": "The selected real stock cap reduces unseen edge; do not deploy this version." }, "tried": [ "Calibration hash-order control: 0 of 600 run records differ between default-hash and fixed-hash processes; same strategy code and simulator source.", "Initial default-hash baseline calibration: mean edge 109.27408535278173, difference 0.0015424626097342298 from supplied author reference. Unfinished default-hash batches were discarded; final paired study fixes PYTHONHASHSEED=0 for all processes. Default and fixed-hash calibrations match every run; hash order did not explain the observed reference discrepancy.", "Official unmodified CLI, 600 runs, seeds 1000..1599: mean edge 109.27408535278173; difference from supplied reference 0.0015424626097342298; 0 failures.", "Official unmodified CLI, 4000 runs, seeds 50000..53999: {'mean_edge': 98.27031805983209, 'retail_edge': 126.02025543371683, 'arb_edge': -27.74993737388321}; difference from supplied reference -0.010315827197729277; 0 failures.", "Patched CLI, unchanged base, same 4000 seeds: {'mean_edge': 108.87052221956026, 'retail_edge': 126.02025543371683, 'arb_edge': -17.149733214153766}; paired mean gain 10.600204159728674, sample standard error 0.47489436001810703; 0 failures.", "Fixed-hash paired pilot: 10 runs on seeds 4000..4009, mean gain 12.009740650429999; all non-edge outputs match exactly; used as an accounting check, not unseen evidence.", "Accounting boundary tests: 8 cases passed in each simulator (negative/positive/zero BUY edge, ticks 1/5/6, SELL exclusion, retail/arb allocation, no state mutation).", "Across all 4000 paired runs, every reported non-edge field matched exactly, including regime, fill count, quantity, notional, inventory summaries, final cash, holdings, settlement, wealth, and failure status.", "Training only, seeds 4000..4599, stock_cap_50: mean edge 77.84093679246887, paired gain -22.776270008591155, standard error 2.7875326952395807; 0 failures.", "Training only, seeds 4000..4599, stock_cap_200: mean edge 84.86057113909268, paired gain -15.756635661967321, standard error 2.023887976833277; 0 failures.", "Training only, seeds 4000..4599, stock_cap_1000: mean edge 92.78343130417436, paired gain -7.833775496885666, standard error 1.0903452616605493; 0 failures.", "Training baseline, seeds 4000..4599: mean edge 100.61720680106001; 0 failures.", "Frozen training winner stock_cap_1000, unseen seeds 50000..53999: mean edge 90.93456495448173, paired gain -7.335753105350152, standard error 0.5921357294213615; 0 failures." ], "next": "Open stock-acquisition research. First trace cheap-bid losses against inventory, remaining time, and jump overshoot in spread 1 (ceiling 16.15857656548119); then model the joint marginal value of YES stock, cash collateral, and future covered retail asks. Preserve probes and the cheap crossing bank: all three current-ask stock caps lost on training, so do not start with a plain inventory reduction. Tune on seeds 4000..7999 and use a fresh holdout after this evaluation." }