{ "key": "f5-large-own-fill", "mechanism": "Pause the filled side for one callback when its aggregate own fill exceeds a running pooled nonzero-fill quantile.", "approach": "Blind subclass of verified RC. Use nearest-rank quantile of prior nonzero per-side callback fills, pooled across the run. Above-threshold fills cancel that side and suppress its placements for one callback. Act only on initial-quote spreads 2/3/4. Tune q=0.8/0.9/0.95, freeze the best target-spread mean, then screen on disjoint holdout.", "tried": [ { "name": "q0.8", "q": 0.8, "mean_diff": -22.10411317473767, "se": 2.695863824176402, "code_sha256": "ad80c2b2dbe80b158664de9576f3e6a75aae921613314060ba3a8d81cc7b2e74", "tuning": { "mean_diff": -22.10411317473767, "se": 2.695863824176402, "n": 146 }, "tuning_stats": { "overall": { "n": 200, "sum_d": -3227.2005235117, "sum_d2": 225191.22843678406, "mean_diff": -16.1360026175585, "se": 2.0855877323905165, "comparator_mean": 97.4672213329115, "candidate_mean": 81.331218715353, "retail_diff": -14.838183432032999, "arb_diff": -1.2978191855385004 }, "by_spread": { "1": { "n": 54, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "comparator_mean": 46.182844338938885, "candidate_mean": 46.182844338938885, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "n": 57, "sum_d": -775.6452288639999, "sum_d2": 34983.297809453696, "mean_diff": -13.607811032701754, "se": 2.7664105990137724, "comparator_mean": 87.23571902150526, "candidate_mean": 73.6279079888035, "retail_diff": -15.292879910938597, "arb_diff": 1.6850688782754377 }, "3": { "n": 49, "sum_d": -1247.9504353506, "sum_d2": 112086.26987035405, "mean_diff": -25.468376231644896, "se": 5.843152522819505, "comparator_mean": 112.87398754092447, "candidate_mean": 87.4056113092796, "retail_diff": -25.024146034561227, "arb_diff": -0.4442301971285716 }, "4": { "n": 40, "sum_d": -1203.6048592971001, "sum_d2": 78121.66075697631, "mean_diff": -30.090121482427502, "se": 5.182876411793672, "comparator_mean": 162.4077324637125, "candidate_mean": 132.317610981285, "retail_diff": -21.74398439474, "arb_diff": -8.3461370877525 } }, "target": { "n": 146, "sum_d": -3227.2005235117, "sum_d2": 225191.22843678406, "mean_diff": -22.10411317473767, "se": 2.695863824176402, "comparator_mean": 116.43541556355889, "candidate_mean": 94.33130238882124, "retail_diff": -20.326278674017807, "arb_diff": -1.7778345007376717 } } }, { "name": "q0.9", "q": 0.9, "mean_diff": -20.688235536117123, "se": 2.518855196802289, "code_sha256": "a2d112e100b66635b7f9f048bef06d26dc3a5176a895402a7f2cfba8f012ca2e", "tuning": { "mean_diff": -20.688235536117123, "se": 2.518855196802289, "n": 146 }, "tuning_stats": { "overall": { "n": 200, "sum_d": -3020.4823882731, "sum_d2": 196804.29998832115, "mean_diff": -15.1024119413655, "se": 1.9490219857609172, "comparator_mean": 97.4672213329115, "candidate_mean": 82.364809391546, "retail_diff": -13.623019411679497, "arb_diff": -1.4793925296880002 }, "by_spread": { "1": { "n": 54, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "comparator_mean": 46.182844338938885, "candidate_mean": 46.182844338938885, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "n": 57, "sum_d": -764.9460048328, "sum_d2": 33327.42169547299, "mean_diff": -13.42010534794386, "se": 2.6879107033112075, "comparator_mean": 87.23571902150526, "candidate_mean": 73.8156136735614, "retail_diff": -14.518698745954385, "arb_diff": 1.098593398043859 }, "3": { "n": 49, "sum_d": -1145.9136122725, "sum_d2": 95414.40553831302, "mean_diff": -23.385992087193877, "se": 5.401250125714968, "comparator_mean": 112.87398754092447, "candidate_mean": 89.4879954537306, "retail_diff": -22.89013904680408, "arb_diff": -0.49585304036530664 }, "4": { "n": 40, "sum_d": -1109.6227711678, "sum_d2": 68062.47275453513, "mean_diff": -27.740569279195, "se": 4.8885597347526755, "comparator_mean": 162.4077324637125, "candidate_mean": 134.6671631845175, "retail_diff": -19.38553101307749, "arb_diff": -8.355038266205 } }, "target": { "n": 146, "sum_d": -3020.4823882731, "sum_d2": 196804.29998832115, "mean_diff": -20.688235536117123, "se": 2.518855196802289, "comparator_mean": 116.43541556355889, "candidate_mean": 95.74718002744179, "retail_diff": -18.661670426958217, "arb_diff": -2.026565109161644 } } }, { "name": "q0.95", "q": 0.95, "mean_diff": -16.364870095714384, "se": 2.153875255442447, "code_sha256": "44170086f66d6f9f34de57d10df5367f05161ce58bbaf54f2d2594b19c49a245", "tuning": { "mean_diff": -16.364870095714384, "se": 2.153875255442447, "n": 146 }, "tuning_stats": { "overall": { "n": 200, "sum_d": -2389.2710339743, "sum_d2": 137311.5213953164, "mean_diff": -11.946355169871499, "se": 1.6531410716187218, "comparator_mean": 97.4672213329115, "candidate_mean": 85.52086616304001, "retail_diff": -10.273979050149501, "arb_diff": -1.67237611976 }, "by_spread": { "1": { "n": 54, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "comparator_mean": 46.182844338938885, "candidate_mean": 46.182844338938885, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "n": 57, "sum_d": -641.3287177393, "sum_d2": 24180.408725024972, "mean_diff": -11.251381012970176, "se": 2.3053668836900876, "comparator_mean": 87.23571902150526, "candidate_mean": 75.98433800853508, "retail_diff": -11.477692803464912, "arb_diff": 0.2263117905087717 }, "3": { "n": 49, "sum_d": -932.8610110161999, "sum_d2": 72065.27112975669, "mean_diff": -19.037979816657142, "se": 4.805108270275112, "comparator_mean": 112.87398754092447, "candidate_mean": 93.83600772426733, "retail_diff": -17.642959871197963, "arb_diff": -1.3950199455265304 }, "4": { "n": 40, "sum_d": -815.0813052187999, "sum_d2": 41065.84154053476, "mean_diff": -20.377032630469998, "se": 3.9594825474216537, "comparator_mean": 162.4077324637125, "candidate_mean": 142.03069983324252, "retail_diff": -13.401557163592503, "arb_diff": -6.975475467005 } }, "target": { "n": 146, "sum_d": -2389.2710339743, "sum_d2": 137311.5213953164, "mean_diff": -16.364870095714384, "se": 2.153875255442447, "comparator_mean": 116.43541556355889, "candidate_mean": 100.07054546784451, "retail_diff": -14.073943904314385, "arb_diff": -2.290926191452055 } } } ], "next": "Retain RC; this is a screen, not a promotion. Do not use the reserved finalist seeds for this item.", "spreads_acted": [ 2, 3, 4 ], "off_equals_rc": true, "code": "from bisect import insort\nfrom math import ceil\nfrom orderbook_pm_challenge.types import CancelAll, CancelOrder, PlaceOrder, Side\n\n# During standalone assembly, the incumbent Strategy is renamed _RC.\nclass Strategy(_RC):\n \"\"\"One-callback same-side pause after a historically large own fill.\"\"\"\n fill_quantile = 0.95\n overlay_enabled = True\n\n def on_step(self, state):\n actions = super().on_step(state)\n if not self.overlay_enabled:\n return actions\n if state.step == 0 or not hasattr(self, '_lof_history'):\n self._lof_history = []\n bid = state.competitor_best_bid_ticks\n ask = state.competitor_best_ask_ticks\n self._lof_spread = (max(1, (ask - bid + 1) // 2)\n if bid is not None and ask is not None else None)\n if self._lof_spread not in (2, 3, 4):\n return actions\n history = self._lof_history\n threshold = (history[max(0, ceil(self.fill_quantile * len(history)) - 1)]\n if history else float('inf'))\n fills = ((Side.BUY, state.buy_filled_quantity),\n (Side.SELL, state.sell_filled_quantity))\n paused = {side for side, quantity in fills if quantity > threshold}\n for side, quantity in fills:\n if quantity > 0:\n insort(history, quantity)\n if not paused:\n return actions\n # Cancel outstanding orders and suppress replacements only this callback.\n already_cancelled = {action.order_id for action in actions\n if isinstance(action, CancelOrder)}\n clears_all = any(isinstance(action, CancelAll) for action in actions)\n cancels = [CancelOrder(order.order_id) for order in state.own_orders\n if order.side in paused and not clears_all\n and order.order_id not in already_cancelled]\n return cancels + [action for action in actions\n if not (isinstance(action, PlaceOrder) and action.side in paused)]\n", "code_sha256": "44170086f66d6f9f34de57d10df5367f05161ce58bbaf54f2d2594b19c49a245", "assembled_sha256": "d315d75aa3211b264b83c6d175925d36a12553d9744afd40d2b3dba6e41355be", "comparator_sha256": "3c07bdcd221ac25729c3ebc92b21ac9e2014510ecb85816c4391ed81e933291f", "assembly_validation": { "seed": 330000, "equal": true, "literal_sha256": "d315d75aa3211b264b83c6d175925d36a12553d9744afd40d2b3dba6e41355be", "semantic_sha256": "d9fa713c82a80f219ec05cd14b060ec1d4074fb20bb0df9a43b83515212e7299", "renamed_identifiers": 3, "literal_import_error": "NameError: name 'Strategy' is not defined", "semantic_assembly": "Rename NAME tokens Strategy to _RC from the class declaration onward, preserving all other text, then append one newline and overlay." }, "tuning": { "seeds": { "start": 330000, "end": 330199 }, "n": 200, "complete": true, "missing_seeds": [], "selected": { "q": 0.95, "name": "q0.95", "code_sha256": "44170086f66d6f9f34de57d10df5367f05161ce58bbaf54f2d2594b19c49a245", "selection": "Highest mean paired difference on tuning spreads 2, 3 and 4; no holdout consulted." }, "overall": { "n": 200, "sum_d": -2389.2710339743, "sum_d2": 137311.5213953164, "mean_diff": -11.946355169871499, "se": 1.6531410716187218, "comparator_mean": 97.4672213329115, "candidate_mean": 85.52086616304001, "retail_diff": -10.273979050149501, "arb_diff": -1.67237611976 }, "by_spread": { "1": { "n": 54, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "comparator_mean": 46.182844338938885, "candidate_mean": 46.182844338938885, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "n": 57, "sum_d": -641.3287177393, "sum_d2": 24180.408725024972, "mean_diff": -11.251381012970176, "se": 2.3053668836900876, "comparator_mean": 87.23571902150526, "candidate_mean": 75.98433800853508, "retail_diff": -11.477692803464912, "arb_diff": 0.2263117905087717 }, "3": { "n": 49, "sum_d": -932.8610110161999, "sum_d2": 72065.27112975669, "mean_diff": -19.037979816657142, "se": 4.805108270275112, "comparator_mean": 112.87398754092447, "candidate_mean": 93.83600772426733, "retail_diff": -17.642959871197963, "arb_diff": -1.3950199455265304 }, "4": { "n": 40, "sum_d": -815.0813052187999, "sum_d2": 41065.84154053476, "mean_diff": -20.377032630469998, "se": 3.9594825474216537, "comparator_mean": 162.4077324637125, "candidate_mean": 142.03069983324252, "retail_diff": -13.401557163592503, "arb_diff": -6.975475467005 } }, "target": { "n": 146, "sum_d": -2389.2710339743, "sum_d2": 137311.5213953164, "mean_diff": -16.364870095714384, "se": 2.153875255442447, "comparator_mean": 116.43541556355889, "candidate_mean": 100.07054546784451, "retail_diff": -14.073943904314385, "arb_diff": -2.290926191452055 } }, "paired": { "overall": { "n": 500, "sum_d": -5740.063372926799, "sum_d2": 520667.7247732747, "mean_diff": -11.480126745853598, "se": 1.3500850856031663, "comparator_mean": 101.4369248623788, "candidate_mean": 89.95679811652519, "retail_diff": -9.283316413807201, "arb_diff": -2.1968103320234 }, "by_spread": { "1": { "n": 142, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "comparator_mean": 28.7146791778493, "candidate_mean": 28.7146791778493, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "n": 110, "sum_d": -1319.2412443432, "sum_d2": 99425.93620936786, "mean_diff": -11.993102221301818, "se": 2.640610240958344, "comparator_mean": 88.78927784058273, "candidate_mean": 76.7961756192809, "retail_diff": -11.741701236251819, "arb_diff": -0.25140098501454494 }, "3": { "n": 119, "sum_d": -1686.4739793565, "sum_d2": 220856.69156463648, "mean_diff": -14.172050246693278, "se": 3.7451568458273243, "comparator_mean": 143.37780997585293, "candidate_mean": 129.20575972915967, "retail_diff": -16.618703168737817, "arb_diff": 2.4466529220806716 }, "4": { "n": 129, "sum_d": -2734.3481492271, "sum_d2": 200385.09699927032, "mean_diff": -21.196497280830233, "se": 2.936943229122517, "comparator_mean": 153.5829305298, "candidate_mean": 132.38643324896978, "retail_diff": -10.639111580124805, "arb_diff": -10.557385700679845 } }, "target": { "n": 358, "sum_d": -5740.063372926799, "sum_d2": 520667.7247732747, "mean_diff": -16.033696572421228, "se": 1.8313330266850176, "comparator_mean": 130.28206141881228, "candidate_mean": 114.24836484639106, "retail_diff": -12.965525717607822, "arb_diff": -3.0681708547812847 }, "seeds": { "start": 440000, "end": 440499 }, "n": 500, "complete": true, "missing_seeds": [] }, "failures": [ { "stage": "Discarded initial implementation check", "seeds": { "start": 330000, "end": 330049 }, "failed_simulations": 111, "error": "OrderBookError: Unknown participant order (duplicate cancellation)", "resolution": "Deduplicate new cancellations against RC CancelOrder actions and skip additions when RC returns CancelAll. Discarded all initial records and restarted tuning with the repaired implementation. No failed runs enter reported statistics. These were repairs of the same three specified quantile variants, not additional tuned mechanisms." }, { "stage": "Literal assembly validation", "error": "NameError: name 'Strategy' is not defined", "resolution": "Requested assembled_sha256 preserves the literal class-declaration-only rename. See assembly_validation for a successful automatic semantic class/reference rename and its hash; no incumbent source was displayed or manually inspected." } ], "check_values": { "310000": 17.5126554022, "310001": 20.6298693268, "310002": 2.3903113616, "310003": 248.006347259 }, "python_version": "3.12.3", "seconds_per_simulation": 0.8891348318837504, "notes": { "scoreboard": { "current_best_code_sha256": "3c07bdcd221ac25729c3ebc92b21ac9e2014510ecb85816c4391ed81e933291f", "comparator": "RC", "candidates": "tried contains each overlay hash and paired tuning difference; paired contains selected holdout difference", "promotion": false }, "quantile": "Nearest rank ceil(q*n), using all prior positive buy and sell aggregate callback fills pooled in a sorted list. Current observations are added only after threshold comparison. With no history, no pause. Strictly greater than threshold. Consecutive qualifying fills can pause consecutive callbacks.", "cancellation": "Call super first. Keep all RC actions except PlaceOrder on paused sides; prepend only missing CancelOrder actions for currently resting orders on those sides. Do not duplicate RC cancellations or add cancellations when RC clears all.", "validation": "Synthetic tests: cold start, strict threshold, small-fill identity, same-side placement suppression, outstanding cancellation, pause expiry, off-target identity. Disabled-overlay action lists and final edges equal RC on seeds 320000..320002, all 6000 callbacks. All reported simulations completed without failure.", "off_target": "Spread 1 asserted to have exactly zero total, retail, and arb difference for every tuning and holdout pair.", "simulator": "Pinned b244530ee2a6247b5f930830eeca9dd950da0e7e, unmodified; sample_config(ChallengeConfig(), ParameterVariance(), seed=seed); SimulationEngine(cfg, factory, seed=seed).run(); 2000 steps; serial, no sandbox. Each command covers at most 50 seeds. Records appended, unique and complete per strategy checked.", "assembly": "UTF-8 RC text with first class Strategy renamed class _RC, followed by one newline and the exact overlay code. Overlay expects _RC in assembly; for separate import inject _RC=incumbent_module.Strategy.", "timing": "Mean wall seconds per simulation across final tuning, holdout, and four RC check runs; discarded implementation checks and disabled-overlay validation excluded.", "blind": "RC fetched, hash checked and imported without displaying or inspecting its source. No other strategy, leaderboard or previous result was read. Finalist seeds 900000..903999 untouched.", "missing": "The requested screen is complete. Literal class-declaration-only assembly raises NameError because an incumbent reference still uses Strategy; semantic class/reference renaming fixes this, validated against the frozen candidate. No results or data fabricated." } }