yambda-discovery-ranking-basic
Build a leakage-safe music-discovery ranker within a fixed CPU and time budget.
Task Description
Inputs: Chronological listen and preference events, target users, cutoff-correct candidate sets, explicit preference state, and a public temporal backtest. No starter model is supplied.
Task: Design, tune, and refit a ranker that returns 100 eligible tracks for every target user. Any modeling approach is allowed inside the platform-owned candidate boundary. The agent has 30 min on 4 CPU · 8 GB · no GPU.
Verification: The offline verifier validates the ranking artifact and scores it on a hidden seven-day outcome window using graded NDCG@10. A run passes at NDCG@10 ≥ 0.025.
Results
55 qualified runs
Individual attempts
One dot per qualified run, grouped by model, harness, and reasoning effort across the full history. Select a dot or its Open trace link to read the recording. Higher hidden quality NDCG@10 is better. Attempts without a valid artifact or verifier reward (including failed validation metrics) are shown at zero reward. All attempts remain in the table below; missing measured rewards and costs are shown as —, not zero. Calibration evidence only.
All attempts
| Run | Model | Harness | Effort | Private reward | Attempt cost |
|---|---|---|---|---|---|
| opus-high-01Full trace | opus-5.0 | Claude Code v2.1.251 | high | 0.021172 | $1.30 |
| opus-high-02Full trace | opus-5.0 | Claude Code v2.1.251 | high | 0.020738 | $1.65 |
| opus-high-03Full trace | opus-5.0 | Claude Code v2.1.251 | high | 0.030378 | $1.88 |
| opus-high-04Full trace | opus-5.0 | Claude Code v2.1.251 | high | 0.026023 | $1.19 |
| opus-high-05Full trace | opus-5.0 | Claude Code v2.1.251 | high | 0.009092 | $2.69 |
| opus-5-5-high-01Full trace | opus-5.5 | Claude Code v2.1.283 | high | 0.033498 | $0.43 |
| opus-5-5-high-02Full trace | opus-5.5 | Claude Code v2.1.283 | high | 0.038827 | $0.42 |
| opus-5-5-high-03Full trace | opus-5.5 | Claude Code v2.1.283 | high | 0.032849 | $0.51 |
| opus-5-5-high-04Full trace | opus-5.5 | Claude Code v2.1.283 | high | 0.022903 | $0.54 |
| opus-5-5-high-05Full trace | opus-5.5 | Claude Code v2.1.283 | high | 0.032805 | $0.57 |
| fable-5-1-high-01Full trace | fable-5.1 | Claude Code v2.1.288 | high | 0.034059 | $1.41 |
| fable-5-1-high-02Full trace | fable-5.1 | Claude Code v2.1.288 | high | 0.031604 | $1.67 |
| fable-5-1-high-03Full trace | fable-5.1 | Claude Code v2.1.288 | high | 0.030161 | $1.75 |
| fable-5-1-high-04Full trace | fable-5.1 | Claude Code v2.1.288 | high | 0.033316 | $1.18 |
| fable-5-1-high-05Full trace | fable-5.1 | Claude Code v2.1.288 | high | 0.021079 | $1.40 |
| sol-high-01Full trace | gpt-5.6-sol | Codex v0.147.0 | high | — | $3.03 |
| sol-high-02Full trace | gpt-5.6-sol | Codex v0.147.0 | high | 0.026471 | $2.66 |
| sol-high-03Full trace | gpt-5.6-sol | Codex v0.147.0 | high | 0.032421 | $2.31 |
| sol-high-04Full trace | gpt-5.6-sol | Codex v0.147.0 | high | 0.033728 | $2.66 |
| sol-high-05Full trace | gpt-5.6-sol | Codex v0.147.0 | high | 0.030634 | $4.00 |
| grok-high-01Full trace | grok-4.6 | Grok Build v1.0.44 | high | 0.008780 | — |
| grok-high-02Full trace | grok-4.6 | Grok Build v1.0.44 | high | 0.030785 | — |
| grok-high-03Full trace | grok-4.6 | Grok Build v1.0.44 | high | 0.009092 | — |
| grok-high-04Full trace | grok-4.6 | Grok Build v1.0.44 | high | 0.008780 | — |
| grok-high-05Full trace | grok-4.6 | Grok Build v1.0.44 | high | 0.008780 | — |
| sol-0.153.4-high-01Full trace | gpt-5.6-sol | Codex v0.153.4 | high | 0.024660 | $2.08 |
| sol-0.153.4-high-02Full trace | gpt-5.6-sol | Codex v0.153.4 | high | 0.025398 | $1.49 |
| sol-0.153.4-high-03Full trace | gpt-5.6-sol | Codex v0.153.4 | high | 0.009092 | $1.92 |
| sol-0.153.4-high-04Full trace | gpt-5.6-sol | Codex v0.153.4 | high | — | $1.95 |
| sol-0.153.4-high-05Full trace | gpt-5.6-sol | Codex v0.153.4 | high | 0.028513 | $1.81 |
| astra-0.153.4-high-01Full trace | gpt-6-astra | Codex v0.153.4 | high | 0.042554 | $6.67 |
| astra-0.153.4-high-02Full trace | gpt-6-astra | Codex v0.153.4 | high | 0.046185 | $8.49 |
| astra-0.153.4-high-03Full trace | gpt-6-astra | Codex v0.153.4 | high | 0.049784 | $8.05 |
| astra-0.153.4-high-04Full trace | gpt-6-astra | Codex v0.153.4 | high | 0.049333 | $7.01 |
| astra-0.153.4-high-05Full trace | gpt-6-astra | Codex v0.153.4 | high | 0.048050 | $8.06 |
| sol-0.153.4-medium-01Full trace | gpt-5.6-sol | Codex v0.153.4 | medium | 0.027901 | $0.98 |
| sol-0.153.4-medium-02Full trace | gpt-5.6-sol | Codex v0.153.4 | medium | 0.035517 | $0.94 |
| sol-0.153.4-medium-03Full trace | gpt-5.6-sol | Codex v0.153.4 | medium | 0.026425 | $1.26 |
| sol-0.153.4-medium-04Full trace | gpt-5.6-sol | Codex v0.153.4 | medium | 0.019697 | $1.03 |
| sol-0.153.4-medium-05Full trace | gpt-5.6-sol | Codex v0.153.4 | medium | 0.034125 | $0.90 |
| sol-0.153.4-max-01Full trace | gpt-5.6-sol | Codex v0.153.4 | max | 0.009092 | $3.50 |
| sol-0.153.4-max-02Full trace | gpt-5.6-sol | Codex v0.153.4 | max | 0.035815 | $2.08 |
| sol-0.153.4-max-03Full trace | gpt-5.6-sol | Codex v0.153.4 | max | 0.037559 | $2.55 |
| sol-0.153.4-max-04Full trace | gpt-5.6-sol | Codex v0.153.4 | max | 0.039252 | $4.16 |
| sol-0.153.4-max-05Full trace | gpt-5.6-sol | Codex v0.153.4 | max | 0.009092 | $2.02 |
| gemini-3-8-flash-high-01Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.57 |
| gemini-3-8-flash-high-02Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.49 |
| gemini-3-8-flash-high-03Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.53 |
| gemini-3-8-flash-high-04Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.60 |
| gemini-3-8-flash-high-05Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009070 | $0.85 |
| gemini-3-8-flash-high-06Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.89 |
| gemini-3-8-flash-high-07Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.65 |
| gemini-3-8-flash-high-08Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009112 | $0.73 |
| gemini-3-8-flash-high-09Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.74 |
| gemini-3-8-flash-high-10Full trace | gemini-3.8-flash | Gemini CLI v0.61.0 | high | 0.009092 | $0.72 |