Methodology 1.0.0
Released 2026-04-20. Initial public methodology. Per-track scores are raw; composite is not yet computed until cross-lab replication data is available.
Current scoring formula
{
"composite": {
"note": "Composite is suppressed until a run has at least one published metric in every track.",
"weights": {
"signal-quality": 0.15,
"responsiveness": 0.15,
"plasticity": 0.2,
"closed-loop-learning": 0.2,
"retention": 0.15,
"reproducibility": 0.15
}
}
}Tracks and metrics
Electrophysiological recording fidelity from the preparation: SNR, channel yield, spike sortability.
A closed-loop system cannot score above its recording floor. Signal quality is the baseline precondition for every downstream claim.
Composite of normalized SNR, active-channel ratio, and spike-sorting unit yield. Equal weights per sub-metric, clipped to [0, 1].
- Active channel ratioratio · ↑
- ISI violation rateratio · ↓
- Sorted units per arrayunits · ↑
- Spike SNRdB · ↑
Does input change output? Evoked response magnitude, latency, and selectivity under matched controls.
Responsiveness separates preparations that merely produce spontaneous activity from those whose activity is contingent on input.
Weighted mean of evoked-vs-sham effect size (z-scored), response latency (inverted), and input-pattern selectivity (d-prime).
- Evoked response effect sized · ↑
- Pattern selectivityd' · ↑
- Response latencyms · ↓
Does repeated input durably change the input-to-output mapping? Open-loop plasticity with matched controls.
Plasticity is the mechanistic substrate any learning claim depends on. Without paired/unpaired controls, drift and adaptation look identical.
Post-minus-pre response change normalized by sham-matched change. Requires paired-vs-unpaired control to score.
- Induction half-lifemin · ↓
- Paired vs unpaired deltadelta · ↑
Task performance under stimulus-contingent feedback, versus matched no-feedback control.
Closed-loop performance improvement (vs control) is the field's operational definition of stimulus-contingent adaptation at the task level. Not a claim about internal cognition.
Performance improvement over training (feedback minus no-feedback), z-scored against a batch-level null distribution.
- Learning curve AUCauc · ↑
- Terminal performance deltadelta · ↑
Does an induced change persist? Re-test at fixed delays against matched pre-induction baseline.
Retention is how a plasticity or learning claim transitions from 'change' to 'memory'. Requires specified delays and no intervening exposure.
Retained effect at 1h/24h/7d expressed as fraction of immediate post-induction effect. Score is weighted mean across delays.
- Retention at 1hratio · ↑
- Retention at 24hratio · ↑
- Retention at 7dratio · ↑
Are results stable across batches and labs? Within-lab and cross-lab coefficient of variation on headline metric.
Per-track scores without reproducibility data hide cross-batch variance. This track is a first-class citizen, not a footnote.
1 minus normalized coefficient of variation across batches (within-lab) and across labs (cross-lab). Both must be reported for a full score.
- Cross-lab CVcv · ↓
- Independent replicationscount · ↑
- Within-lab CVcv · ↓
Changelog
Every methodology revision is versioned. Scores on affected runs are recomputed on release.
| Version | Released | Summary |
|---|---|---|
| 1.0.0(current) | 2026-04-20 | Initial public methodology. Per-track scores are raw; composite is not yet computed until cross-lab replication data is available. Initial release. Tracks, metrics, tasks, and control vocabulary established. |