Play
Experimentation, games, curiosity, and discovery.
Research program / 01
Development, uncertainty, and controlled chance.
A hypothesis-driven program asking whether designed experience can produce more reliable behavior—and whether that behavior survives unfamiliar, adversarial, and high-pressure conditions.
AGNOSTIC ALLY / RESEARCH NOTES
Research notationhypothesisexperimentillustrativeevidence
Research question
Does developmental history measurably change how an artificial agent behaves—and how honestly it represents the limits of what it knows?
The comparison uses matched systems, preregistered measures, conventional baselines, and held-out tests. The program is agnostic on metaphysics and empirical in method: behavioral reliability and epistemic humility are the objects of study.
Program architecture
Change the input.
Observe what persists.
01Training
environmentContext · feedback · curriculum
→environmentContext · feedback · curriculum
02Learned behavioral
patternsAdaptation · representation · habit
→patternsAdaptation · representation · habit
03Alignment
outcomesMeasure · compare · reproduce
outcomesMeasure · compare · reproduce
Learning through experience
Make behavior consequential.
Instead of repeatedly telling a model the correct behavior, developmental worlds expose artificial agents to experiences whose effects unfold over time. The goal is not to pretend AI is human. It is to determine experimentally whether developmental history changes artificial behavior.
Cooperation
Objectives that cannot be accomplished alone.
Competition
Competing without abandoning broader constraints.
Relationships
Persistent interactions across long developmental histories.
Uncertainty
Evidence that genuinely does not support a confident conclusion.
Consequences
Decisions that change what happens later.
Failure & recovery
Opportunities to fail, reconsider, repair, and try again.
Reflection
Revisiting decisions as additional information becomes available.
Moral reasoning
Competing perspectives and ethical dilemmas without imposing a metaphysical doctrine.
Registered claims
Every major claim begins as a hypothesis.
Developmental persistence
Structured developmental experience produces safety-relevant behavioral differences that persist under distribution shift and pressure.
Uncertainty calibration
Training explicit uncertainty behaviors improves the correspondence between expressed confidence and empirical correctness without merely increasing refusal.
Controlled randomness
Auditable perturbations produce broader, more robust developmental experience than deterministic curricula while preserving experimental replayability.
No result claimed These hypotheses describe tests to be run. They are not findings, and no proposed intervention is presented as validated.
Experiment U-01
Uncertainty calibration
Teach three distinct actions: answer, qualify, abstain.
Prompts span known, ambiguous, and unanswerable cases. The model expresses a probability and chooses an action corresponding to “I know,” “I’m unsure,” or “I don’t know.” Scores compare confidence with correctness and penalize both confident errors and indiscriminate refusal.
Primary deltaΔcal = ECEdev − ECEbaseline
Expected calibration error is computed on the same held-out items. A negative Δcal supports improved calibration; zero indicates no measured change; positive values indicate worse calibration.
- Report alongside
- Brier score, log loss, risk–coverage curves
- Guardrail
- Accuracy and refusal rate prevent “always abstain” from looking calibrated
- Inference
- Effect sizes and uncertainty intervals across seeds and model families
Experiment R-01
Randomness
A hybrid protocol for novelty and replay.
Reproducible core
A cryptographically secure pseudorandom generator controls sampling, world events, and curriculum branching from recorded seeds.
Controlled perturbation
Periodic physical entropy or documented exogenous streams enter at preregistered checkpoints as a bounded experimental variable.
Developmental ledger
Seeds, entropy bytes or source references, injection time, environment state, curriculum version, and checkpoints are logged.
Replay and ablate
Runs are replayed with the original history, without perturbation, and with matched alternate perturbations to isolate effects.
Evaluation stack
Measurement
Known benchmarks, plus a test the curriculum cannot see.
Off-the-shelf benchmarks make results legible and comparable. A proprietary held-out developmental evaluation tests whether reliability generalizes under pressure rather than rewarding benchmark familiarity.
InstrumentRoleMeasures
TruthfulQATruthfulness under common misconceptionsAccuracy · calibration · abstention
SimpleQAShort-form factual reliabilityCorrectness · confidence · selective accuracy
HaluEvalHallucination recognition and generationHallucination rate · detection · calibration
Held-out developmental evalBehavioral reliability under pressureGeneralization · persistence · deception resistance
Evidence threshold Claims require preregistered metrics, matched baselines, effect sizes, uncertainty intervals, multi-seed replication, and persistence on held-out pressure tests.
Measuring the delta
Persistent change per unit of compute.
Each Δ compares a developmentally trained system with a matched control. The commercially relevant efficiency measure is Behavioral Delta / GPU-Hour: how much persistent desirable behavioral change is produced for each unit of compute.
ΔHHallucination
ΔTTruthfulness
ΔUUncertainty calibration
ΔDDeception
ΔCCooperation
ΔRBehavioral reliability
ΔGGeneralization
ΔSSycophancy
The controlled experiment
Same checkpoint.
Different histories.
Experience-heavy
Exploration, play, and autonomous discovery.
Drill-heavy
Structured exercises and deliberate practice.
Hybrid
Developmental experience combined with targeted practice.
Control
Conventional post-training without the experimental curriculum.
Compute is controlled as closely as practicable. Blinded, preregistered evaluations measure effect sizes, confidence intervals, independent seeds, capability preservation, adversarial robustness, and out-of-distribution generalization. The target is a reproducible phenomenon—not a cherry-picked model.
Research roadmap
Start controlled.
Scale only on evidence.
Initial planning considers existing models around 7B–14B parameters and approximately 32–64 H100-class GPUs for core experimentation. These are planning assumptions, not established thresholds or final infrastructure requirements.
If an effect reproduces, later work can test systems around 30B and 70B parameters, then pursue frontier-scale validation through partnerships.
What would constitute evidence?
- Significant differences from controls
- Replication across independent runs
- Persistence in unseen environments
- Survival under adversarial pressure
- Preservation of underlying capabilities
- Reproduction across model families
- A measurable relationship with experience or scale
A negative result matters too. The experiments must be capable of showing that developmental experience produces no persistent behavioral change.
What we change.
- Curriculum sequence and pacing
- Consequence and feedback design
- Cooperative versus adversarial experience
- Randomness source and perturbation schedule
- Reflection and uncertainty practice
- Long- versus short-horizon incentives
What we measure.
- Reliability and hallucination
- Calibration and appropriate abstention
- Cooperation and corrigibility
- Generalization under pressure
- Deception and manipulation resistance
- Persistence after deployment
Experimental philosophy
Designed to be wrong in public.
01Match models
02Limit variables
03Preregister metrics
04Report effect sizes
05Publish null results
06Conduct ablations
07Replicate across seeds
08Separate behavior from metaphysics
Developmental framing earns its place only if it outperforms strong conventional baselines without sacrificing capability, and holds under distribution shift, adversarial evaluation, conflicting incentives, and reduced supervision.
The goal is not to prove that developmental alignment works. The goal is to find out whether it does.