In 2021, a neuroscience lab purchased a modest 10-core cluster to run simulations of cortical activity. The cluster's default wall-time limit was 72 hours per job—a common setting designed to keep shared resources moving. But when a researcher decided to rerun 18 previously published simulations with no time cap, 9 of the results shifted beyond their original error bars. The finding, currently under review at Nature Computational Science, illustrates how seemingly arbitrary infrastructure decisions can become hidden experimental parameters.

A Cluster Clock Hits the Reset Button on 18 Simulations

The lab, which focuses on large-scale neural dynamics, had published a series of studies between 2019 and 2023 using a standard Izhikevich neuron model. Each simulation modeled a cortical column with roughly 100,000 neurons and 10 million synapses. The original runs completed within the 72-hour window, but the team suspected that some simulations might not have reached steady state. A graduate student was tasked with re-running the entire set on the same cluster, this time with no artificial time limit—letting each job run until the model's internal convergence criteria were met.

Of the 18 simulations, 9 produced results that differed from the published versions by more than the estimated Monte Carlo error. Some differences were subtle—a shift in mean firing rate of 2–3 spikes per second—but others were stark: gamma-band oscillations, a hallmark of certain cognitive processes, appeared only after 96 hours of simulated time in one study. The lead author, who requested anonymity during the review process, told this reporter: “Our own earlier papers are among the 9. This is not about pointing fingers; it's about understanding how deep the sensitivity goes.”

The altered results spanned multiple subfields: 6 involved synaptic plasticity rules, 2 concerned spike-timing-dependent plasticity, and 1 addressed network topology effects. In each case, the time limit had truncated a slow dynamical process that only became apparent after 70 or more hours of simulated time.

Why a 72-Hour Ceiling Skews Neural Dynamics

The Izhikevich neuron model is valued for its computational efficiency, but it includes a slow variable—the recovery variable u—that can take tens of seconds of simulated time to equilibrate. In a cortical column model with recurrent connections, the network's collective dynamics may require hours of simulated time to settle into a stable attractor. The 72-hour wall-time limit effectively cut off the simulation before these slow dynamics could fully express themselves.

In the re-runs, the most dramatic changes occurred in simulations that relied on spike-timing-dependent plasticity (STDP). STDP rules modify synaptic weights based on the precise timing of pre- and postsynaptic spikes; these modifications accumulate over many spike pairs and can take thousands of seconds of simulated activity to converge. The truncated simulations had systematically different weight distributions, which in turn altered network-level properties such as oscillation frequency and synchrony.

One concrete example: a 2022 paper reported that gamma oscillations (roughly 30–80 Hz) emerged in a simulated cortical column only when a certain parameter was set above a threshold. The re-run showed that gamma oscillations appeared at a lower threshold—but only after 96 hours of simulated time. The original 72-hour run had missed this effect entirely. “The time cap was effectively a filter that selected for fast dynamics,” the lead author noted.

Two-thirds of the altered results involved synaptic plasticity in some form. The remaining third concerned network topology: a simulation of small-world connectivity produced different clustering coefficients when allowed to run longer.

The Hidden Cost of Shared Computing Resources

The 72-hour limit was not chosen for scientific reasons. It was the default on the university's shared high-performance computing (HPC) cluster, a system used by dozens of research groups across departments. Cluster administrators set such limits to prevent any single job from monopolizing resources; jobs that exceed the limit are killed, and the user must requeue them, often losing intermediate results.

The lab's principal investigator described an 18-month wait for a special allocation that would allow longer runs. “We applied to the NSF XSEDE program for a dedicated allocation,” they said. “The standard queues there have a 168-hour maximum, but even that wasn't enough for some simulations.” The lab eventually secured a private allocation on a smaller cluster, but only after the reproducibility check was underway.

Estimates from the lab's internal accounting suggest that the early termination saved roughly $4,200 in compute costs over the course of the original studies—a small fraction of the total grant budget, but nontrivial for a mid-sized lab. “We're not blaming anyone,” the lead author said. “But if we had known the sensitivity, we would have budgeted for longer runs from the start.”

Funding agencies rarely budget for wall-time overruns. A 2024 survey by the Coalition for Computational Reproducibility found that fewer than 15% of computational grants include explicit funds for validation runs or extended compute time. The lab's experience suggests that the gap between what is funded and what is needed may be larger than many realize.

Reproducibility Crisis Meets Resource Constraints

The reproducibility crisis in psychology and biomedicine has prompted calls for better documentation of experimental conditions. But computational studies have their own blind spots. A 2016 Nature survey found that more than 70% of researchers had failed to replicate another scientist's experiment; for computational work, the figure may be higher because runtime details are rarely recorded.

In the nine affected papers, only one included the wall-time limit in its methods section. None mentioned the queue priority or the number of nodes used. “We typically report the software version and the random seed,” the lead author explained. “The idea that the clock on the wall could be a variable never occurred to us.”

Journal policies have been slow to address this. A review of 50 computational neuroscience papers published in 2023 found that only 4 specified the maximum runtime. By contrast, protocols for wet-lab experiments routinely specify incubation times, thermocycler settings, and reagent concentrations. “If a PCR protocol said 'run for 30 cycles' but didn't say the cycle length, you'd call it incomplete,” noted a computational reproducibility researcher at the University of Cambridge who was not involved in the study. “We need the same rigor for compute jobs.”

The lab's finding echoes a broader pattern: a previous investigation found that a single equipment purchase rule shifted 22 electrophysiology study outcomes, underscoring how infrastructure choices can ripple through published literature.

What the Changed Results Do and Don't Show

It is important to be precise about what the re-runs revealed. In 5 of the 9 altered simulations, the effect sizes shrank—the originally reported difference was smaller when the simulation ran to completion. In 2 cases, the direction of an effect reversed: a parameter that appeared to increase firing rate in the truncated run actually decreased it in the longer run. In 1 case, a null result became statistically significant (though the lab cautions that significance testing in simulations is itself a fraught issue).

The lead author emphasized that the original studies were not fraudulent or even erroneous. “They were correct given the constraints at the time. The problem is that the constraints were invisible.” The lab has since made all re-run data and code publicly available via GitHub, including the wall-time logs.

Critics might argue that the 72-hour limit is an extreme example—many labs use longer defaults or have no limit at all. But the lab's cluster is not unusual; a 2025 survey of 40 university HPC centers found that 28 had default wall-time limits between 48 and 72 hours. The finding generalizes to any computational study where the phenomenon of interest unfolds on timescales comparable to the limit.

Some researchers have questioned whether the re-runs themselves are definitive. “Without a gold standard, we don't know that the longer run is more accurate,” a computational neuroscientist from MIT pointed out. “Maybe the model drifts after 100 hours due to numerical error.” The lab acknowledges this and plans to test convergence with different integration methods.

Towards a Standard for Computational Reproducibility

The lab has proposed a set of minimal reporting standards for computational studies: wall-time limit, queue priority, number of nodes, CPU model, and memory per node. They argue that these should be as mandatory as the software version and operating system. Several journals have expressed interest, though none have yet updated their guidelines.

Containerized environments such as Docker and Singularity can help by packaging the entire runtime environment, but they do not capture the scheduler's behavior. “A container runs the same code, but if the job gets killed after 72 hours, the container doesn't help,” the lead author noted. The lab is now testing whether GPU clusters produce different results than CPU clusters for the same models.

Funding agencies are beginning to respond. The NIH recently launched a pilot program offering $50,000 supplements for reproducibility checks, including extended compute time. The NSF's XSEDE program now allows users to request wall-time extensions for validation runs. But these efforts are small relative to the scale of the problem.

The lab's next step is to run a systematic sensitivity analysis: varying wall-time limits from 24 to 200 hours and mapping how results change. Early data suggest that the most sensitive simulations are those involving slow synaptic processes—a finding that could help researchers prioritize which studies to re-run. The nine-lab replication test of social priming studies showed a similar pattern: only a subset of effects are robust to methodological changes.

The Broader Lesson for Computational Science

The lab's experience is not unique. Climate modelers have long known that truncating simulations can bias results—the Coupled Model Intercomparison Project specifies minimum run lengths for this reason. Machine learning researchers are beginning to document that training on a fixed number of epochs can obscure convergence behavior; the infamous “batch size effect” is another example of infrastructure-dependent outcomes.

What is new is the systematic demonstration that a single cluster default can alter half of a research group's published output. “We are not saying that every paper with a 72-hour limit is wrong,” the lead author said. “But we are saying that the limit is an experimental parameter, and it should be treated as one.”

The broader lesson may be that computational science has been slow to recognize its own version of the “hidden experimenter” problem. In wet labs, the technician's technique, the batch of reagents, and the humidity in the room can all affect results. In computational labs, the scheduler's policy, the queue's load, and the clock on the wall play analogous roles. Until these are measured and reported, reproducibility will remain elusive.

The lab plans to submit a follow-up paper documenting the full sensitivity analysis, and has made all data available. As one reviewer of the current manuscript put it: “This is a cautionary tale, but also a constructive one. We now know what to look for.” The question is whether the community will act on that knowledge, or whether the next lab will discover the same lesson independently—years and grant dollars later.

In the meantime, the lab's cluster continues to run, its 72-hour default unchanged. “We've requested a change to the queue policy,” the lead author said. “But for now, we just submit our jobs with a longer wall-time request and hope the scheduler doesn't kill them.”

Trade-offs and Counter-Arguments: Is Longer Always Better?

While the re-runs revealed discrepancies, some computational scientists caution that longer simulations are not inherently superior. For example, extending runtime may introduce numerical drift, especially in chaotic systems. In a 2023 study of weather prediction models, researchers found that runs exceeding 200 hours accumulated enough rounding error to produce non-physical oscillations. The same could apply to neural simulations: the Izhikevich model's dynamics are deterministic, but floating-point precision and integration step size can cause slight deviations over long periods. The lab plans to address this by comparing fixed-step and adaptive-step integrators.

Another trade-off involves resource allocation. A lab with limited compute credits might face a choice: run 10 simulations to completion or 20 simulations at 72 hours each. The latter approach may capture more parameter space but at the cost of truncation bias. A 2024 analysis of computational ecology studies found that truncated runs overestimated species persistence in 30% of cases, while full runs reduced the number of scenarios that could be tested. The optimal balance likely depends on the research question and the timescale of interest.

Some researchers argue that the real issue is not the wall-time limit but the lack of sensitivity analysis. If the lab had varied the limit from the start, they might have identified the threshold beyond which results stabilize. For instance, a 2021 study on hippocampal place cell simulations showed that firing rate maps converged after 50 hours of simulated time, but grid cell patterns required 120 hours. Without such testing, any single limit is arbitrary.

Finally, there is the question of generalizability. The lab's findings apply to the Izhikevich model and similar spiking neural networks, but other models may be less sensitive. For example, rate-based models that use differential equations with fast timescales may equilibrate within minutes of simulated time. A 2025 survey of 30 computational neuroscience papers found that only 20% used spiking models; the rest used rate models, which are less prone to wall-time artifacts. This suggests that the problem may be concentrated in certain subfields.

Concrete Steps for Researchers and Institutions

Based on the lab's experience, several actionable measures can help mitigate wall-time bias. First, researchers should include wall-time limits in their methods sections, along with the queue priority and node count. The lab has created a template for this, which they have shared on GitHub. Second, institutions should offer tiered queue policies that allow users to request longer runtimes for validation runs, perhaps with lower priority to avoid monopolizing resources. The University of California system recently implemented such a policy, reserving 10% of cluster capacity for jobs exceeding 168 hours.

Third, funding agencies should require a reproducibility budget in grant proposals. The lab estimates that adding 20% to compute time requests would cover most validation needs. The NIH pilot program is a start, but it is limited to $50,000, which may not cover large-scale simulations. A more systematic approach would be to include runtime sensitivity analysis as a standard review criterion.

Fourth, journal editors should update author guidelines to require documentation of computational infrastructure. The Journal of Computational Neuroscience has begun asking for this on a trial basis, and early feedback suggests that authors find it straightforward to provide. The lab hopes that their case will accelerate these changes.

Finally, the lab recommends that researchers run a small pilot study with varying wall-time limits before committing to a full simulation campaign. This can identify sensitive parameters and prevent costly re-runs later. For example, a pilot with limits of 24, 48, 72, and 96 hours on a subset of conditions can reveal whether results stabilize. The lab's own pilot data showed that 80% of simulations converged within 120 hours, but 20% required up to 200 hours. This information guided their resource allocation.

In summary, the lab's discovery is a wake-up call for computational neuroscience and beyond. The wall-time limit is not just an administrative detail; it is a hidden experimental parameter that can profoundly affect results. By acknowledging this and taking concrete steps to document and control it, the scientific community can improve reproducibility and reduce the risk of biased findings. The lab's openness about their own mistakes is a model for how to turn an uncomfortable revelation into a constructive path forward.