What is lab reproducibility?
Lab reproducibility is the ability to obtain consistent results when an experiment is repeated — by the same scientist, by colleagues in the same lab, or by another lab following the same protocol. It depends not only on the written method but on materials, equipment, data analysis, and the hands-on technique of the people doing the work.
Also known as Experimental reproducibility · Research reproducibility
Reproducibility, replicability, and repeatability
These words are used inconsistently across disciplines, and sometimes with opposite meanings, so it helps to say which definitions you're using. Two framings are especially common.
In analytical and quality control labs, precision is described in three tiers, following metrology standards and the ICH guideline on analytical method validation:
| Term | Conditions that vary | Typical lab question |
|---|---|---|
| Repeatability | Nothing deliberately — same analyst, instrument, lab, and short time interval | Do back-to-back runs of this assay agree? |
| Intermediate precision | Day, analyst, or equipment within the same lab | Does the method hold up when a different analyst runs it next week? |
| Reproducibility | The laboratory itself | Will a partner lab or a second site get the same answer? |
In research more broadly, the U.S. National Academies of Sciences, Engineering, and Medicine drew a different line in their 2019 report on the topic: reproducibility means getting consistent results from the same data using the same code and analysis, while replicability means getting consistent results in a new study that collects its own data.
This page uses "lab reproducibility" in the everyday bench sense: can the experiment be run again — tomorrow, by someone else, or somewhere else — and produce the same result?
How big is the problem?
Concern about reproducibility in the life sciences is well documented:
- In a 2016 Nature survey, more than 70% of researchers said they had tried and failed to reproduce another scientist's experiments, and more than half had failed to reproduce their own.
- In 2012, scientists at Amgen reported that they had been able to confirm the findings of only 6 of 53 landmark preclinical cancer studies they attempted to reproduce.
- A 2015 analysis in PLOS Biology estimated that about US$28 billion a year is spent in the United States on preclinical research that is not reproducible.
Each of these figures has its caveats — survey responses are self-reported, and a failure to reproduce doesn't prove the original finding was wrong. But the direction is consistent, and the cost lands where it hurts: drug programs built on results that don't hold, months spent troubleshooting methods, and assays that work at one site and fail at the next.
Why experiments fail to reproduce
Irreproducibility rarely has a single cause. The usual contributors fall into a few groups:
- Under-specified protocols. Methods sections compress hours of work into a paragraph. Instructions like "mix gently," "incubate overnight," or "wash thoroughly" leave room for each person to do something slightly different.
- Biological reagents and reference materials. Antibodies that don't bind what they're supposed to, cell lines that are misidentified, contaminated, or drifted over many passages, and lot-to-lot variation in serum, enzymes, and media.
- Equipment and environment. Uncalibrated pipettes, incubator temperature gradients, differences between instrument models, and ambient conditions that nobody recorded.
- Technique. Pipetting angle and speed, timing between steps, how a pellet is resuspended, how cells are handled — skills that are hard to write down and rarely measured.
- Analysis and reporting. Small sample sizes, flexible analysis choices, selective reporting of experiments that "worked," and missing raw data.
- Incentives. Pressure to publish novel, positive results leaves little reward for careful replication or for reporting failures.
The technique gap
Of these causes, technique is the one most often overlooked, because it is invisible in the record. Two scientists can follow the same written protocol, record the same steps, and get different results — and nothing in either notebook explains why.
Experienced scientists accumulate a large body of unwritten know-how: which step is fragile, how a healthy culture looks, what to do when a reagent behaves oddly. That tribal knowledge transfers slowly, mostly through side-by-side work. It's common for a struggling method transfer between sites to be resolved only when someone travels to watch the originating team run the assay in person.
The same gap affects onboarding. New team members often need weeks of supervised practice before their results can be trusted, and the knowledge that makes an expert reliable leaves with them when they move on.
How labs improve reproducibility
No single fix solves reproducibility, but labs that take it seriously tend to work on several fronts at once:
- Write protocols for execution, not publication. Detailed, versioned, step-by-step procedures — ideally as digital work instructions — with explicit parameters, timings, and acceptance criteria instead of judgment words.
- Authenticate and track materials. Verify cell line identity, validate antibodies for the application, record lot numbers, and use research resource identifiers (RRIDs) when publishing.
- Control equipment and environment. Keep calibration and maintenance current, and record conditions that plausibly affect the result.
- Plan analyses in advance. Pre-specify sample sizes and analysis plans, keep raw data, and share data and code where possible.
- Capture how the work was actually done. Record which steps were performed, by whom, when, and with which materials and instruments — and note every departure from the protocol. Execution observability makes this practical without asking scientists to stop and write everything down.
- Treat technique as a skill to verify. Have new team members demonstrate key procedures to a defined standard, and compare results across analysts as a routine check.
Reproducibility in regulated labs
In GxP environments, reproducibility is formalized rather than left to good intentions. Analytical methods are validated for precision, including repeatability and intermediate precision, before routine use. Methods moving between sites go through a documented transfer with predefined acceptance criteria. System suitability tests confirm that each run is fit for purpose, and out-of-specification results trigger formal investigations.
Even so, regulated labs face the same root problem as research labs: the written procedure is controlled, but the hands-on execution of it often isn't observed. When results diverge, the investigation becomes a deviation that has to be explained with whatever evidence the record contains.
Where QURE fits
QURE Science addresses the execution side of reproducibility. AI-guided protocol execution gives each scientist the same step-by-step guidance with in-line deviation detection, and every run produces a structured record of how the work was performed. Telepresence lets an experienced scientist observe and coach a run at another site in real time, which shortens technique transfer. QURE integrates with ELN, LIMS, robotics platforms, and simulation engines, so execution data sits alongside the results it explains.
Frequently asked questions
What is the reproducibility crisis?
The reproducibility crisis refers to the widespread finding that many published scientific results, particularly in biomedical and social sciences, cannot be reproduced when other researchers repeat the work. Surveys, systematic replication projects, and industry reports from the 2010s brought the problem to prominence and prompted changes in publishing, funding, and laboratory practice.
What is the difference between reproducibility and replicability?
It depends on the field. In the 2019 National Academies report, reproducibility means obtaining consistent results from the same data and code, while replicability means obtaining consistent results in a new study with new data. In analytical labs, reproducibility usually means agreement between different laboratories running the same method.
What are the main causes of irreproducible lab results?
Common causes include under-specified protocols, variable or misidentified biological reagents, uncalibrated equipment and unrecorded environmental conditions, differences in hands-on technique, and flexible data analysis or selective reporting. Most failures involve several of these at once, which is why fixes need to address materials, methods, execution, and analysis together.
How can a lab measure its own reproducibility?
Useful measures include agreement between repeat runs, the spread of results across analysts, days, and instruments, control-chart trends for reference standards, the success rate of method transfers to other sites, and how often runs have to be repeated. Tracking these over time shows whether changes to protocols or training are working.
Sources
- Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454.
- Begley, C. G. & Ellis, L. M. (2012). Raise standards for preclinical cancer research. Nature 483, 531–533.
- Freedman, L. P., Cockburn, I. M. & Simcoe, T. S. (2015). The economics of reproducibility in preclinical research. PLOS Biology 13(6): e1002165.
- National Academies of Sciences, Engineering, and Medicine (2019). Reproducibility and Replicability in Science.
Related terms: Execution observability, Tribal knowledge, Digital work instructions, GxP, Time to competence. Browse the full glossary, read field notes in the QURE Journal, or request a demo to see execution intelligence on your own procedures.