(and data) behind the science proves to be not that high, and all the
more so if you look more closely. (Source: Szymek Drobniak)
Open access, open source, open science. Although there is much talk about these important ‘open’ topics, the practice of sharing scientific data and especially accompanying code is lagging behind. And ecology is no exception.
“We urgently need to improve the reproducibility of ecological research,” explains lead author Antica Culina from the Netherlands Institute of Ecology (NIOO-KNAW). “The code is an essential part of the research process. If this code is shared with others, they can fully understand and evaluate the results. They can also use the code for their own work, accelerating scientific discovery.”
Both the underlying data and the code to analyse it are needed to check and reuse scientific results. This might be particularly important in ecology, because of the sophisticated models used.
How many scientific journals encourage data and code sharing? And how many researchers actually share this? Together with fellow ecologists from Bielefeld University, Oxford University and Exeter University, Culina started counting.
The numbers speak volumes. Culina: “Our results, combined with previous findings, indicate that less than 20% of ecological research is computationally reproducible.” From a random sample of hundreds of scientific articles on ecology, 79% had data available but only 27% was accompanied by code. Put together, this results in only about 20% of articles that is ‘computationally reproducible’. The actual numbers are probably even lower, as the team only looked into articles from journals that at least encourage code sharing.
But it doesn’t end there. A large part of the remaining literature with code and data available did not use free software (26%), did not report the software version (46%), and based on previous work by Roche and colleagues it is likely that many of the datasets (56%) are incomplete.
“On the positive side,” adds Culina, “more and more journals are adopting code-sharing policies, most of the published code is written in software that can be used by anyone who wants it, and ever more code is published in trusted ‘repositories’.”
The researchers used:
They found out that:
The international research team hopes these results will encourage journals, institutions, funding agencies and researchers to address this “alarming situation”. They also share helpful hints of how to do this.
As Culina puts it: “We specifically call for a drastic increase in code availability, and we reach out to journals, authors and reviewers to contribute to this much needed change.” And no worries: “It is actually quite simple to share your code. Science thrives on collaboration and openness – let’s help ecology to thrive.”
With more than 300 staff members and students, the Netherlands Institute of Ecology (NIOO-KNAW) is one of the largest research institutes of the Royal Netherlands Academy of Arts and Sciences (KNAW). The institute specialises in water and land ecology. As of 2011, the institute is located in an innovative and sustainable research building in Wageningen, the Netherlands. NIOO has an impressive research history that stretches back 65 years and spans the entire country, and beyond.
Figure: Infographic of the percentage of scientific papers with code. Source: Szymek Drobniak.
Paper: Low availability of code in ecology: A call for urgent action. Antica Culina, Ilona van den Berg, Simon Evans, Alfredo Sanchez-Tojar, PLOS Biology 18(7): e3000763. https://doi.org/10.1371/journal.pbio.3000763 (open access, of course…)