We Need to Talk More About Bad Data
When Gavrielle Untracht examined a dataset of scanning images from a colleague, one thing stood out: a portion of the scans was unusable. They were simply too poor in quality and had to be discarded. And pointing this out is not offensive—it’s completely normal. Still, researchers rarely talk about “bad data.” That has to change, says Gavrielle Untracht
As physicist and assistant professor Gavrielle Untracht began her position at the Skin Cancer Research Center, one of her first tasks was to analyze an existing dataset from a clinical study. The study originally aimed to investigate whether an active substance could penetrate the skin via a cream. The initial analyses showed no effect—the cream appeared to make no difference, and subsequent scans using an OCT scanner also showed no effect.
From a clinical perspective, it is a reality that not all scans produce clear and readable images. However, it is rare for someone like Gavrielle Untracht to address the premise of bad data and treat it as a subject in a scientific article. In general, bad data is not something widely discussed in the research world. It is simply filtered out to avoid distorting the overall analysis. That is why it is quite unusual that Gavrielle Untracht has now published an article about bad data.
But since we know such data exists, it should also be declared and included in final research papers, she argues:
“Why is it that we as researchers should not talk about bad data? These scans were performed by a doctor on a patient who contributed their body and time. The scans with poor outcomes should be mentioned in a study, even if they are excluded from the analysis. If you have a deviation, you should report it,” says Gavrielle Untracht.
She insists on this, not least out of respect for the volunteer participants who take time to support research:
“Otherwise, they may feel that their contribution was entirely in vain, and that is not respectful,” she says.
If There Is No Story
Gavrielle Untracht decided to examine the bad data to see whether something could be learned from it—whether it could provide useful insights, since such data exists in almost all research projects.
In journalism, there is a saying: “If there is no story, maybe that’s the story.” The dataset of failed or unclear scans could neither confirm nor refute the original hypothesis behind the study.
Instead, Gavrielle Untracht turned her analytical focus to the discarded scans and used them as a starting point to define what makes a scan poor or unusable.
“There was no existing material that defined what constitutes a good or bad scan. It is important for all of us to be able to produce high-quality scans. If I can use the bad dataset to define and establish a standard for a good scan, it can help future research projects,” she says.
An Ethical Perspective
A noble and unusual project—but also one that did not receive the easiest reception during peer review:
“We received quite a bit of criticism for focusing on bad data. The reviewers insisted that we also publish and include the successful, good data, even though no real conclusions could be drawn from them,” says Gavrielle Untracht.
However, the article was eventually accepted and published. She hopes that in the future it can serve as a reference point for other researchers when evaluating good and bad scanning data.
“There is also an ethical dimension: the efforts of research participants should count. Their contributions should not be lost without explanation. If we want to conduct reliable medical studies, it requires high-quality data, clear standards, and transparency about errors. And that requires that we are also willing to talk about bad data", she says.
The published article kan be read here.
