Biomedical Innovation for Everyone, Bias-Free
Professor Londa Schiebinger discusses bias and how to remove it from research practices.
“It’s called implicit bias. No one intends to be biased; they just have assumptions from their own experiences and those assumptions don’t really cover everyone.”
Sex/Gender Biases
Gender bias is not the only issue, said Schiebinger, who is also the director of Gendered Innovations in Science, Health & Medicine, Engineering, and Environment [5], and John L. Hinds Professor of History of Science at Stanford University (Figure 3). Age, race, and culture are also important factors to consider. “The U.S. Food and Drug Administration (FDA) today asks that studies have a certain percent population of African Americans, but this rarely rises to statistical significance unless you’re really focusing on that demographic.” Bias is now more transparent with the addition of an FDA online Drug Trials Snapshots [6], or “dashboard” as Schiebinger describes it, which reveals the sex, race, and age distribution of each study. That is a positive step, but examination of the inclusiveness on the dashboard reveals that many studies still fall short, she said. As a 65-plus person, she has scanned the dashboard to check drug studies for participants in her age group, and often finds them lacking. “And yet who takes more drugs?” she asked.
“Everybody thinks it’s more expensive to do inclusive research, but if you make a drug that is developed on a male pipeline all the way through, it’s going to fail when you get to using it on real live women,” said Schiebinger, asserting that the same holds true for research that underrepresents any population. “So, you can say it’s more difficult or expensive, but really for whom?”
How to Cut Bias
The answer is to eliminate bias in biomedical technologies. That might not be simple, but it is possible and worth the effort, Schiebinger reiterated. “It’s just like doing any research: you need to do a literature review, you need to consider what methods you’re going to use, what your target population is – maybe you can’t target everyone, but you want to make sure you have included people who need this technology, this medical device or this drug.”
She recommended that biomedical innovators take a look at the “Detailed Methods” section of the Gendered Innovations 2 report for a step-by-step approach to identifying the potential sources of bias in design, data collection, and analysis; and to explain how bias was considered. That section also includes a wealth of basic literature to further describe each step. “We also have new methods to demonstrate how to consider this in advance, because it’s all about designing the research correctly from the very beginning,” Schiebinger said. “So many people just do something and then plan to fix it later, but that is unnecessary and expensive.”
When it comes to reducing bias in AI algorithms and models, Zou urged innovators to employ training databases that are appropriately diverse, but noted that is not an easy task because so few are available. “Creating a training dataset is a very expensive process, so once somebody does it, a lot of companies and researchers will use the same one. My group right now is investing a lot of resources to come up with and curate more diverse datasets,” he said. One that his group has just curated is the Stanford Diverse Dermatology Image Dataset, which will be available soon. He described it as “a large collection of skin images, which are confirmed with biopsy samples. And images come from darker-skinned patients.” In addition, innovators must do comprehensive testing of AI algorithms once they are deployed and over time to make sure it is functioning properly.
Although Schiebinger is pleased with the increasing awareness about bias in research and biomedical technologies, it requires constant vigilance on all three pillars of the science infrastructure [9]. “We need the funding agencies on board at the beginning of the project to help make sure research is inclusive. We need peer-reviewed journals on board at the end of the project to make sure the manuscript took bias into account, or it shouldn’t be published,” she said, noting that both funding agencies and journals have been making good progress. “The third pillar of the scientific infrastructure is universities, and we just aren’t doing our part yet. Methods to reduce bias are not taught systematically in the engineering curriculum, and the medical curriculum does not include everything that is important for sex and gender, let alone race and ethnicity. These are variables that need to be part of undergraduate, graduate, and professional preparation.”
She added, “I think we’re in a period of huge change surrounding sex, gender, and diversity in research and design. We need people to continue working on this, and calling attention to it.”
Photo courtesy of Linda Cicero, Stanford University
Article written by Leslie Mertz, Ph.D., IEEE Pulse