Two recent studies highlight the potential for artificial intelligence tools to be less accurate in some patients than others, if the tools are not trained properly.
It's well known that if AI tools are trained on data collected from unequal proportions of patients from various demographic groups, they have a harder time making an accurate diagnosis in minority groups that aren't well-represented in the training set.
But in the current analysis, the models sometimes performed worse in one demographic group even when the sample sizes were comparable, researchers reported in Cell Reports Medicine.
The reason may be that some cancers are more common in certain groups, so the models become better at making a diagnosis in those groups. As a result, the models may have difficulty diagnosing cancers in populations where they aren't as common, the researchers discovered.
In addition, subtle molecular differences may exist in biopsy samples from different demographic groups, and AI may detect those differences and use them as a proxy for cancer type, potentially making it less effective at diagnosis in populations in which these mutations are less common.
"We found that because AI is so powerful, it can differentiate many obscure biological signals that cannot be detected by standard human evaluation," study leader Kun-Hsing Yu of Harvard Medical School said in a statement.
As a result, the models may pick up signals that are more related to demographics than disease - and inferring demographic information from pathology slides could affect their diagnostic ability across groups.
Together, Yu said, these explanations suggest that bias in pathology AI stems not only from the variable quality of the training data but also from how researchers train the models.
When his team applied a new framework to the models they'd tested, it reduced the diagnostic disparities by some 88%, they said.
"We show that by making this small adjustment, the models can learn robust features that make them more generalizable and fairer across different populations," Yu said.
The finding is encouraging, he added, because it suggests that bias can be reduced even without training the models on completely fair, representative data.
In a separate study published in PLOS Biology, researchers found that even with broad samples of bacterial populations, bias may be blocking the potential of AI to predict and combat antibiotic resistance.
No comments:
Post a Comment