Female Doctors May Face Unfair Rating Penalties, Study Finds

People increasingly read online reviews before choosing a doctor, with a rating below 3.7 out of 5 often considered a red flag. These reviews have stakes for physicians’ ability to serve patients, since research shows that patients trust reviews more than government ratings and even more than patient experience surveys.

But written reviews reflect unconscious biases around gender, according to new research by Peter Ubel, the Madge and Dennis T. McLawhorn University Professor at Duke University’s Fuqua School of Business, with Duke Ph.D. Farrah Madanay (the study lead), and Duke professor M. Kate Bundorf.

In a new paper published in JAMA Network Open, Ubel and colleagues found that female primary care doctors are disproportionately judged for their interpersonal manners in online reviews, and for that, they are more likely to receive negative ratings compared with their male colleagues.

“Negative comments on female physicians' personalities are more common,” Ubel said.

The researchers also found that female surgeons tend to receive lower star ratings than their male colleagues for negative comments on their technical skills.

As healthcare increasingly ties physician pay to patient ratings, their research signals that a troubling gender bias could cost female doctors income.

Using AI to classify hundreds of thousands of online reviews

Ubel and colleagues studied 345,053 reviews written between 2015-2020. The reviews focused on 167,150 physicians, almost 80% of whom were primary care physicians (PCPs) — family doctors, general medicine practitioners, and pediatricians — and more than 20% were surgeons.

They sourced the reviews from Healthgrades.com, a commercial physician rating and review website where patients can rate their doctors on a 1 to 5 star scale, with or without a written review.

Ubel and colleagues hypothesized that most patients evaluate doctors along the two dimensions of “interpersonal manners” (character, behavior, communication) and “technical competence” (expertise, treatment approach, outcomes).

They wondered whether the medical setting would reflect common gender stereotypes discovered by social psychology, and specifically whether male physicians would be expected to be higher in competence and female physicians higher in warmth — a bias that generally harms women’s professional outcomes.

“We thought that having good ‘bedside manners’ would play out along the lines of gender stereotypes, where people just expect women to be warmer and more caring and will be professionally punished for not meeting that expectation,” Ubel said.

The researchers employed a natural language processing algorithm (RoBERTa) to classify the reviews.

They first manually coded a sample of reviews, instructing the AI model on how to spot interpersonal manners versus technical competence in the language of the written reviews. 

“For example, an ‘interpersonal manners’ comment could be something as straightforward as ‘the doctor was incredibly nice’ or ‘listened to me carefully’ or ‘explains things to you,’ Ubel said. “Or, ‘the doctor was rushed,’ which is negative but still ‘interpersonal’, because it wasn't whether they were good at diagnosing, or about their medical knowledge.” 

Women receive lower ratings than men when they lack warmth

To further strengthen the reliability of their results, Ubel and colleagues manually evaluated a few hundred reviews they had withheld from training the AI model, to compare the human-processed results with the results of the machine learning tool.

After the test confirmed that the pre-trained model was highly accurate in its classifications, the researchers moved on with the AI-powered analysis of the sample of 345,053 reviews.

The goal was to identify whether female/male PCPs and surgeons had received positive, negative, or no evaluations at all on interpersonal manners and technical competence. They also examined star ratings, lumping them into either bad scores (1-3 ratings) or good ones (4-5 star ratings).

The main findings mostly confirmed the gender bias hypotheses:

• Female physicians are roughly 19% more likely than their male colleagues to receive comments about their interpersonal manner and are 22% more likely for that comment to be negative.
• Female physicians who received a positive comment about technical competence are 18% less likely to receive a high star rating (female PCPs, in particular, have almost 40% lower chances of receiving high ratings).
• Female surgeons are about 33% less likely than men to receive high star ratings when they receive a negative comment about their technical competence.

Why biased reviews may impact female physician pay

The researchers concluded that patients seem to harbor gender bias in medical settings, and these perceptions filter through written reviews and star ratings, especially for primary care physicians.

Since women and PCPs are “expected to be warm,” it’s no surprise that “seemingly cold female PCPs” are reviewed more harshly, the researchers note.

This could also explain why in fields where technical competence is valued, women may be rewarded for breaking norms. Female surgeons are comparatively less penalized than PCPs, as women excelling in a “male-dominated” specialty like surgery seem to be rewarded for deviating from the social norm.

“Primary care physicians are going to be judged in a more feminine way — about their personality — while surgeons are judged about their competence, a trait that gender stereotypes often assign to men,” Ubel said.

This study may have implications for healthcare policy, he said, because insurance companies and the federal government are increasingly tying reimbursement — how much they pay healthcare providers — based on measures of quality, which include written reviews and star ratings.

“We need to be very cautious about tying too much income to a potentially flawed measure,” Ubel said. “You may end up with less money going to women physicians who are disproportionately punished for personality and manners.”