In 2013, your Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) performance was at an all-time low. You knew the scores were a fair reflection of your hospital; more than one patient said they were inclined to “walk off” a broken leg rather than see your surgeons, and you’re fairly certain your nurses were one overtime shift away from violent mutiny.
But it’s 2018 now, and after five years of dedicated leadership, administrative restructuring, and communication and safety training, your hospital hums like a well-oiled machine. You’ve turned the old lemon into a flashy sports car and everyone on staff is clearly proud of the feedback they’ve gotten from patients and peers alike. You eagerly check the latest HCAHPS scores, ready to see your effort in print, and you see that the scores are…fine.
It’s true you’re no longer at the bottom of the barrel, and you’re incredibly proud of the hard work your staff has put in, but as you stare at your scores, you’re stumped as to why you aren’t seeing more dramatic results. You’ve done everything right, your team has done everything you’ve asked and more, and your patients generally seem to leave both extremely satisfied and markedly healthier. Where are you going wrong?
Deceiving Data
Believe it or not, situations like these are all too common; hospitals that are excellent by all observable counts often don’t meet the mark on paper. This has been observed at organizations across the nation, from the smallest rural areas to booming urban locations. After researching this phenomenon, Hospital Quality Institute (HQI), published California HCAHPS Improvers Playbook, claiming a relationship between response rates and HCAHPS top-box scores. HQI observed, “Each 1-percentage-point increase in a hospital’s HCAHPS response rate is expected to result in a 0.5-percentage-point increase in the mean top-box score.” Knowing that our response rates outpace the national average, PRC scientists were intrigued by these results and studied them further.
Utilizing the July 2016 – June 2017 data from Hospital Compare, PRC replicated this analysis and successfully identified a similar positive correlation between HCAHPS scores and response rates for hospitals. It became clear that hospitals are suffering from lower scores as a result of lower response rates. Mathematically, if only the most and least satisfied patients respond to patient surveys, the average score decreases.
If these statistics surprise you, you’re not alone. When the patients who complete the HCAHPS survey truly represent the total population, the correlation found by HQI and PRC is unlikely, and it practically disappears when a response rate reaches 40%, well above the national average. However, because organizations with low response rates have data which may not reflect a truly representative sample, the correlation becomes more pronounced. Put another way, when your response rate is lower than the 30% to 40% range, chances are, your HCAHPS scores will correlate with your response rate.
What Can You Do?
The key to getting a more representative sample is not administering more surveys, but rather but rather increasing the number of selected patients who complete a survey, which yields a higher response rate. Using PRC’s industry best telephone methodology increases the percentage of responding patients, providing valuable feedback from a higher proportion of patients and generating a more representative sample of the total hospital patient population.
In any survey, you’re most likely to receive feedback from those who were either extremely happy or unhappy with their experience, as they’re most motivated to share their opinion. You’ve probably noticed that most online reviews of local restaurants or attractions are either one- or five-star ratings; most people aren’t inclined to make the effort to share what they consider to be an average experience, even if that “average experience” was positive.
The same is true in healthcare. Over 95% of healthcare consumers evaluate their experience positively, but only when asked directly—no one hears from 95% of their patients! However, by providing the best possible survey experience, responses from a larger percentage of patients ensure that you receive feedback from both the patients you would expect (those on the extremes) as well as the patients who aren’t inclined to respond (those with a positive, but average, experience). Therefore, the broader range of patient responses allows for a more balanced representation of the true patient population at your facility.
Working with you and your team to increase your response rate is incredibly important to everyone at PRC; we understand how a low response rate can negatively impact your hospital’s HCAHPS scores. With a low response rate and an inaccurate patient sample, it can be difficult to rely on patient experience scores to demonstrate the effectiveness of your improvements. In these cases, the hospital may not have a high enough response rate to accurately showcase efforts made by the hospital staff, as seen in the fictional example at the beginning of this post.
In addition to misrepresenting publicly-reported patient care, these lower scores also negatively impact hospital staff, diminishing morale as everyone questions their initiative efforts. Troubled staff wonder, “Why did we waste so much time fixing things if we have nothing to show for it?” It’s undeniably frustrating to see effort without payoff, but in reality, these low HCAHPS scores are not truly an indication of their efforts and instead reflect an unrepresentative sample. Rather than redoubling hospital improvement initiatives—a costly measure in most cases—it is critical to instead recognize when the challenge actually rests in obtaining a representative sample to accurately evaluate patient experience.
PRC offers the expertise and resources to increase your response rate and capture the truly excellent patient experience your hospital offers. For more detailed information about how response rates impact HCAHPS scores, download PRC’s white paper, “Impact of Response Rate on HCAHPS Dimension Scores” or contact a team member today!