When you’re dealing with data, it’s not uncommon to reach an investigative dead end. You’re trying to get to the root of an issue or discover a new course of action, then — like a pair of brakes screeching — you can go no further. You’ve hit a major data blind spot.
In healthcare data analytics as in other industries, knowing the limitations of your data is the ticket to avoiding this scenario. It’ll also help refine the types of questions you ask and leads you pursue. No more wasted time, no more red herrings — just a seamless relationship between the data you have, the data you want, and the discoveries yet to be made.
In data, it’s often easy to see the perimeter of what you know — you have 150,000 patients’ EHRs, for example, plus claims information and lab results from within the system. Then, you can sometimes identify what you lack — SDoH information, let’s say, the social factors that are impacting patients’ health outside the hospital’s walls.
The danger, then, is the data that would impact and inform your research that you need, but aren’t aware that you need (“unknown unknowns”). This is a data blind spot.
SDoH could be an example here, too — if an analyst is trying to see why a certain population regularly skips routine care, they’d need to think ahead about things like transportation access and job security. Imagine researching this question without SDoH indicators or census track data — someone might erroneously conclude that people in a certain neighborhood are just irresponsible, or that a group is ignoring emails intentionally, when they don’t have access to regular internet or a smart phone.
Implicit bias (like the ones mentioned above) and confirmation bias are two of the worst consequences of data blind spots.
Confirmation bias refers to the human impulse to support one’s own beliefs and preconceived notions. Implicit bias is when stereotypes about race, gender, or socioeconomic class seep into otherwise data-backed arguments, making the results unreliable.
“Everyone uses the data points that back up their own arguments,” says Jake Hochberg, Arcadia’s Executive Director of Customer Insights. “So, to an extent, sometimes people want to tell a story. And sometimes people use data to reinforce the story.” Knowing that urge is one way to actively resist it.
All data has some limitations. Following the steps outlined here will help you avoid these negative outcomes, engaging in thoughtful research and analytics so that the results are trustworthy, effective, and ethical.
According to Hochberg, your starting point should be a clear question. Are you trying to see whether diabetic patients in one county have higher rates of ketoacidosis? Are you trying to see if monthly blood tests reduce complications from a certain sensitive medication? The more pointed, the better, and the more you can ensure you’re interpreting the data correctly.
Much like a Magic 8 Ball, the answers could run the gamut, from “who knows, there’s not enough data” to “inconclusive, try again in a few months” to “here’s a distinct, statistically significant correlation.”
From these highly specific queries, sometimes you’ll get lucky enough to chance on other discoveries.
“Typically, there’s a specific business use case we’re trying to solve for, then we will analyze the data in that in that vein,” Hochberg says. “Sometimes we’ll find something tangential and then decide to go down a different path, because we find the data point valuable and interesting. So sometimes one analysis actually spurs an entirely different analysis.”
In an effort to reduce ED admissions, an analyst might unearth an entirely different insight about care that happens at an annual physical. Either way, the system and patients it serves benefit from this data deep-dive.
While analysts are interpreting a healthcare system’s data, administrators and C-suite executives are setting business objectives. In between these roles, a lot can get lost in translation. According to Hochberg, it’s essential that both parties get clearly aligned on the business objective and use case.
“The best way to be successful is to ensure that there’s good partnership between the analysts who are doing the analysis and the requesters of the analysis,” Hochberg says. “There’s a whole layer of nuance that gets lost in the middle.”
Here’s an instance where it may be beneficial to meet face to face and speak plainly about the work you hope to do, whether it’s lowering one group’s A1Cs or reducing complications after a procedure. Trust and collaboration between these two groups can inspire better questions and better interpretation of complicated info.
“I think the most important thing is that feedback loop for a business operator who’s asking the question — don’t ask for specific data points, but instead, make sure you’re clear on what it is that you’re trying to do in the real world,” he says. “That way, you’re giving the analyst a chance to think about what the data actually has, and feed back to the requestor the best way to use the data provided.”
If the hope is to uncover those “unknown unknowns,” to reach the limits of a data pool and say, “Here’s what in here, here’s what’s not,” it’s imperative that data analysts regularly audit and inventory the data they have.
“Maintaining a knowledge and governance of the data sources you have is important,” says Hochberg. With so much information flowing into a given healthcare network every day, week, month, and year, that’s a formidable challenge, but it’s crucial. The map of data points is constantly expanding, and data analysts and healthcare leadership can work together like cartographers, broadening their knowledge.
“Everybody’s data landscape is different,” Hochberg says. “And everybody who wants to be using data powerfully within their own organization needs to understand what they have, and what each data source and type’s limitations are.”
Arcadia’s data analytics platform stacks with existing tech infrastructure, so in-house data analysts can gain even more perspective on the past, present, and future of their organization’s trajectory. Together, we’ll team up to help you identify and eradicate data blind spots, a clear view for accelerating progress.