The challenges with massive amounts of data in healthcare
As healthcare providers enter more patient data in electronic health records (EHRs), much of the data lies unused or contributes to inefficient processes. Healthcare analytics tools play a crucial role in utilizing these data, helping health systems improve patient outcomes, make data-driven decisions, and gain an understanding of the overall health of a population.
However, these health systems face numerous challenges — here we explore the hurdles with putting massive data assets to work.
Surveys found implementation of EHRs can contribute to burnout:
Practices that implemented an EHR saw an increase in stress levels, followed by a decrease that did not return to the baseline 3
Fully mature EHR systems, combined with shorter visits, are associated with stress, burnout, and provider intent to leave the practice 3
While the data is collected, the sheer size of it all is overwhelming...
The average hospital produces roughly 137 terabytes every day, or 50 petabytes every year 4
Structured data makes up 20% of all healthcare data:
Structured data includes first and last name, date of birth, home address, and gender and are relatively straightforward to manage 4
Unstructured data makes up 80% of all healthcare data:
Unstructured data includes clinical notes, medical images, chest x-rays, 3d mammograms, and digital pathology files and data and much harder to codify 4
Analysis can’t be run on irregular data structures:
Organizations work to convert their unstructured data to codified structured formats to enable easier storage, analysis and decision support
Though the data is captured by providers, it might not be correct...
1-in-5 patients found errors in their records:
40% were considered serious mistakes that included the wrong diagnoses, medical history, medications, physical examination, test results, notes on the wrong patient, and sidedness 5
Even then, that data may not be compatible with other hospital systems, or even internal systems...
62% of hospitals in the US are functioning in all four domains of interoperability and can easily share and receive patient data without issues 6
In other words: 1-in-3 patients run the risk their medical records aren’t easily transferred to another provider
Inconsistent data standards can produce errors through issues like:
Date formats: Do the dates 1987-01-12 and 12-01-1987 match?
Terminology variations: Are “hypertension” and “high blood pressure” recognized as the same condition?
Abbreviations and acronyms: Does "ACS" mean “Acute Coronary Syndrome” or “Acute Chest Syndrome”?
Units of measure: Was weight captured in LB or KG? Was height captured in CM, IN, or FT IN?
Data field definitions: Does “date of admission” mean when the patient was admitted to the hospital or registered in the system?
Code sets and classification systems: Is this data using ICD-9, ICD-10, ICD-10-CM, or ICD-10-PCS?
Data standards issues are solved by using a standardized framework, like FHIR, or creating a data map so the databases can interchange:
Creating data mappings between medical databases varies from 100s to 1,000s of standard and customized fields — the actual number varies entirely on the two systems’ configurations
All this is happening while there’s an IT talent shortfall...
64% of IT executives see the talent shortage as the most significant barrier to emerging technologies 7 as well as being the leading factor inhibiting adoption in key tech areas
And the healthcare industry faces never-ending challenges...
34% of hospitals reported being the victim of a ransomware attack aimed at locking them out of their EHR 8
ICD-10-CM (one of the most common coding systems used) had 1,826 diagnostic codes added, revised, or deleted in 2023 alone with more updates released every year 9
Risk Adjustment Data Validation (RADV) is enforcing new standards for documentation — reporting is stricter than ever before 10
Addressing these challenges is crucial to fully harness the power of healthcare analytics to improve patient care, operational efficiency, and informed decision-making.
See how top healthcare organizations are overcoming these hurdles to optimize data collection, analysis, and utilization for the benefit of patients and providers.