New tool for early identification of COVID-19 surges

OAKLAND, Calif. — Health systems could use data from routine clinical care to identify the onset of upcoming COVID-19 surges as many as 6 weeks before they occur, Kaiser Permanente researchers found in a study published July 26 in the journal BMJ Open.

Even though the impact on hospitals of COVID-19 cases has subsided in Northern California with successful vaccination campaigns, there are concerns that the virus and hospitalizations may surge again locally as they are elsewhere in the world, said lead author Vincent Liu, MD, MS, a research scientist with the Kaiser Permanente Division of Research.

“Over the course of 2020, COVID-19 surprised us at nearly every turn, making longer-term predictions of its impact on our patients, health system, and communities extremely challenging,” said Dr. Liu, who is also a practicing intensivist with The Permanente Medical Group and regional director of hospital advanced analytics for Kaiser Permanente in Northern California.

“At the same time, shorter-term predictions — looking only 1 to 3 weeks out — left little time to respond adequately,” he said. “By examining hundreds of millions of data points across our entire health system, we have now identified key leading indicators that forecast impending COVID-19 surges as much as 6 weeks before they hit our medical centers.”

The team tested dozens of different data elements from the electronic health record system and data used by Kaiser Permanente in Northern California to find any elements that might be indicative of an impending surge in COVID-19 admissions. To determine what data points might be leading indicators, they looked at seasonal surges of influenza as far back as 2015.

Ultimately, the investigators chose 10 indicators and determined how many weeks of lead time they could provide health system and community officials. While individual indicators could forecast an upcoming surge in the next 1 to 3 weeks, the combined CHOTS — the “COVID-19 HotSpotting Score” — significantly increased the lead time to as far as 6 weeks prior to a surge.

CHOTS includes 4 major variables — cough and cold calls, relevant subject headers from patient emails, positive COVID-19 test rates, and current COVID-19 hospital census — along with 6 minor variables, including COVID-19-related calls to the call center, both routine and urgent clinic visits for respiratory infections, clinic and urgent clinic COVID-19 visits, and respiratory virus tests ordered.

Public health officials and individual health systems could use the information to boost local testing, expand contact tracing, adjust policy measures, and prepare expanded hospital space and supplies for anticipated patients. The authors found early indicators of waning COVID-19 activity were also valuable because they could help identify when relief from the surge of COVID-19 was on the horizon.

“The development of CHOTS is an inspiring example of how Kaiser Permanente’s integrated care delivery system and robust research capabilities can come together to provide the greater community with vital public health tools,” said co-author Stephen Parodi, MD, national infectious disease leader for Kaiser Permanente. “What is even more impressive is that this work was carried out while our medical centers were caring for large numbers of COVID-19 patients. Our researchers worked alongside operational leaders to better prepare our management of future surges.”

Developers chose simplicity

Because of the urgency to develop an early warning system for the COVID-19 hospital surge in May 2020, the authors chose to keep CHOTS relatively simple. They wanted to ensure the tool was available in real time by June 2020 and could be duplicated by other health care organizations. The authors are making the CHOTS methodology freely available for others to use.

“We use machine learning and artificial intelligence every day in our research group to develop predictive models to improve patient care. We applied these tools when we were developing CHOTS but didn’t find that they improved the tool’s value,” said co-author Patricia Kipnis, PhD, principal statistician at the Division of Research. “Our research group focuses on pairing the right algorithm with the right use case and, in this case, a simpler tool showed excellent performance and could be readily implemented and shared.”

CHOTS went live in June 2020 and the team evaluated it against actual COVID-19 hospital activity through December 31, 2020, throughout Northern California, and for each of 20 Kaiser Permanente hospitals in the region. The correlation of the regional CHOTS with hospital census was very strong, peaking with a 28- to 35-day lead time, but with continued correlation when tested out to 6 weeks. Because the impact of COVID-19 varied tremendously within individual medical centers over time, the team also evaluated CHOTS at each hospital and found strong, leading correlation at most sites.

To understand whether the CHOTS was better than individual leading indicators, the investigators compared the performance of each indicator with COVID-19 census. Current hospital census and new positive tests were the strongest indicators of census in the following 1 to 2 weeks, while call center and email volumes were strong indicators of census in the following 2 to 3 weeks. However, the signal from the individual indicators diminished over time and the CHOTS proved to be the most indicative of future activity with the most lead time.

The authors noted a limitation to CHOTS: It relies on comprehensive data from a robust electronic health record system, which may not be available to health systems in some locations and those without regional call centers. A simplified tool, without these indicators, also showed strong, but modestly lower, correlation.

The study was funded by The Permanente Medical Group, Kaiser Foundation Hospitals, and the National Institute of General Medical Sciences.

Co-authors also included Gabriel Escobar, MD, Laura Myers, MD, Lawrence Gerstley, PhD, and Khanh K. Thai, MS, of the Division of Research; and Yi-Fen “Irene” Chen, MD, Nancy Goler, MD, and Jessica Galin, MPH, of The Permanente Medical Group.

About Kaiser Permanente

Kaiser Permanente is committed to helping shape the future of health care. We are recognized as one of America’s leading health care providers and not-for-profit health plans. Founded in 1945, Kaiser Permanente has a mission to provide high-quality, affordable health care services and to improve the health of our members and the communities we serve. We currently serve approximately 12.5 million members in 8 states and the District of Columbia. Care for members and patients is focused on their total health and guided by their personal Permanente Medical Group physicians, specialists, and team of caregivers. Our expert and caring medical teams are empowered and supported by industry-leading technology advances and tools for health promotion, disease prevention, state-of-the-art care delivery, and world-class chronic disease management. Kaiser Permanente is dedicated to care innovations, clinical research, health education, and the support of community health.