Feature Story

Unveiling the patterns in TDM data ‘dots’

March 2000
Mark Uehling

You can think of a lab report as a lab report. Or you can think of it as a nightmarishly jumbled, neo-Cubist connect-the-dots diagram. Suppose the patient’s test results are right there on the page, in neat rows and columns. Is the change in test A over the first three columns clinically significant? Is the value of test B in the second column more or less important than usual when considered in combination with the value of test C on the second page of the report?

Connecting the dots of these diagrams to reveal clinically important patterns is like trying to duplicate a Picasso in your head. Some of the patient’s dots may not connect to anything on the page. Some dots may obscure or distract from others. Important dots may be missing, located elsewhere, or drawn in a different colored ink. Clinicians faced with the task of connecting the dots may retreat to a superficial comparison of the most recent results with reference ranges, losing the deeper view.

Now James H. Harrison Jr., MD, PhD, has devised a way to make the dots connect beautifully. Dr. Harrison, associate professor and associate director of pathology informatics, University of Pittsburgh Medical Center, has spent a decade tinkering on computers, old and new, and devised software to translate time-sensitive laboratory data into graphs that are easy on the eyes.

In more technical terms, Dr. Harrison has written a program that ingests and analyzes therapeutic drug-monitoring results from hundreds of patients. Comparing the actual data with patterns specified in the program, his software can alert the user when several days’ worth of laboratory data from a particular patient or a large population of patients conform to—or diverge from—a stipulated pattern of clinical interest.

You want to know if three of four sequential trough values for gentamicin were elevated? You can’t remember if there have been more than two lithium values in the last three days? Or you recall too many patients showing variable peaks for vancomycin? Dr. Harrison’s LabScanner program will show you the data.

As someone with a longstanding interest in therapeutic drug monitoring and a doctorate in pharmacology, Dr. Harrison has a measured opinion of his medical colleagues’ abilities to watch key values over any interval longer than 24 hours. He understands that busy clinicians and laboratory professionals cannot be expected to sit down with pencils and graph paper and spend the morning drawing a few hundred charts. LabScanner is a tool to identify quickly data patterns that may warrant attention.

But it may be helpful to clarify what LabScanner does not do. Dr. Harrison does not expect his software to issue critical alerts or perform delta-checking for any value that demands immediate action by a physician. He knows existing laboratory information system products already provide such warnings. Rather, LabScanner is intended for clinical followup and teaching purposes, even to support consultation, and to allow pathologists to add value to the information that leaves their laboratories. Nothing controversial there.

Dr. Harrison does see problems with the traditional ways in which laboratory data are presented to physicians: The data are generally arranged in a table or as a list of numbers, sometimes for a single day, sometimes for a few days. Either way, he says, those columns of numbers make it hard to discern macroscopic trends.

"We don’t really do a very good job of providing physicians with data that express temporal changes well," Dr. Harrison says. "The best you can do is columns of numbers progressing with dates across the top in a spreadsheet-like format. That kind of reporting has been standard for years. And then the way it was done on paper was translated into the typical computer systems. Some computer systems are now providing a graphical display as an alternative. But no one has time to use individual graphical displays to search for important patterns."

Dr. Harrison has data to support his view. Many years ago, primarily as an intellectual exercise, he asked his institution’s computer staff for a month’s worth of cases involving phenytoin. He knew monitoring phenytoin was something that tested even experienced clinicians. The computer staff gave him 90 cases, all exported from the LIS into a generic text file, and he went to work with pencil and paper.

"I plotted them all and looked at the profiles to see the sorts of things that were really happening. Some of them were pretty interesting," Dr. Harrison says. The charts of 20 patients, he felt, deserved a closer inspection. And the values for two of those proved eye-opening. In both cases, for a week or so, phenytoin levels were in the target range, leading to discharge. In the charts of both patients, physicians dutifully noted phenytoin levels were in the range that would permit the patient to be discharged.

But, as Dr. Harrison’s exercise revealed, there were clear upward trends in both patients that suggested neither may have been ready to leave the hospital. (One of the patients later returned to the hospital and was readmitted and detoxified.)

"The patients were not stable. The physicians had done a data reduction in their heads each time they had looked at the results," Dr. Harrison recalls. "It’s unrealistic to think that physicians would remember many individual numbers over several days. They would classify them as ‘in’ or ‘out’ and remember that. They were losing data, and they had not perceived that it was important data. They figured that if it stayed in the target range for a week, it was going to be okay. That was not the case."

To say the least, the experience suggested the existing manner in which laboratory results were presented and analyzed could be improved. "That’s a situation where just because the pattern wasn’t perceived, the patient was discharged with a subsequent fairly significant expense involved in having missed the data pattern," Dr. Harrison adds.

He began to ruminate on a tool to help pathologists present data more effectively. Dr. Harrison continues: "I really wasn’t wanting physicians to graph every result on every patient. They just don’t have time to do that." What he had in mind emerged from his own training and a sense of what other doctors might find useful. Dr. Harrison asked himself this question: "Would it be possible to create a system that would examine these profiles and select the ones that would be important to review?" The answer turned out to be yes.

Fortunately for pathology informatics, Dr. Harrison was willing to teach himself a bit of programming to accomplish the task. He hoped to help pathologists recognize problem patterns in therapeutic drug monitoring and communicate them to clinicians.

His first task was to prove that a computer could identify patterns of clinical import. He published such a paper in the American Journal of Clinical Pathology in 1995. But the tedious work of polishing a software program that any laboratory professional or clinician could use took a few years. His program now functions more or less identically on Windows and Macintosh machines and is surprisingly sophisticated for a homegrown endeavor.

Perhaps most importantly, LabScanner creates small graphs. These include linear regressions and half-life calculations. For every drug, on every graph, the reference range is easily seen in a gray horizontal band. Users of the program can select just a few laboratory results and graph only those. Need to print them? LabScanner remembers the selected set and prints just that. Need to focus only on results from patients younger than 18 months? No problem. "The first idea is to just identify these patterns and display them," Dr. Harrison says. "You don’t have to sit down with a computer to figure out what it is. You can seat-of-the-pants look at it, figure out what might be going on." The computer connects the dots and lets physicians interpret the line according to their abilities.

Since last summer, LabScanner has been refined and is now available free through Dr. Harrison’s University of Pittsburgh Internet site, thelab.upmc.edu/jharrison/labscanner.

Give-away software, known as freeware, generally is respected in the computer industry. Software reviewers typically give higher praise to freeware and open-source software than to commercial products. (The most notorious case is Linux, an operating system popular among those displeased with Windows.) Dr. Harrison’s aim, naturally, is not just computational: He also wants physicians to really see laboratory data over longer intervals.

To accomplish that, Dr. Harrison acknowledges, he needs the help of colleagues. In the computer realm, such allies are called "beta testers," intrepid souls who download software that is in development via the Internet, test it, and suggest improvements. Such beta testers could be crucial to enhancing LabScanner.

This is not to say using developmental software is risk-free. Bugs may exist in LabScanner, Dr. Harrison warns. His program includes advisories and instructions, and he suggests it be operated on noncritical computers. "This is developmental software. It’s not commercial level—yet—which may, correctly, scare off some people. But it should run as advertised for most people who use it."

Dr. Harrison does not believe pathologists and laboratory directors should approach LabScanner with dread, just caution. "They should treat this like a new lab test, in that they should do confirmation testing against what amounts to control data—data that they’ve put together and know what’s in it, so that they can make sure it’s behaving correctly. If it doesn’t behave correctly, I want to know about it."

What’s in it for the beta testers? They can shape the program with their suggestions. Says Dr. Harrison: "The hope is that I can find some pathologists and laboratory directors who are not only interested in using this and using its current capabilities, but in communicating with me about additional features they’d like, and helping design those."

In fact, Dr. Harrison’s program can already be customized fairly easily. "Anyone can set up data-scanning rules right away," he explains. "It comes with preset patterns that I’ve found useful, but people can add rules to it or subtract those of mine, or edit mine. They have complete freedom. LabScanner can find levels outside a defined range, tell you about increasing or decreasing values, monitor testing frequency, and find variability between values. Those are the four classes of rules that people can play with. Using those four classes, you can detect a lot."

Take digoxin as an example. The rule that comes with the program for digoxin has 10 subcomponents that were carefully considered by Dr. Harrison. For a reference range of 0.9 to 2.2 ng/mL, the program will flag a specimen if three of four sequential values for digoxin are less than 0.9. Likewise, it will issue an alert if three of four values show an increase of greater than 0.1 ng/mL per day and the last value is greater than 2 ng/mL. If the data about digoxin are unexpected in eight other respects, alerts will be issued. Best of all, anyone who has different thresholds of concern about the drug can easily tinker with LabScanner and adjust the settings.

In a larger sense, the power of LabScanner is not merely that the computer mechanically checks whether the last value for, say, theophylline violates a rule or not. It is that the computer can check all of the laboratory results for a thousand patients receiving 50 medications and note all of the instances in which the rules have been violated.

The technical term for casting such a broad net is "data mining." But the starting point is radically simple. All LabScanner needs as its raw material is the generic text output from a conventional LIS. "The data mining is simple in that all you’re really doing is scanning through a large text file," Dr. Harrison says. "The goal is the same: to select out patterns of data that match certain conditions."

The data mining has been especially useful as an educational aid, helping Dr. Harrison find good cases for teaching clinical pharmacology. His medical students are given a puzzling LabScanner graph and asked to investigate the patient to explain the clinical or physiological circumstances behind the graph. To Dr. Harrison’s delight, attending physicians responsible for such patients have asked for a copy of the medical students’ LabScanner printouts to add to patient charts, confirming the value of this data presentation.

What’s more, in the course of such teaching exercises, Dr. Harrison and his colleagues have used LabScanner to identify more systemic problems throughout the hospital.

Helping colleagues monitor digoxin is a perennial issue. But during his time as a young professor at Tulane University, for example, the program also shed light on the dosing of aminogly-coside antibiotics. "There were patients who had a trough drawn, and then a peak drawn, and the peak was lower than the trough," he explains. "What was going on? This wasn’t a case of unusual pharm-acokinetics needing dosage individualization. In the graphical display, this really looked like a problem with medical processes."

By that, Dr. Harrison means all of the vagaries introduced by the interaction of nurses, pharmacists, and house staff. LabScanner can identify patterns of laboratory values that are awry, and a pathologist can then investigate these as a part of laboratory quality assurance.

With the aminoglycosides, Dr. Harrison says, "When patterns like this occurred, they were viewed by the physician and nursing staff as laboratory error. Any time they got something that was bizarre, they would just go, ’Oh, it was a lab error,’ and would not follow up. When we showed this stuff to the pharmacy, they were extremely interested."

After investigation, the pharmacy changed its procedures to prevent IV bags from being sent to patients with IV solution but no drug. "That’s an example of one of the ways this thing can be used for quality assurance," Dr. Harrison says. "You can detect these patterns—problems in drug treatment and drug sampling-and optimize those processes. Pathologists can be important in recognizing the significance of clinical data patterns and figuring out who in a health care enterprise is best able to follow them up so that patients benefit."

Thus LabScanner is a sort of crescent wrench that can, in theory, be used to tighten any bolt in the hospital, helping pathologists to alert the administration, laboratory, pharmacy, or other physicians to trends in drug levels or other laboratory tests they would otherwise miss in the hurly-burly of a typical day. Explains Dr. Harrison: "This is to detect more subtle patterns that typically develop over a period of days. When a physician looks across multiple columns of numbers, the patterns may not be that evident—or to see them, it would take so long that it would really be a drag on their workflow."

That workflow, it would appear, is not yet perfect. Dr. Harrison has fed several months of laboratory data into LabScanner using records from 1,944 patients in New Orleans (at Tulane) and 2,560 patients in Boston (at Brigham & Women’s). Dr. Harrison’s analysis was retrospective, but it suggested that substantial insights could be gained whenever LabScanner was installed. LabScanner evaluated 6,410 laboratory specimens at Tulane and 10,923 at Brigham & Women’s, with 27 percent and 52 percent of the samples getting flagged.

At Tulane, LabScanner reported that 3.1 patients per day had values the program did not expect; at Brigham & Women’s, the number was 8.4 patients per day. This represented about one new patient per day per 100 hospital beds. The gravity of these flagged specimens has not yet been established. It’s hard to say whether LabScanner’s flags are identifying minor issues or major ones. But, at the least, it does appear Dr. Harrison has statistical support for his position that eye-numbing tables of laboratory data do not always tell clinicians what they need to know, that there may be significant trends in the data of which clinicians on the floor are unaware.

At the same time, Dr. Harrison points out, Cerner and Sunquest, as well as other vendors, are not oblivious to such issues. Their newest systems—some of which are being installed at the University of Pittsburgh—offer rule-based searching tools similar to those in LabScanner. "The interesting things about these is that they are really not targeted to find these temporal patterns, which may be subtle. They’re more targeted to, if values from several different tests are within defined ranges, then issue an ’alert.’ They’re looking at a snapshot of a patient at one point in time."

That’s too bad. Dr. Harrison would prefer the features of his system be built into a hospital’s bedrock informational infrastructure from the start. The main impediment to that sort of shift, he notes, is attitude: helping colleagues become more aware of the importance of longer time intervals.

"The roadblock for stuff like this being in the big lab systems really isn’t being able to write the code. They could write code like this. The issue is demonstrating it’s useful in medical care," he explains. "My goal is to lower the barrier of entry enough so that people will try it and so that I and others can write papers about it and maybe convince some of the vendors to include technology like this in their lab systems."

One day, the uses for such tools might be esoteric and ordinary. On the first score, LabScanner could be systematically used to perform quality assurance and outcomes analysis, even confirming that procedural changes are successful. Beyond therapeutic drug monitoring, Dr. Harrison makes a convincing case that his program could be used to evaluate platelet counts, length of patient stays, even renal and liver function, as well as play a role in tumor marker analysis.

In a more down-to-earth way, Dr. Harrison is also hoping to help his colleagues ask more of their existing laboratory reports. "The way physicians are presented with data makes it difficult to see the patterns," says Dr. Harrison. "They see the numbers. But seeing the patterns is hard. We’d like to make it easy for them to see the numbers if they need the numbers but still easy for them to see the patterns if the patterns are important to show."

Dr. Harrison is quick to clarify that the computer is not in any sense thinking. "It’s not actually taking the place of the physician and doing an analysis," he says. "Systems like this don’t critically analyze the data, and, therefore, they don’t take the place of a clinician or a pathologist. Instead, they augment the physician by directing his or her attention to important features of the data so that medical decisions can be made more quickly and accurately."

Mark Uehling is a freelance writer in Chicago.

Dr. Harrison (jhrsn@pop.pitt.edu) is distributing LabScanner free at thelab.upmc.edu/jharrison/labscanner. He will present a seminar on data mining April 10 at the ASCP/CAP Spring Meeting in Boston and in October at the ASCP/CAP Fall Meeting and Exhibits in San Diego.