A data-driven acute inflammation therapy
© Radosavljevic et al; licensee BioMed Central Ltd. 2013
Published: 11 November 2013
Acute inflammation is a severe medical condition defined as an inflammatory response of the body to an infection. Its rapid progression requires quick and accurate decisions from clinicians. Inadequate and delayed decisions makes acute inflammation the 10th leading cause of death overall in United States with the estimated cost of treatment about $17 billion annually. However, despite the need, there are limited number of methods that could assist clinicians to determine optimal therapies for acute inflammation. We developed a data-driven method for suggesting optimal therapy by using machine learning model that is learned on historical patients' behaviors. To reduce both the risk of failure and the expense for clinical trials, our method is evaluated on a virtual patients generated by a mathematical model that emulates inflammatory response. In conducted experiments, acute inflammation was handled with two complimentary pro- and anti-inflammatory medications which adequate timing and doses are crucial for the successful outcome. Our experiments show that the dosage regimen assigned with our data-driven method significantly improves the percentage of healthy patients when compared to results by other methods used in clinical practice and found in literature. Our method saved 88% of patients that would otherwise die within a week, while the best method found in literature saved only 73% of patients. At the same time, our method used lower doses of medications than alternatives. In addition, our method achieved better results than alternatives when only incomplete or noisy measurements were available over time as well as it was less affected by therapy delay. The presented results provide strong evidence that models from the artificial intelligence community have a potential for development of personalized treatment strategies for acute inflammation.
Acute inflammation is a progressively severe medical condition defined as an inflammatory response of the body to a trauma, a surgery, a burn, or an infection. Fast progression of acute inflammation requires quick and accurate decisions from clinicians. Inadequate and delayed decisions make acute inflammation the 10th leading cause of death overall in United States, with the estimated cost of treatment about $17 billion annually . However, despite the need, there are no computational methods that can help clinicians in planning optimal therapies for acute inflammation.
Model predictive control (MPC) is a method often used in clinical applications for planing of optimal treatment [2–5] mainly due to its inherent capability to handle clinically relevant constraints and to take into account multiple variables at the same time. A predictive model is the most important part of MPC, which is deployed to estimate patient's response to the therapy. These estimations together with a set of clinically relevant constraints are used in MPC to compute optimal therapies that will lead the patient to the healthy condition. The efficacy of MPC based treatment is completely dependent on model's capability to accurately estimate the patient's response to therapy.
To construct an accurate predictive model, practitioners often rely on domain-based assumptions about patients' behavior. Such a domain-driven model was previously used as a component in MPC to find optimal acute inflammation treatment . The authors showed that pro-inflammatory and anti-inflammatory medications within MPC setup decreased the mortality rate. However, therapy outcome was greatly dependent on the set of parameters used in the model.
To relax the dependency on parameter settings, a data-driven predictive model that learns patients' responses from observed data and doesn't rely on any domain-based hypothesis can be used instead. MPC with data-driven predictive models was successfully applied in medical applications, including optimal glucose control , the control of anticancer medications dosage , the control of ventilation , and finding an adequate dosage of anesthesia . The main objective of this study is to show that MPC with a data-driven predictive model can improve the overall quality of acute inflammation treatments.
Our preliminary results  indicated that a data-driven MPC method significantly improved the number of healthy outcomes when compared to acute inflammation therapy strategies from literature and clinical practice. This paper reports a more detailed study by evaluating the data-driven MPC and alternative approaches over different clinically relevant setups as follows. All experiments are done on virtual patients generated by a mathematical model that emulates inflammatory response, which is a standard approach in pharmacological research. We show that in achieving results presented in , our method used much lower doses of medications than other methods. Furthermore, our method provides good accuracy even in the presence of incomplete measurements, as well as additive Gaussian noise. It is also less affected by possible therapy delay than the alternatives. We also identified characteristics of patients who may benefit the most from application of our method. Consequently, our data-driven tool for the optimal treatment of acute inflammation along with domain-based knowledge may provide a platform for the development of personalized treatment strategies that will increase survival rates.
The rest of the paper is organized as follows. In the next section we introduce a virtual patient model that emulates therapy response and we define clinically relevant therapy constraints. Then we propose a data-driven predictive model that determines optimal therapy, which is followed by evaluation of the model. In the final section we give a conclusion.
Mathematical model for virtual patient and treatment constraints
Virtual patients are carefully developed mathematical models made to mimic the human body behavior when exposed to circumstances that are in the interest of a study. They allow biomedical researchers to perform various types of experiments on the same patient as well as to compare the outcomes. Thus, the main purpose of having virtual patients is to reduce expense for clinical trials and to lower chance of failures. A mathematical model in the form of the system of ordinary differential equations (ODE), recently proposed in , simulates inflammatory response to an infection, including medication effect on inflammatory response. It models the dynamics of concentration of:
bacterial pathogen (P),
early pro-inflammatory mediators (N),
tissue damage markers (D),
anti-inflammatory mediators (CA),
Mathematical model in this form is capable of modeling the complex cascade of inflammation initiated by pathogen (P). An increase of pathogen level P leads to the series of positive and negative feedback reactions that are all successfully modeled by ODE. In particular, an increase of P causes the development of a pro-inflammatory response (the increase of N) and the development of tissue damage (the increase of D). Equation (1) simulates a positive effect of inflammation, where an increase of N reduces level of pathogen P. However, (3) simulates a negative effect of inflammation, where an increase of N further damages tissue causing rapid increase of D. An increase of D activates a negative feedback in (4), or anti-inflammatory response (CA), which lowers the level of N and prevents tissue damage (decrease of D) . The strength of positive and negative feedbacks depends on the parameter values in ODE. By varying parameter values we can simulate variability among patients.
healthy (P <1, N <0.05, D <1),
aseptic death (P <1, N >0.05, D >1),
septic death (P >1, N >0.05, D >1).
Evolution of the patient to the final state can be modulated by carefully constrained pro-inflammatory (PIDOSE) and anti-inflammatory (AIDOSE) medications. Therefore, model predictive control is applicable to inflammation therapy only if constraints on medication doses obey clinical rules. Oppositely, for example, a large amount of medication given at once can cause the death of a patient due to overdose. Also, the prescription of a high level of anti-inflammatory doses with long duration may predispose the patient to other infections, which might cause death. We follow well-defined medication constraints for inflammation treatment stated in :
, where is the difference between N max (N max = 0.5) and the current level of N = N k ;
, where is the difference between a maximum allowable level of CA (initialized to CA max = 0.6264) and the current level of CA = CA k ;
saturation of anti-inflammatory mediator: the saturation of CA for long durations is avoided in clinical practice because of other infections that can occur and endanger organ recovery. Thus, if the level of CA has been elevated for 48 hours, then CA max was halved.
Model predictive control (MPC)
prediction horizon (p) is the time window that governs how many future outputs of the predictive model will be used in the objective function;
predicted patient states are estimations of output variables obtained from the predictive model;
control horizon (c, c ≤ p) is the time window that governs how many future controls (medication dosage) will be determined by the optimization algorithm;
future medication doses are doses that are obtained by the optimization algorithm assuming constant doses from k + c to k + p − 1 set to the values of.
where w D , w P , and w c are weighting constants. Then, the first doses and are applied to a virtual patient model to obtain new patient state at k + 1. At the next time point k + 1, the optimization procedure is repeated and a new sequence of doses is obtained.
Finding successful treatment by applying two medications is a challenging problem. Applying a large amount of AIDOSE would increase the level of CA, which would reduce the level of D. On the other hand, this might cause extreme growth of pathogen P. Also, applying a large amount of PIDOSE would increase the level of N, which would reduce the level of P. However, this can trigger pro-inflammatory response to focus on eliminating P, after which the level of tissue damage D could not be controlled anymore. Thus, both medication doses and their timing are critical for successful treatment.
Accuracy of the predictive model over the prediction horizon directly affects the quality of model predictive control. Design of our data-driven predictive model involves two steps: 1) specification of model structure 2) learning model parameters from training data. These steps will be further explained in following paragraphs. For the sake of clarity we named our predictive model as DD-MPC.
Structure of DD-MPC predictive model
where j = 1,..., p; F P is a function with unknown parameters β P ; n py and n pu are time lags. Other sub-models responsible for prediction of N, D, and CA have similar functional forms, but with different parameters β N , β D , and β CA .
The next step is to define input-output functional relations F P , F N , F D , or F CA in terms of model parameters. Data driven non-linear predictive models such as Gaussian processes , neural networks , particle filters  have been successfully developed for industrial applications, although designing non-linear models in control theory is difficult task . Also, non-linear models require much more data to be trained then linear models. Moreover, in the case of multivariate process, the difference in necessary training data is even larger. Due to high cost and limited availability of clinical data we applied linear functional forms in terms of β P , β N , β D , β CA .
Learning of DD-MPC predictive model
To follow a real life scenario regarding the availability of representative training data, we assume that the data come from a small population of diverse patients. To generate training data we select N tr virtual patients represented by ODE and consider that hourly measurements are available for a whole week. During the data generation process, initial conditions and parameters for each patient are randomly chosen from valid ranges while dosages are determined as follows.
In spite of control theory where control signals are randomly generated, random generation would not be clinically relevant for medication doses and thus we designed the following approach. We use MPC for each patient in the training set, deploying its own ODE as a predictive model. In this setup, the predictive model provides perfect predictions for patient's future states because predictions and future observations are the same at every time point. Therefore, by applying MPC we obtain ideal medication dosage. Ideal dosage is neither realistic in clinical practice nor suitable for learning data-driven models. Therefore, we added random Gaussian noise to the ideal sequence of doses, obtaining dosage that is close-to-ideal. On the other hand, close-to-ideal doses applied every hour over an entire week would lead many patients to a non-healthy state. Instead, close-to-ideal dosage strategy is applied in the first 10 hours of therapy (the most critical period) and then therapy is continued with the ideal doses. This strategy is called close-to-ideal dosage strategy.
Parameters of DD-MPC
In order to evaluate DD-MPC we need to decide on time lags, prediction horizon, and control horizon. The model with time lags equal to 2 had significantly larger predictive accuracy than the models with time lags different from 2. Therefore we use the model with time lags set to 2 in all of our experiments. The duration of the prediction horizon had to be set to achieve an equilibrium between following requirements: (1) the duration should be long enough in order to allow the therapy to fully manifest, (2) the duration should be short enough since predictive models are usually less reliable as the duration increases. The best performance was achieved with the length of prediction horizon window set to 5. The longer duration of control horizon makes response to be faster, but the system then becomes susceptible to model uncertainties . Therefore we set the window length of control horizon to 2. Weights w P , w D , and w c in objective function were set to 3, 1, and 1 respectively in order to amortize the effect of unequal scalings.
Number of patients to train DD-MPC
Obtaining the data is expensive in clinical applications. Hence, it is important that we investigate the least number of patients who are needed for the training data set so that the model learned on such data achieves acceptable accuracy. We generated a population of 65 virtual patients with the close-to-ideal dosage strategy for each of them. Each patient was observed for 168 hours and data were recorded on hourly bases. Finally, there were 43 healthy, 14 aseptic, and 8 septic patients' outcomes. It is worth noting that during training of the proposed data-driven model we did not take into account any prior knowledge about the data generation process.
Average number of healthy, aseptic and septic patients after DD-MPC therapy on a set of 50 patients.
24.1 ± 9.7
12.7 ± 9.0
13.2 ± 7.4
26.6 ± 12.1
10.0 ± 5.8
13.4 ± 9.2
37.8 ± 11.1
7.4 ± 4.7
4.8 ± 8.1
41.7 ± 3.5
6.0 ± 2.0
2.3 ± 4.6
41.7 ± 3.2
5.8 ± 2.3
2.5 ± 5.0
44.1 ± 1.1
5.8 ± 0.7
0.3 ± 1.0
44.3 ± 1.2
5.3 ± 0.5
0.4 ± 1.2
43.6 ± 0.8
5.9 ± 0.9
0.5 ± 1.2
The proposed DD-MPC was evaluated on a population of 500 virtual patients recorded for 168 hours (1 week) by hourly measurements of P, N, D, and CA. In the case when treatment is not applied, the population has 85 septic, 117 aseptic, and 298 healthy virtual patients. In order to determine which patients require treatment, we follow the criteria from . According to , if N exceeds 0.05 at any time point, then the corresponding patient needs to receive treatment. We found 321 out of 500 patients who were supposed to be treated. These patients represent a population on which we compare DD-MPC and baseline methods.
We compare therapy outcomes of our DD-MPC to models used in :
no therapy applied model (Placebo);
the therapy currently used in the clinical practice: a constant anti-inflammatory therapy; we simulated this kind of therapy by applying AIDOSE = 0.005 hourly during the fist 72 hours from therapy onset (N >0.05), after 72 hours the therapy was terminated (Static);
therapy based on MPC that used the mathematical model which parameters were fixed to parameters from a single patient  (Mismatch).
Therapy outcome was classified based on patient state at the end of 168 hours of simulation. If the patient did not reach stable state after 168 hours, then the simulation was extended for an additional 300 hours with no therapy applied. We report percentage and number of septic, aseptic and healthy outcomes at the end. In addition, we distinguish different groups of patients based on the effect of therapy:
harmed - designates cases when the outcomes of therapy without using any medications (Placebo) were healthy, while outcomes after applying therapy were aseptic or septic (the lower percentage of harmed the better);
rescued - designates cases when outcomes after therapy were healthy, while outcomes without therapy were aseptic or septic (the higher percentage of rescued the better).
Fully observed data
Number and fraction of patients on fully observed data for: model with no therapy applied (Placebo), model with constant anti-inflammatory dose (Static), MPC with mathematical predictive model with set of parameters equal to parameters of a single patient (Mismatch) and MPC with data-driven predictive model learned on small data sample (DD-MP C) and MPC with data-driven predictive model learned on small data sample with 5% additive Gaussian noise in observations (DD-MPC+nois e).
Healthy (total 321)
Aseptic (total 321)
Septic (total 321)
Harmed (total 119)
Rescued (total 202)
To identify patients (represented by initial conditions and model parameters) who benefit the most from using DD-MPC, we compare the outcomes of the DD-MPC model to the outcomes of Mismatch. According to possible outcomes, we split patients into two groups: (1) 266 patients who were successfully treated by both models, and (2) 33 patients who were successfully treated by DD-MPC but not Mismatch. The group of 33 patients is characterized by high initial level of pathogen P0 (P0 = 0.78 ± 0.13). In order to detect the conditions for which DD-MPC outperforms Mismatch, we compare 33 patients to a subset of 165 patients from another group who had similar P0 (P0 = 0.79 ± 0.12). Average values of parameters k cn and k nd were significantly different between the group of 33 patients and the subset of 165 patients according to two tailed t-test with 95% confidence. We discovered that the Mismatch mathematical model had similar values of k cn and k nd to the ones found in the subset of 165 patients. On the other hand, the difference in the parameter values of Mismatch and the group of 33 patients affected predictive power of Mismatch on these patients and thus it was not able to provide adequate therapy. Oppositely, data driven DD-MPC generalized well what it learned on training data which resulted in succesful treatment outcomes on both groups of patients.
Comparison of therapy strategies with respect to average per healthy patient of: area under curve (AUC) of pathogen level P, AUC of tissue damage D, anti-inflammatory therapy AIDOSE, and pro-inflammatory therapy PIDOSE (lower score is better).
Delay in diagnosis
Since measurement noise is inevitable, we tested the robustness of DD-MPC to additive 5% Gaussian noise (standard deviation is 5% of the measurement) that influenced each hourly measurement of all four outputs. From Table 2 we can notice that there was no significant decrease in results compared to the results obtained with an ideal noise free system.
Partially observed data
Number and fraction of healthy, aseptic and septic patients on partially observed data for: MPC with mathematical predictive model with set of parameters equal to parameters of a single patient (Mismatch) and MPC with data-driven predictive model learned on small data sample (DD-MP C).
Healthy (total 321)
Septic (total 321)
Harmed (total 119)
Rescued (total 202)
We presented a data-driven method for acute inflammation therapy (DD-MPC), which was evaluated through a number of experiments performed on virtual patients. We demonstrated that a population of 18 healthy, aseptic and septic patients was enough to learn an efficient data-driven predictive model. Obtained results showed that DD-MPC outperformed clinically relevant alternatives, providing good results even in the presence of incomplete measurements as well as additive Gaussian noise. Furthermore, the therapy based on DD-MPC did not harm any of the healthy patients, which is a property of high importance when the method is applied in practice. Finally, promising results presented in this paper provide evidence that research on acute inflammation treatment can benefit from methods from the machine learning community.
This work was funded, in part, by DARPA grant [DARPA-N66001-11-1-4183] negotiated by SSC Pacific grant.
The publication costs for this article were funded by the corresponding author.
This article has been published as part of BMC Medical Genomics Volume 6 Supplement 3, 2013: Selected articles from the IEEE International Conference on Bioinformatics and Biomedicine 2012: Medical Genomics. The full contents of the supplement are available online at http://www.biomedcentral.com/bmcmedgenomics/supplements/6/S3.
- Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Critical care medicine. 2001, 29 (7): 1303-1310. 10.1097/00003246-200107000-00002.View ArticlePubMedGoogle Scholar
- Doyle F, Jovanovic L, Seborg D: I. Glucose control strategies for treating type 1 diabetes mellitus. Journal of Process Control. 2007, 17: 572-576. 10.1016/j.jprocont.2007.01.013.View ArticleGoogle Scholar
- Li Y, Lenaghan SC, Zhang M: A Data-Driven Predictive Approach for Drug Delivery Using Machine Learning Techniques. PLoS ONE. 2012, 7 (2): e31724-10.1371/journal.pone.0031724.PubMed CentralView ArticlePubMedGoogle Scholar
- Parker RS: II. Modeling for anti-cancer chemotherapy design. Journal of Process Control. 2007, 17 (7): 576-582. 10.1016/j.jprocont.2007.01.014.View ArticleGoogle Scholar
- Pinheiro J, Lemos J: Multi-drug therapy design for HIV-1 infection using Nonlinear Model Predictive Control. Mediterranean Conference on Control and Automation. 2011, 485-490.Google Scholar
- Clermont G, Rubin J, Day J: Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation. Mathematical Biosciences and Engineering. 2010, 7 (4): 739-763.View ArticlePubMedGoogle Scholar
- Wang Y, Zisser H, Dassau E, Jovanovic L, Doyle FJ: Model predictive control with learning-type set-point: Application to artificial pancreatic β-cell. AIChE Journal. 2009, 56 (6): 1510-1518.Google Scholar
- Noble SL, Sherer E, Hannemann RE, Ramkrishna D, Vik T, Rundell AE: Using adaptive model predictive control to customize maintenance therapy chemotherapeutic dosing for childhood acute lymphoblastic leukemia. Journal of theoretical biology. 2010, 264 (3): 990-1002. 10.1016/j.jtbi.2010.01.031.View ArticlePubMedGoogle Scholar
- Tehrani F, Roum J: Intelligent decision support systems for mechanical ventilation. Artificial Intelligence in Medicine. 2008, 44 (3): 171-182. 10.1016/j.artmed.2008.07.006.View ArticlePubMedGoogle Scholar
- Yelneedi S, Lakshminarayanan S, Rangaiah G: A comparative study of three advanced controllers for the regulation of hypnosis. Journal of Process Control. 2009, 19 (9): 1458-1469. 10.1016/j.jprocont.2009.06.003.View ArticleGoogle Scholar
- Radosavljevic V, Ristovski K, Obradovic Z: A data mining approach for optimization of acute inflammation therapy. Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on: 4-7 October 2012. 2012, 1-6. 10.1109/BIBM.2012.6392659.View ArticleGoogle Scholar
- Kocijan J, Murray-Smith R, Rasmussen C, Girard A: Gaussian process model based predictive control. American Control Conference, 2004. Proceedings of the 2004, IEEE. 2004, 2214-2219.Google Scholar
- Norgaard M, Ravn O, Poulsen NKHL: Neural networks for modeling and control of dynamic systems. 2000, London: SpringerView ArticleGoogle Scholar
- Hogg J, Clermont G, Parker R: Acute Inflammation Treatment via Particle Filter State Estimation and MPC. Proceedings of the 9th International Symposium on Dynamics and Control of Process Systems DYCOPS 2010. 2010, 258-263.Google Scholar
- Camacho EF, Bordons C: Model predictive control. 2004, London: Springer-VerlagGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.