A datadriven acute inflammation therapy
 Vladan Radosavljevic†^{1},
 Kosta Ristovski†^{1} and
 Zoran Obradovic^{1}Email author
https://doi.org/10.1186/175587946S3S7
© Radosavljevic et al; licensee BioMed Central Ltd. 2013
Published: 11 November 2013
Abstract
Acute inflammation is a severe medical condition defined as an inflammatory response of the body to an infection. Its rapid progression requires quick and accurate decisions from clinicians. Inadequate and delayed decisions makes acute inflammation the 10th leading cause of death overall in United States with the estimated cost of treatment about $17 billion annually. However, despite the need, there are limited number of methods that could assist clinicians to determine optimal therapies for acute inflammation. We developed a datadriven method for suggesting optimal therapy by using machine learning model that is learned on historical patients' behaviors. To reduce both the risk of failure and the expense for clinical trials, our method is evaluated on a virtual patients generated by a mathematical model that emulates inflammatory response. In conducted experiments, acute inflammation was handled with two complimentary pro and antiinflammatory medications which adequate timing and doses are crucial for the successful outcome. Our experiments show that the dosage regimen assigned with our datadriven method significantly improves the percentage of healthy patients when compared to results by other methods used in clinical practice and found in literature. Our method saved 88% of patients that would otherwise die within a week, while the best method found in literature saved only 73% of patients. At the same time, our method used lower doses of medications than alternatives. In addition, our method achieved better results than alternatives when only incomplete or noisy measurements were available over time as well as it was less affected by therapy delay. The presented results provide strong evidence that models from the artificial intelligence community have a potential for development of personalized treatment strategies for acute inflammation.
Introduction
Acute inflammation is a progressively severe medical condition defined as an inflammatory response of the body to a trauma, a surgery, a burn, or an infection. Fast progression of acute inflammation requires quick and accurate decisions from clinicians. Inadequate and delayed decisions make acute inflammation the 10th leading cause of death overall in United States, with the estimated cost of treatment about $17 billion annually [1]. However, despite the need, there are no computational methods that can help clinicians in planning optimal therapies for acute inflammation.
Model predictive control (MPC) is a method often used in clinical applications for planing of optimal treatment [2–5] mainly due to its inherent capability to handle clinically relevant constraints and to take into account multiple variables at the same time. A predictive model is the most important part of MPC, which is deployed to estimate patient's response to the therapy. These estimations together with a set of clinically relevant constraints are used in MPC to compute optimal therapies that will lead the patient to the healthy condition. The efficacy of MPC based treatment is completely dependent on model's capability to accurately estimate the patient's response to therapy.
To construct an accurate predictive model, practitioners often rely on domainbased assumptions about patients' behavior. Such a domaindriven model was previously used as a component in MPC to find optimal acute inflammation treatment [6]. The authors showed that proinflammatory and antiinflammatory medications within MPC setup decreased the mortality rate. However, therapy outcome was greatly dependent on the set of parameters used in the model.
To relax the dependency on parameter settings, a datadriven predictive model that learns patients' responses from observed data and doesn't rely on any domainbased hypothesis can be used instead. MPC with datadriven predictive models was successfully applied in medical applications, including optimal glucose control [7], the control of anticancer medications dosage [8], the control of ventilation [9], and finding an adequate dosage of anesthesia [10]. The main objective of this study is to show that MPC with a datadriven predictive model can improve the overall quality of acute inflammation treatments.
Our preliminary results [11] indicated that a datadriven MPC method significantly improved the number of healthy outcomes when compared to acute inflammation therapy strategies from literature and clinical practice. This paper reports a more detailed study by evaluating the datadriven MPC and alternative approaches over different clinically relevant setups as follows. All experiments are done on virtual patients generated by a mathematical model that emulates inflammatory response, which is a standard approach in pharmacological research. We show that in achieving results presented in [11], our method used much lower doses of medications than other methods. Furthermore, our method provides good accuracy even in the presence of incomplete measurements, as well as additive Gaussian noise. It is also less affected by possible therapy delay than the alternatives. We also identified characteristics of patients who may benefit the most from application of our method. Consequently, our datadriven tool for the optimal treatment of acute inflammation along with domainbased knowledge may provide a platform for the development of personalized treatment strategies that will increase survival rates.
The rest of the paper is organized as follows. In the next section we introduce a virtual patient model that emulates therapy response and we define clinically relevant therapy constraints. Then we propose a datadriven predictive model that determines optimal therapy, which is followed by evaluation of the model. In the final section we give a conclusion.
Mathematical model for virtual patient and treatment constraints
Virtual patients are carefully developed mathematical models made to mimic the human body behavior when exposed to circumstances that are in the interest of a study. They allow biomedical researchers to perform various types of experiments on the same patient as well as to compare the outcomes. Thus, the main purpose of having virtual patients is to reduce expense for clinical trials and to lower chance of failures. A mathematical model in the form of the system of ordinary differential equations (ODE), recently proposed in [6], simulates inflammatory response to an infection, including medication effect on inflammatory response. It models the dynamics of concentration of:

bacterial pathogen (P),

early proinflammatory mediators (N),

tissue damage markers (D),

antiinflammatory mediators (CA),
Mathematical model in this form is capable of modeling the complex cascade of inflammation initiated by pathogen (P). An increase of pathogen level P leads to the series of positive and negative feedback reactions that are all successfully modeled by ODE. In particular, an increase of P causes the development of a proinflammatory response (the increase of N) and the development of tissue damage (the increase of D). Equation (1) simulates a positive effect of inflammation, where an increase of N reduces level of pathogen P. However, (3) simulates a negative effect of inflammation, where an increase of N further damages tissue causing rapid increase of D. An increase of D activates a negative feedback in (4), or antiinflammatory response (CA), which lowers the level of N and prevents tissue damage (decrease of D) [6]. The strength of positive and negative feedbacks depends on the parameter values in ODE. By varying parameter values we can simulate variability among patients.

healthy (P <1, N <0.05, D <1),

aseptic death (P <1, N >0.05, D >1),

septic death (P >1, N >0.05, D >1).
Evolution of the patient to the final state can be modulated by carefully constrained proinflammatory (PIDOSE) and antiinflammatory (AIDOSE) medications. Therefore, model predictive control is applicable to inflammation therapy only if constraints on medication doses obey clinical rules. Oppositely, for example, a large amount of medication given at once can cause the death of a patient due to overdose. Also, the prescription of a high level of antiinflammatory doses with long duration may predispose the patient to other infections, which might cause death. We follow welldefined medication constraints for inflammation treatment stated in [6]:

$0\le PIDOSE\le PIDOS{E}_{{}^{k}}^{MAX}$, where $PIDOS{E}_{k}^{MAX}$ is the difference between N_{ max }(N_{ max }= 0.5) and the current level of N = N_{ k };

$0\le AIDOSE\le AIDOS{E}_{{}^{k}}^{MAX}$, where $AIDOS{E}_{k}^{MAX}$ is the difference between a maximum allowable level of CA (initialized to CA_{ max }= 0.6264) and the current level of CA = CA_{ k };

saturation of antiinflammatory mediator: the saturation of CA for long durations is avoided in clinical practice because of other infections that can occur and endanger organ recovery. Thus, if the level of CA has been elevated for 48 hours, then CA_{ max }was halved.
Model predictive control (MPC)

prediction horizon (p) is the time window that governs how many future outputs of the predictive model will be used in the objective function;

predicted patient states$\left\{{\left[\widehat{P}\phantom{\rule{0.25em}{0ex}}\widehat{N}\widehat{D}\u0108A\right]}_{k+j},j=\mathsf{\text{1}},...,p\right\}$ are estimations of output variables obtained from the predictive model;

control horizon (c, c ≤ p) is the time window that governs how many future controls (medication dosage) will be determined by the optimization algorithm;

future medication doses$\left\{{\left[PI\widehat{D}OSEAI\widehat{D}OSE\right]}_{k+j},j=0,...,c1\right\}$ are doses that are obtained by the optimization algorithm assuming constant doses from k + c to k + p − 1 set to the values of. ${\left[PI\widehat{D}OSEAI\widehat{D}OSE\right]}_{k+c\mathsf{\text{1}}}$
where w_{ D }, w_{ P }, and w_{ c } are weighting constants. Then, the first doses $AI\widehat{D}OS{E}_{k}$ and $PI\widehat{D}OS{E}_{k}$are applied to a virtual patient model to obtain new patient state at k + 1. At the next time point k + 1, the optimization procedure is repeated and a new sequence of doses is obtained.
Finding successful treatment by applying two medications is a challenging problem. Applying a large amount of AIDOSE would increase the level of CA, which would reduce the level of D. On the other hand, this might cause extreme growth of pathogen P. Also, applying a large amount of PIDOSE would increase the level of N, which would reduce the level of P. However, this can trigger proinflammatory response to focus on eliminating P, after which the level of tissue damage D could not be controlled anymore. Thus, both medication doses and their timing are critical for successful treatment.
Predictive model
Accuracy of the predictive model over the prediction horizon directly affects the quality of model predictive control. Design of our datadriven predictive model involves two steps: 1) specification of model structure 2) learning model parameters from training data. These steps will be further explained in following paragraphs. For the sake of clarity we named our predictive model as DDMPC.
Structure of DDMPC predictive model
where j = 1,..., p; F_{ P } is a function with unknown parameters β_{ P }; n_{ py } and n_{ pu } are time lags. Other submodels responsible for prediction of N, D, and CA have similar functional forms, but with different parameters β_{ N }, β_{ D }, and β_{ CA }.
The next step is to define inputoutput functional relations F_{ P }, F_{ N }, F_{ D }, or F_{ CA } in terms of model parameters. Data driven nonlinear predictive models such as Gaussian processes [12], neural networks [13], particle filters [14] have been successfully developed for industrial applications, although designing nonlinear models in control theory is difficult task [15]. Also, nonlinear models require much more data to be trained then linear models. Moreover, in the case of multivariate process, the difference in necessary training data is even larger. Due to high cost and limited availability of clinical data we applied linear functional forms in terms of β_{ P }, β_{ N }, β_{ D }, β_{ CA }.
Learning of DDMPC predictive model
To follow a real life scenario regarding the availability of representative training data, we assume that the data come from a small population of diverse patients. To generate training data we select N_{ tr } virtual patients represented by ODE and consider that hourly measurements are available for a whole week. During the data generation process, initial conditions and parameters for each patient are randomly chosen from valid ranges while dosages are determined as follows.
In spite of control theory where control signals are randomly generated, random generation would not be clinically relevant for medication doses and thus we designed the following approach. We use MPC for each patient in the training set, deploying its own ODE as a predictive model. In this setup, the predictive model provides perfect predictions for patient's future states because predictions and future observations are the same at every time point. Therefore, by applying MPC we obtain ideal medication dosage. Ideal dosage is neither realistic in clinical practice nor suitable for learning datadriven models. Therefore, we added random Gaussian noise to the ideal sequence of doses, obtaining dosage that is closetoideal. On the other hand, closetoideal doses applied every hour over an entire week would lead many patients to a nonhealthy state. Instead, closetoideal dosage strategy is applied in the first 10 hours of therapy (the most critical period) and then therapy is continued with the ideal doses. This strategy is called closetoideal dosage strategy.
Evaluation
Parameters of DDMPC
In order to evaluate DDMPC we need to decide on time lags, prediction horizon, and control horizon. The model with time lags equal to 2 had significantly larger predictive accuracy than the models with time lags different from 2. Therefore we use the model with time lags set to 2 in all of our experiments. The duration of the prediction horizon had to be set to achieve an equilibrium between following requirements: (1) the duration should be long enough in order to allow the therapy to fully manifest, (2) the duration should be short enough since predictive models are usually less reliable as the duration increases. The best performance was achieved with the length of prediction horizon window set to 5. The longer duration of control horizon makes response to be faster, but the system then becomes susceptible to model uncertainties [15]. Therefore we set the window length of control horizon to 2. Weights w_{ P }, w_{ D }, and w_{ c } in objective function were set to 3, 1, and 1 respectively in order to amortize the effect of unequal scalings.
Number of patients to train DDMPC
Obtaining the data is expensive in clinical applications. Hence, it is important that we investigate the least number of patients who are needed for the training data set so that the model learned on such data achieves acceptable accuracy. We generated a population of 65 virtual patients with the closetoideal dosage strategy for each of them. Each patient was observed for 168 hours and data were recorded on hourly bases. Finally, there were 43 healthy, 14 aseptic, and 8 septic patients' outcomes. It is worth noting that during training of the proposed datadriven model we did not take into account any prior knowledge about the data generation process.
Average number of healthy, aseptic and septic patients after DDMPC therapy on a set of 50 patients.
N _{ tr }  Healthy  Aseptic  Septic 

3  24.1 ± 9.7  12.7 ± 9.0  13.2 ± 7.4 
6  26.6 ± 12.1  10.0 ± 5.8  13.4 ± 9.2 
9  37.8 ± 11.1  7.4 ± 4.7  4.8 ± 8.1 
12  41.7 ± 3.5  6.0 ± 2.0  2.3 ± 4.6 
15  41.7 ± 3.2  5.8 ± 2.3  2.5 ± 5.0 
18  44.1 ± 1.1  5.8 ± 0.7  0.3 ± 1.0 
21  44.3 ± 1.2  5.3 ± 0.5  0.4 ± 1.2 
24  43.6 ± 0.8  5.9 ± 0.9  0.5 ± 1.2 
Dataset
The proposed DDMPC was evaluated on a population of 500 virtual patients recorded for 168 hours (1 week) by hourly measurements of P, N, D, and CA. In the case when treatment is not applied, the population has 85 septic, 117 aseptic, and 298 healthy virtual patients. In order to determine which patients require treatment, we follow the criteria from [6]. According to [6], if N exceeds 0.05 at any time point, then the corresponding patient needs to receive treatment. We found 321 out of 500 patients who were supposed to be treated. These patients represent a population on which we compare DDMPC and baseline methods.
Baseline methods
We compare therapy outcomes of our DDMPC to models used in [6]:

no therapy applied model (Placebo);

the therapy currently used in the clinical practice: a constant antiinflammatory therapy; we simulated this kind of therapy by applying AIDOSE = 0.005 hourly during the fist 72 hours from therapy onset (N >0.05), after 72 hours the therapy was terminated (Static);

therapy based on MPC that used the mathematical model which parameters were fixed to parameters from a single patient [6] (Mismatch).
Results
Therapy outcome was classified based on patient state at the end of 168 hours of simulation. If the patient did not reach stable state after 168 hours, then the simulation was extended for an additional 300 hours with no therapy applied. We report percentage and number of septic, aseptic and healthy outcomes at the end. In addition, we distinguish different groups of patients based on the effect of therapy:

harmed  designates cases when the outcomes of therapy without using any medications (Placebo) were healthy, while outcomes after applying therapy were aseptic or septic (the lower percentage of harmed the better);

rescued  designates cases when outcomes after therapy were healthy, while outcomes without therapy were aseptic or septic (the higher percentage of rescued the better).
Fully observed data
Number and fraction of patients on fully observed data for: model with no therapy applied (Placebo), model with constant antiinflammatory dose (Static), MPC with mathematical predictive model with set of parameters equal to parameters of a single patient (Mismatch) and MPC with datadriven predictive model learned on small data sample (DDMP C) and MPC with datadriven predictive model learned on small data sample with 5% additive Gaussian noise in observations (DDMPC+nois e).
Healthy (total 321)  Aseptic (total 321)  Septic (total 321)  Harmed (total 119)  Rescued (total 202)  

Placebo  119 (37.07%)  117 (36.45%)  85 (26.48%)  N/A  N/A 
Static  140 (43.61%)  96 (29.91%)  85 (26.48%)  3 (2.52%)  24 (11.88%) 
Mismatch  267 (83.18%)  50 (15.58%)  4 (1.25%)  0 (0%)  148 (73.27%) 
DDMPC  294 (91.59%)  24 (8.41%)  0 (0%)  0 (0%)  175 (86.63%) 
DDMPC+noise  284 (88.47%)  33 (10.28%)  4 (1.25%)  0 (0%)  165 (81.68%) 
To identify patients (represented by initial conditions and model parameters) who benefit the most from using DDMPC, we compare the outcomes of the DDMPC model to the outcomes of Mismatch. According to possible outcomes, we split patients into two groups: (1) 266 patients who were successfully treated by both models, and (2) 33 patients who were successfully treated by DDMPC but not Mismatch. The group of 33 patients is characterized by high initial level of pathogen P_{0} (P_{0} = 0.78 ± 0.13). In order to detect the conditions for which DDMPC outperforms Mismatch, we compare 33 patients to a subset of 165 patients from another group who had similar P_{0} (P_{0} = 0.79 ± 0.12). Average values of parameters k_{ cn } and k_{ nd } were significantly different between the group of 33 patients and the subset of 165 patients according to two tailed ttest with 95% confidence. We discovered that the Mismatch mathematical model had similar values of k_{ cn } and k_{ nd } to the ones found in the subset of 165 patients. On the other hand, the difference in the parameter values of Mismatch and the group of 33 patients affected predictive power of Mismatch on these patients and thus it was not able to provide adequate therapy. Oppositely, data driven DDMPC generalized well what it learned on training data which resulted in succesful treatment outcomes on both groups of patients.
Therapy efficiency
Comparison of therapy strategies with respect to average per healthy patient of: area under curve (AUC) of pathogen level P, AUC of tissue damage D, antiinflammatory therapy AIDOSE, and proinflammatory therapy PIDOSE (lower score is better).
AUC(P)  AUC(D)  PIDOSE  AIDOSE  

Mismatch  4.56  182.23  0.3053  0.8301 
DDMPC  4.39  147.46  0.2261  0.7814 
Delay in diagnosis
Noisy data
Since measurement noise is inevitable, we tested the robustness of DDMPC to additive 5% Gaussian noise (standard deviation is 5% of the measurement) that influenced each hourly measurement of all four outputs. From Table 2 we can notice that there was no significant decrease in results compared to the results obtained with an ideal noise free system.
Partially observed data
Number and fraction of healthy, aseptic and septic patients on partially observed data for: MPC with mathematical predictive model with set of parameters equal to parameters of a single patient (Mismatch) and MPC with datadriven predictive model learned on small data sample (DDMP C).
Healthy (total 321)  Aseptic (total 321)  Septic (total 321)  Harmed (total 119)  Rescued (total 202)  

Mismatch  250 (77.88%)  59 (18.38%)  12 (3.74%)  2 (1.68%)  133 (65.84%) 
DDMPC  267 (83.18%)  12 (3.74%)  42 (13.08%)  0 (0%)  148 (73.27%) 
Conclusion
We presented a datadriven method for acute inflammation therapy (DDMPC), which was evaluated through a number of experiments performed on virtual patients. We demonstrated that a population of 18 healthy, aseptic and septic patients was enough to learn an efficient datadriven predictive model. Obtained results showed that DDMPC outperformed clinically relevant alternatives, providing good results even in the presence of incomplete measurements as well as additive Gaussian noise. Furthermore, the therapy based on DDMPC did not harm any of the healthy patients, which is a property of high importance when the method is applied in practice. Finally, promising results presented in this paper provide evidence that research on acute inflammation treatment can benefit from methods from the machine learning community.
Notes
Declarations
Acknowledgments
This work was funded, in part, by DARPA grant [DARPAN660011114183] negotiated by SSC Pacific grant.
Declarations
The publication costs for this article were funded by the corresponding author.
This article has been published as part of BMC Medical Genomics Volume 6 Supplement 3, 2013: Selected articles from the IEEE International Conference on Bioinformatics and Biomedicine 2012: Medical Genomics. The full contents of the supplement are available online at http://www.biomedcentral.com/bmcmedgenomics/supplements/6/S3.
Authors’ Affiliations
References
 Angus DC, LindeZwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Critical care medicine. 2001, 29 (7): 13031310. 10.1097/0000324620010700000002.View ArticlePubMedGoogle Scholar
 Doyle F, Jovanovic L, Seborg D: I. Glucose control strategies for treating type 1 diabetes mellitus. Journal of Process Control. 2007, 17: 572576. 10.1016/j.jprocont.2007.01.013.View ArticleGoogle Scholar
 Li Y, Lenaghan SC, Zhang M: A DataDriven Predictive Approach for Drug Delivery Using Machine Learning Techniques. PLoS ONE. 2012, 7 (2): e3172410.1371/journal.pone.0031724.PubMed CentralView ArticlePubMedGoogle Scholar
 Parker RS: II. Modeling for anticancer chemotherapy design. Journal of Process Control. 2007, 17 (7): 576582. 10.1016/j.jprocont.2007.01.014.View ArticleGoogle Scholar
 Pinheiro J, Lemos J: Multidrug therapy design for HIV1 infection using Nonlinear Model Predictive Control. Mediterranean Conference on Control and Automation. 2011, 485490.Google Scholar
 Clermont G, Rubin J, Day J: Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation. Mathematical Biosciences and Engineering. 2010, 7 (4): 739763.View ArticlePubMedGoogle Scholar
 Wang Y, Zisser H, Dassau E, Jovanovic L, Doyle FJ: Model predictive control with learningtype setpoint: Application to artificial pancreatic βcell. AIChE Journal. 2009, 56 (6): 15101518.Google Scholar
 Noble SL, Sherer E, Hannemann RE, Ramkrishna D, Vik T, Rundell AE: Using adaptive model predictive control to customize maintenance therapy chemotherapeutic dosing for childhood acute lymphoblastic leukemia. Journal of theoretical biology. 2010, 264 (3): 9901002. 10.1016/j.jtbi.2010.01.031.View ArticlePubMedGoogle Scholar
 Tehrani F, Roum J: Intelligent decision support systems for mechanical ventilation. Artificial Intelligence in Medicine. 2008, 44 (3): 171182. 10.1016/j.artmed.2008.07.006.View ArticlePubMedGoogle Scholar
 Yelneedi S, Lakshminarayanan S, Rangaiah G: A comparative study of three advanced controllers for the regulation of hypnosis. Journal of Process Control. 2009, 19 (9): 14581469. 10.1016/j.jprocont.2009.06.003.View ArticleGoogle Scholar
 Radosavljevic V, Ristovski K, Obradovic Z: A data mining approach for optimization of acute inflammation therapy. Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on: 47 October 2012. 2012, 16. 10.1109/BIBM.2012.6392659.View ArticleGoogle Scholar
 Kocijan J, MurraySmith R, Rasmussen C, Girard A: Gaussian process model based predictive control. American Control Conference, 2004. Proceedings of the 2004, IEEE. 2004, 22142219.Google Scholar
 Norgaard M, Ravn O, Poulsen NKHL: Neural networks for modeling and control of dynamic systems. 2000, London: SpringerView ArticleGoogle Scholar
 Hogg J, Clermont G, Parker R: Acute Inflammation Treatment via Particle Filter State Estimation and MPC. Proceedings of the 9th International Symposium on Dynamics and Control of Process Systems DYCOPS 2010. 2010, 258263.Google Scholar
 Camacho EF, Bordons C: Model predictive control. 2004, London: SpringerVerlagGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.