Application of Deep Recurrent Q Network with Dueling Architecture for Optimal Sepsis Treatment Policy
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Sepsis is one of the leading causes of mortality globally, and it costs billions of dollars annually. However, treating septic patients is currently highly challenging, and more research is needed into a general treatment method for sepsis. Therefore, in this work, we propose a reinforcement learning method for learning the optimal treatment strategies for septic patients. We model the patient physiological time series data as the input for a deep recurrent Q-network that learns reliable treatment policies. We evaluate our model using an off-policy evaluation method, and the experimental results indicate that it outperforms the physicians’ policy, reducing patient mortality up to 3.04%. Thus, our model can be used as a tool to reduce patient mortality by supporting clinicians in making dynamic decisions.
Abstract
Sepsis is one of the leading causes of mortality globally, and it costs billions of dollars annually. However, treating septic patients is currently highly challenging, and more research is needed into a general treatment method for sepsis. Therefore, in this work, we propose a reinforcement learning method for learning the optimal treatment strategies for septic patients. We model the patient physiological time series data as the input for a deep recurrent Q-network that learns reliable treatment policies. We evaluate our model using an off-policy evaluation method, and the experimental results indicate that it outperforms the physicians’ policy, reducing patient mortality up to 3.04%. Thus, our model can be used as a tool to reduce patient mortality by supporting clinicians in making dynamic decisions.
참고문헌 (12)
- M. Singer, "The third international consensus definitions for sepsis and septic shock(sepsis-3)", J. Am. Med. Assoc, 315(8), pp.801~810 (2016)
- Christopher W. Seymour, "Time to treatment and mortality during mandated emergency care for sepsis", New England Journal of Medicine, 376(23), pp.2235~2244 (2017)
- L. Byrne, "Fluid resuscitation in human sepsis : time to rewrite history?", Ann Intensive Care, 7(4) (2017)
- P. Marik, "A rational approach to fluid therapy in sepsis", Br. J. Anaesth, 116(3), pp.339~349 (2016)
- Z. Wang, "Dueling network architectures for deep reinforcement learning", arXiv:1511.06581, ~ (2015)
- A. Raghu, "Deep Reinforcement Learning for Sepsis Treatment", CoRR, abs, 2017
- X. Peng, "Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning", AMIA Annu Symp Proc, pp.887~896 (2018)
- Diederik P Kingma, "Auto-encoding variational bayes", CoRR, abs, 2014
- E.W. Johnson, "MIMIC-III, a freely accessible critical care database", Scientific Data, 3(160035) (2016)
- A. Raghu, "Continuous state-space models for optimal sepsis treatment-a deep reinforcement learning approach", Proceedings of the 2nd Machine Learning for Healthcare Conference, pp.147~163 (2017)
- M. Hausknecht, "Deep recurrent q-learning for partially observable mdps", CoRR, abs, 2015
- N. Jiang, "Doubly Robust Off-policy Evaluation for Reinforcement Learning", Proceedings of The 33rd International Conference on Machine Learning, 48, pp.652~661 (2016)