Google AI Propose a Deep Learning–based Algorithm to Improve Medical Ventilator Control for Invasive Ventilation

Mechanical ventilators assist patients in breathing undergoing surgery or who cannot breathe independently due to acute disease. The patients are A hollow tube (artificial airway) is inserted into the patient’s mouth and down into their primary airway, or trachea, to connect them to the ventilator. 

The ventilator follows a clinician-prescribed breathing waveform based on a respiratory measurement from the patient(e.g., airway pressure, tidal volume). Therefore, it’s a challenging task and needs systems that are both robust to variances or changes in the lungs of patients and adherent to the ideal waveform to avoid injury. That is one of the reasons why they are monitored closely by highly experienced professionals ensuring that their performance meets the needs of the patients and does not cause lung damage.

Most ventilators are regulated using PID (Proportional, Integral, Differential) methods. This method regulates a system based on the history of errors between the observed and desired states. For ventilator control, a PID controller uses three characteristics: 

• Percentage (“P”), which compares measured and goal pressure
• Integral (“I”), which sums prior data
• Differential (“D”), the difference between two previous measurements. 

PID control establishes a firm foundation, depending on P control’s sharp reactivity to swiftly increase lung pressure while breathing in, and I control stability to hold the breath in before expelling. However, the ventilators must be fine-tuned for individual patients, frequently several times, to balance the “ringing” of overzealous P control with the ineffectively slow rise in lung pressure of dominant I control.

A recent Google study proposes a neural network-based controller that improves medical ventilator control by balancing these properties, reducing the risk for injury to patients’ lungs. The new control algorithm detects airway pressure and computes necessary airflow modifications to meet prescribed values using signals from an artificial lung. This method requires less manual intervention from clinicians and shows great robustness and performance compared to other systems while requiring.

The deep controller is based on deep neural networks and therefore requires a dataset for training, while a limited number of repeated trials could be used to tune the coefficients of a PID controller. Additionally, popular model-free techniques like Q-Learning and Policy Gradient are data-intensive and thus unsuited for the model. Furthermore, these approaches ignore the ventilator dynamical system’s inherent differentiability, which is deterministic, continuous, and contact-free.

The researchers adopt a model-based strategy, a more accurate data-driven alternative to physics-based models. This approach first a DNN-based ventilator-patient dynamical system simulator. 

The team explored the space of controls and the resulting pressures while keeping physical safety in mind (such as not over-inflating a test lung and causing injury) to create a training dataset. They use PID controllers with varying control coefficients to provide control-pressure trajectory data for simulator training to safely explore the system’s behavior. They also added random deviations to the PID controllers to ensure that the dynamics were captured accurately.

They used an open-source ventilator built by Princeton University’s People’s Ventilator Project to perform mechanical ventilation tasks on a physical test lung. They created a ventilator farm on a server rack with ten ventilator-lung systems. This farm captures different airway resistance and compliance settings that span a range of patient lung states, as required for practical ventilator system applications.

They model the system’s status at every given time in the simulator as a collection of prior pressure observations and control actions. These data are sent into a DNN, which forecasts the system’s subsequent pressure. The control-pressure trajectory data gathered through the test lung is used to train this simulator. 

The researchers use a learned simulator to train a DNN-based controller offline. This method enables quick updates while training the controller. Furthermore, the simulator’s differentiable nature allows for the stable application of the direct policy gradient. This strategy is substantially more efficient than model-free alternatives as it computes the loss gradient with respect to the DNN parameters analytically.

The researchers compare the best-performing PID controller to their proposed method for multiple settings. For certain lung settings. The results show that the proposed controller performs better compared to the PID controller and shows 22 percent lower mean absolute error (MAE) between target and actual pressure waveforms.

They also compare the performance of the single best PID controller throughout the complete set of lung parameters to that of their controller trained on a set of learned simulators in the same settings. The team’s controller outperforms the competition in MAE between goal and actual pressure waveforms by up to 32%. This demonstrates that the proposed controller would require less manual intervention between patients or as a patient’s condition changes.

The team also evaluated the viability of employing model-free and other common RL methods (PPO, DQN). For this, they compared their controllers to a straight policy gradient trained on the simulator. Their findings show that the simulator-trained direct policy gradient obtains slightly higher scores while using a more stable training method with orders of magnitude fewer training samples and a substantially smaller hyperparameter search space.

Paper: https://arxiv.org/abs/2102.06779

Reference: https://ai.googleblog.com/2022/02/machine-learning-for-mechanical.html

Suggested