Gone are the days when people use smart speakers -- like Amazon Echo or Google Home -- only as kitchen timers or dinner party music players. These devices have started helping people track their own health, and can even monitor for cardiac arrest.
Now researchers at the University of Washington have developed a new smart speaker skill that lets a device use white noise to both soothe sleeping babies and monitor their breathing and movement.
With this skill, called BreathJunior, the smart speaker plays white noise and records how the noise is reflected back to detect breathing motions of infants' tiny chests. When the researchers tested BreathJunior with five babies in a local hospital's neonatal intensive care unit, it detected respiratory rates that closely matched the rates detected by standard vital sign monitors. The team will present its findings October 22 at the MobiCom 2019 conference in Los Cabos, Mexico.
"One of the biggest challenges new parents face is making sure their babies get enough sleep. They also want to monitor their children while they're sleeping. With this in mind, we sought to develop a system that combines soothing white noise with the ability to unobtrusively measure an infant's motion and breathing," said co-author Dr. Jacob Sunshine, an assistant professor of anesthesiology and pain medicine at the UW School of Medicine.
To make things easy for new parents, the team made a system that could run on a smart speaker that replicates the hardware in an Amazon Echo.
"Smart speakers are becoming more and more prevalent, and these devices already have the ability to play white noise," said co-author Shyam Gollakota, an associate professor in the UW's Paul G. Allen School of Computer Science & Engineering and the director of the UW computing for health group. "If we could use this white noise feature as a contactless way to monitor infants' hand and leg movements, breathing and crying, then the smart speaker becomes a device that can do it all, which is really exciting."
White noise is a combination of different sound frequencies, which makes a seemingly random soothing sound that can help cover up other noises that might wake a sleeping baby. To use white noise as a breathing monitor, the team needed to develop a method to detect tiny changes between the white noise a smart speaker plays and the white noise that gets reflected back from the infant's body into the speaker's array of microphones.
"We start out by transmitting a random white noise signal. But we are generating this random signal, so we know exactly what the randomness is," said first author Anran Wang, a doctoral student in the Allen School. "That signal goes out and reflects off the baby. Then the smart speaker's microphones get a random signal back. Because we know the original signal, we can cancel out any randomness from that and then we're left with only information about the motion from the baby."
Detecting breathing in babies has an extra wrinkle: the movement of their chests is so tiny that the smart speaker needs to know exactly where the babies are to be able to "see" them breathing.
"The breathing signal is so weak that we can't just look for a change in the overall signal we get back," Wang said. "We needed a way to scan the room and pinpoint where the baby is to maximize changes in the white noise signal. Our algorithm takes advantage of the fact that smart speakers have an array of microphones that can be used to focus in the direction of the infant's chest. It starts listening for changes in a bunch of potential directions, and then continues the search toward the direction that gives the clearest signal."
BreathJunior tracks both small motions -- such as the chest movement involved in breathing -- and large motions -- such as babies moving around in their cribs. It can also pick up the sound of a baby crying.
The team created a prototype smart speaker to test BreathJunior on an infant simulator. The researchers could set the simulator to breathe at specific rates, which allowed them to test how well BreathJunior detected a variety of respiratory rates -- from a slow 20 breaths per minute to 60 breaths per minute. The infant simulator also allowed the team to test if BreathJunior could detect abnormal breathing patterns, such as apnea, that are common in babies who are born early and may not have developed respiratory centers in their brains. The system performed well for both tests.
Then the team tested how well their prototype tracked real babies' breathing in the neonatal intensive care unit or NICU. These babies are connected to wired, hospital-grade respiratory monitors, so the team could compare their readouts to BreathJunior's. The system was able to accurately identify respiratory rates up to 65 breaths per minute.
"Infants in the NICU are more likely to have either quite high or very slow breathing rates, which is why the NICU monitors their breathing so closely," Sunshine said. "BreathJunior holds potential for parents who want to use white noise to help their child sleep and who also want a way to monitor their child's breathing and motion. It also has appeal as a tool for monitoring breathing in the subset of infants in whom home respiratory monitoring is clinically indicated, as well as in hospital environments where doctors want to use unwired respiratory monitoring.
"However, it is very important to note that the American Academy of Pediatrics recommends not using a monitor that markets itself as reducing the risk of sudden infant death syndrome, and this research and the team makes no such claim."
While BreathJunior currently uses white noise to track breathing and motion, the researchers would like to expand its capabilities so that it could also use other soothing sounds like lullabies.
The team plans to commercialize this technology through a UW spinout, Sound Life Sciences, Inc.
"In just a few years, we have come a long way from monitoring large motions in adults to extracting the tiny motion of a newborn infant's breathing," Gollakota said. "This has been possible because of algorithmic innovations as well as advances in smart speaker hardware. Looking ahead, one can envision transforming a smart speaker into a 'medical tricorder' that can contactlessly monitor a variety of vital signs beyond just breathing."
