Smart homes could make life easier

By Lori DeBoer

When Michael Mozer purchased a late-1800s former schoolhouse in Marshall, he wanted to do something special with it.

A professor in the Department of Computer Science and the Institute of Cognitive Science at CU Boulder, Mozer was interested in combining his work with artificial intelligence (AI) with renovating his home. His idea quickly turned into a university research project.

Michael Mozer, a CU Boulder professor, created one of the first homes in Marshall, Colo., that used sensors and machine learning to predict the homeowner’s habits. (photo courtesy Michael Mozer)

He and his students installed 75 sensors, miles of conductors and numerous actuators (controls for lighting, ventilation, air and water heating) in the house, so a computer attached to them could learn his behaviors to anticipate household needs. “There are sufficient regularities in my behavior that the system could exploit to save energy and make life more comfortable,” he says. For example, if Mozer got up in the middle of the night, the home’s AI would light the way to the bathroom.

Controlling the lights using machine learning meant that the AI technology had to infer from the motion patterns in a room what the occupant was doing.

Mozer’s project was well ahead of the technology curve. When he started it in 1992, the Internet had been open to the public only a short time. His project wound down in 2004, after a lightning strike fried the electrical system. But his experience gave him insight into AI’s potential in a home environment.

The discussion about home automation is always the same, he says: How your refrigerator is going to tell you if you’re out of milk or that it’s time to take your pills. But the real excitement isn’t solely in the forward-facing, standalone devices you need to program or can talk to, but in AI that actually learns your habits.

“Everything in AI and machine learning is about doing optimization of some sort,” he says. So a home-thermostat system that uses AI could not only balance comfort with energy costs, it could help you hit a targeted budget. “It could use weather forecasts and occupancy patterns to figure out what energy needs it can provide you for your budget,” Mozer says. “To me, that is the most useful implementation of AI: Stuff going on in the background, doing the work for you in a way that you don’t have to think about.” But, he adds, we have a ways to go before we have truly useful systems, not just novelties.

Connectivity Is Key

Coordinating all your home’s smart devices using natural language instead of conventional programming commands is the mission of Alex Capecelatro, who founded Denver-based His work is leaving Siri and Alexa in the dust, because he’s developing a better voice-activated system that can auto­discover all the devices on a home network, including smart televisions, music systems, thermometers, security cameras, lighting and window shades.

Next-generation technology from makes it possible to use natural language to talk to your devices, and automatically connects all the devices on a network. (photo courtesy

Tying single-use products together in a way that works seamlessly for the user is crucial, Capecelatro says. “Our approach to AI is the idea of interacting with technology as you would a person,” he says. His company is working to adapt AI technology to how people really speak, so instead of saying, “Set the temperature to 72 degrees,” being able to say,

“It’s cold in here,” and the system would respond by boosting the temperature.
Capecelatro also wants technology to be self-monitoring so it can let inhabitants know when a light bulb is out or a device is off-line. In the future, homes will have “ambient cognification,” he says, which means AI will constantly monitor the home’s data. “We are close to being able to tell from your breathing, motions or behavior if you are happy or sad, or if you just worked out, to determine what kind of music AI should play or what kind of temperature AI should set.”

A house with this cognitive ability would also notice if you forgot something on the stove when you went to shower, and eliminate that potentially hazardous situation by turning off the stove.

“Currently, the home and your life are separate but we are heading to where they are more connected,” Capecelatro says. “The house will understand your calendar, your emails, your alarms, your timers and just make the whole thing potentially better.”

Home Robots

While we’re at least a decade away from having a viable in-home robot, à la Rosie from The Jetsons, Boulder-based Misty Robotics is moving toward that. The new company is a spin-off from Sphero, a niche maker of smartphone-controlled robotic balls whose products were the basis for the BB-8, the engaging Star Wars droid. Misty CEO Tim Enwall is rolling out robots targeted toward the maker or hobby programmer.

Misty Robotics is rolling out technology like the Misty II for computer programmers to innovate with their own scripts. (photo courtesy Misty Robotics)

Misty robots can perform a number of functions straight out of the box, including managing home devices and photos, playing audio, mapping rooms, and recognizing individuals. The robots will eventually be outfitted with accessories, such as arms, and the company is working on programming each one with its own persona. But it’s the robots’ ability to be programmed with their owners’ personalized scripts that is going to lay the foundation for the future of home robotics, Enwall says.

“The essence of the story [of home robotics] is no different than the story of the personal computer and how it developed, or the Internet and how it developed, or the smartphone and how it developed,” says Enwall. “These stories never start with a consumer-use case; they always start with programmers.”

Misty is banking on delivering the core technology that will allow innovators to develop applications for it. Enwall has been working in the IoT (Internet of Things) domain since 2004 and launched a smart-home business called Evolv that he later sold to Google.

Eventually, he says, home robots will do myriad tasks, including keeping tabs on seniors and kids. Currently, one limitation of home AI is the need for multiple sensors in a home. Robots’ mobility will reduce that need, Enwall believes.

“If you truly want to learn about the habits, practices and patterns of inhabitants of a space—whether that is an office or home—then you are going to have to have a lot of sensors; they need to be everywhere to learn.” If you multiply the price per sensor with all the sensors a smart home needs to be truly intelligent, then your cost is in the thousands, he says. A domestic robot that learns an occupant’s habits would be more affordable and should be able to process information locally, instead of in the cloud, which would alleviate a few privacy concerns.

But one barrier to integrating robots in the home is power requirements. “Batteries of today aren’t sized and capable enough to drive C-3PO,” Enwall says. “It requires a lot of power to lift the laundry out of the washing machine into the dryer or put dishes into the dishwasher.”

Efficient House Hunting

AI will even change the way people purchase houses, predicts Jack Ryan, the CEO of Rex Real Estate Exchange, which opened a Denver office in April. Ryan’s company is currently the only real estate firm using AI to better match home buyers with homes and sellers with buyers.

“There are so many applications for robots and AI in residential real estate,” says Ryan, who was inspired by his own home-buying experience to introduce more efficiency into the process. During his 20 years at Goldman Sachs, he saw trades previously handled by humans done by supercomputers. The efficiencies introduced by automation drove down the price of trading.

Home buyers can use the AI technology developed by Rex Real Estate Exchange to make shopping simpler. (photo courtesy Rex Real Estate Exchange)

His 3-year-old company is driving down home-buying real estate fees from 6 to 2 percent, he says, because AI can make a better prediction about who is going to be a buyer of a house by looking at numerous data points and other information. For example, if a home is in the $500,000 to $800,000 price range, AI can predict that 80 percent of the buyers will already live within a 14-mile radius of the house. Homes in the $1-to-$2-million range are more likely to attract national and international buyers.

AI will not eliminate the human experience, however. “There are some things you still need emotional intelligence for, and buying and selling a home is still an emotional experience,” Ryan says. “People still want to meet a person before they list their home; it’s not just some crazy sci-fi where computers take over the world.”

AI will introduce humans into the home-buying process when it’s convenient for the buyer, Ryan says. Today’s buyers “only want to see a house if it’s a good fit for them, and increasingly narrow their choices online before going to look at an actual property.”
Mozer agrees that some decisions are best left to humans. For instance, a home monitoring the behavior of an elderly person won’t be able to tell if a series of sleepless nights is because the inhabitant is ill or simply uncomfortable.

“You can’t always use statistics of the past to make hard decisions about what actions to take right now,” Mozer says. “It’s a real problem that AI technology is not ready to solve right now.”

Stem to Steam

As artificial intelligence and high-functioning robots further evolve, whose futures will it affect? Kids, of course. To find out what they have to say about it, I chatted with a couple of robotics students at Boulder High School and Alexander Dawson School. Both schools recently competed at the Colorado FIRST (For Inspiration and Recognition of Science and Technology) robotics competition at Denver’s Magness Arena, where 51 teams from the U.S. and countries as far away as Turkey competed.

Ignacio Urbina-Gonzalez of Alexander Dawson School thinks robots and AI can help solve global issues. (photo by kerstin lieff)

Gaelin Shupe of Boulder High told me, “Personally, I am quite interested in pursuing robotics, in particular, security and AI, or cybersecurity.” Ignacio Urbina-Gonzalez of Alexander Dawson School didn’t know if he was interested in pursuing robotics at all. “So much can change going through high school, but the competition is a great thing, as it teaches me dedication and concentration.” When asked what he thought about the future, he was quite positive: “I see robots helping take care of some global issues, as programmed AI has a much better probability of success at solving problems.”

At the University of Colorado’s 2018 Conference on World Affairs, the subject of robots, AI and the future of jobs were the focus of a panel discussion titled “Dreams of a Digitized Humanity.” Eliot Peper, strategist and author of six AI novels, suggested, “It’s important not to create too much alarm, and rather try to solve real problems. Make it the dream, not the nightmare.” In fact, the panelists suggested we change our educational systems’ focus on STEM (science, technology, engineering and math) to STEAM (A is for art), because creative solutions are what are needed in this new technological era.

—Kerstin Lieff

Previous articleThis summer’s Boulder County Farmers Markets
Next articleMeet artist Kevan Krasnoff