Virtual butler ‘Jarvis’ takes up residence in Facebook founder’s home

Mark Zuckerberg on Monday introduced the world to “Jarvis”, an artificial intelligence system the Facebook chief created in his spare time, which can choose and play music, turn on lights, and recognise visitors, deciding whether to open the front door.

Jarvis, named after the virtual assistant in the Iron Man movies, could be a step toward a new product, Zuckerberg wrote, although he cautioned that the system he had created in 100 hours over the last year was customised for his house.

Zuckerberg announced results of the project, a personal challenge he set for himself this year, as digital home assistants by Google Inc and Amazon.com Inc compete for holiday sales and are expected to outsell popular emerging gadgets such as virtual reality headsets and drones.

Creating Jarvis proved humanity is “both closer and farther off” from an AI breakthrough than we imagine, Zuckerberg wrote.

Computers are getting very good at pick out patterns, such as face recognition, but it is difficult to teach them new things, he wrote.

“Everything I did this year — natural language, face recognition, speech recognition and so on — are all variants of the same fundamental pattern recognition techniques,” he wrote. “But even if I spent 1,000 more hours, I probably wouldn’t be able to build a system that could learn completely new skills on its own.”

By the end of the year, Jarvis was able to respond to text and voice commands and it could run music, air conditioning, doors, and other systems. It could recognise visitors, start a toaster and even shoot t-shirts from a cannon in his closet.

With more effort to broaden Jarvis’ use beyond Zuckerberg’s own house, the experiment “could be a great foundation to build a new product,” he wrote.

A dearth of internet-connected devices, lack of common standards for connected devices to communicate and challenges related to speech recognition and machine learning were all obstacles, he said.

At the same time, he said challenges lead to eureka moments.

Adjustments made to help Jarvis recognise the context in commands ultimately helped the system respond to less specific requests in a better fashion, such as asking the system to “play me some music”.

“I’ve found we use these more open-ended requests more frequently than more specific asks. No commercial products I know of do this today, and this seems like a big opportunity,” he wrote.