Artificial intelligence (AI) researchers at Facebook have spent the last year working on a project that saw an AI system learn how to navigate parts of the sprawling conurbation that is New York City. The study is designed to help computers understand natural human language, paving the way for AI assistants to communicate with people more competently.

“It isn’t enough for virtual assistants to offer a rote response to your voice or text,” wrote Douwe Kiela and Jason Weston, FAIR research scientists based in New York in a blog post on Thursday. “For AI systems to become truly useful in our daily lives, they’ll need to achieve what’s currently impossible: full comprehension of human language.”

Teaching machines to understand us is far from easy, and feeding them raw text data isn’t necessarily the best way forward in the eyes of the Facebook Artificial Intelligence Research (FAIR) group.

Instead, the Facebook researchers used an approach called “embodied AI”, also known as “grounded language learning”, which favours learning in the context of a system’s surroundings, rather than training through large data sets of text, such as Wikipedia.

FAIR’s “Talk the Walk” project sees a “tourist” bot navigate their way through 360-degree images of five actual New York City neighbourhoods. This is done with the help of a “guide” bot, which sees nothing but a 2D map of the neighbourhood. The tourist bot essentially describes what it “sees” and the guide bot responds with directions. Facebook found that its bot guide was better than human guides at giving directions.

Facebook isn’t the only tech giant working on natural language understanding. Apple, Google, Microsoft and Amazon are all pursuing similar projects in a bid to give their own AI assistants an edge.

“Good language understand has applications in almost everything that we do,” said Kiela in an email. “In practice, you can think of AI assistants that can communicate better with us, because they understand the world better, in a way that’s similar to how people understand it. Think of how people learn about the world as children: they can take actions in the world, interact with objects and other humans, and get exposed to a lot of language from their parents that is directly about the world.”