First AI that sees like a human could lead to automated search and rescue robots

Computer scientists have taught an artificial intelligence agent how to take in its whole environment by just taking a few snapshots.

The new technology can gather visual information that can be used for a wide range of tasks including search-and-rescue.

Researchers have taught the computer system how to take quick glimpses around a room it has never seen before to create a ‘full scene’.

The scientists used deep learning, a type of machine learning inspired by the brain’s neural networks, to train their agent on thousands of 360-degree images of different environments. 

They say that their research could aid effective search-and-rescue missions by making robots that could relay information to authorities.

Most computer systems are trained for very specific tasks – such as to recognise an object or estimate its volume – in an environment they have experienced before.

The tech, developed by a team of computer scientists from the University of Texas,  gathers visual information that can then be used for a wide range of tasks.

The main aim being that it could quickly locate people, flames and hazardous materials and relay that information to firefighters, the researchers said. 

After each glimpse, it chooses the next shot that it predicts will add the most new information about the whole scene.

They use the example of a human being in a shopping centre they had never visited before, and they saw apples, you would expect to find oranges nearby, but to locate the milk, you might glance the other way. 

Based on these glances, the agent infers what it would have seen if it had looked in all the other directions, reconstructing a full 360-degree image of its surroundings. 

When presented with a scene it has never seen before, the agent uses its experience to choose a few glimpses. 

Professor Kristen Grauman, who led the study, said : ‘Just as you bring in prior information about the regularities that exist in previously experienced environments – like all the grocery stores you have ever been to – this agent searches in a nonexhaustive way.’

‘We want an agent that’s generally equipped to enter environments and be ready for new perception tasks as they arise.

‘It behaves in a way that’s versatile and able to succeed at different tasks because it has learned useful patterns about the visual world.’ 

‘What makes this system so effective is that it’s not just taking pictures in random directions but, after each glimpse, choosing the next shot that it predicts will add the most new information about the whole scene, Professor Grauman said. 

The research was supported, in part, by the U.S. Defense Advanced Research Projects Agency and the US Air Force Office of Scientific Research.


Leave a Reply

Your email address will not be published. Required fields are marked *