DeepMind AI agents surpass humans in Quake III Arena CTF

[Ozaron]

I can't see you ever getting a Bayesian spam filter to play Quake.

...I'll be waiting over here, by the tea dispenser.

[devBunny]

Fundamentally, am I wrong? I have no understanding of AI and have done no work in the field but this seems pretty simple.

Yes, you've grasped it and summarised it well. Proof, indeed, that you are no AI! ;o)

In Backgammon, for instance, there are about 250 inputs and about 250 nodes in the neural net. It's not deep, just a single hidden layer, so that's about 62500 weights there and another 250 for each of the outputs (of which there are six). Definitely a complex equation but, as you say, it's just maths.

Part of the beauty of neural nets is how compact they are. The Backgammon equation, with its 64K variables, effectively encodes a database that could contain gazillions of values (2x10^19). Moreover, it builds the statistics from played games rather than trying to cover possible but unlikely states (eg. reached by sequences of strange or random moves), so it only needs a few million games rather than multiple iterations over gazillions of positions.

[philehidiot]

Yes, you've grasped it and summarised it well. Proof, indeed, that you are no AI! ;o)

Not entirely sure how I managed that whilst rather smashed. Proof, indeed, that I'm smarter when drunk.

My head hurts.

[Tabbykatze]

Without knowing more about the process they used there are actually two different types of "AI" that is generally used.

The first is Machine Learning which is colloquially termed as a "Dumb Intelligence" which is fantastic for highly linear tasks like optimising a process to within an inch of its life or an item to within its glass ceiling of optimisation. What machine learning does is it is given a desired output, say a Girder that is meant to be able to resist/prevent vibrations, and then a spread of inputs and values that are provided weights, length and number of struts and their connections etc. What Machine Learning does is it iterates through variations thousands, millions if not billions of attempts to get an output within the desired "value". This is an incredibly focused form of "intelligence" and can only operate within a very tight linear framework. Machine Learning cannot be "dynamic" by design, if the output requirement changes then it will have to almost re-perform all the iterations to get to the desirable again.

Now the next form of "AI" is Deep Learning which develops and generates its own "neural nodes" which have their own dynamically created/set inputs and weights which can then provide an output to another set of neural nodes that then can provide an output to another set of neural nodes and so on. This is the "Smart"-ish "Artificial Intelligence" that is used for things like driving, voice recognition advancements and many others. This is the current closest approximation of "Artificial Intelligence". Now what differs between Deep Learning and Machine Learning is that Machine Learning is designed for a single or tight set of outputs whereas Deep Learning can have the capability of inputs being generated based on new information and nodal sets created to deal with that. It still sounds like Machine Learning so bear with me!

In the case of Quake 3 Arena CTF, the primary final objective is "win the game" but there are numerous other objectives which range from "don't die", "keep health high", "grab flag", "carry flag", "return flag", "prevent flag theft" and so on and so on. I suspect that this is a pretty advanced form of a Deep Learning Neural Network (DLNN) that was used because of this. What the DLNN can do is generate a new node (or nodal set) for "enemy player grabbed flag with a rocket launcher and I need to retrieve flag while keeping myself alive and killing them while dodging rockets and preventing their flag capture" and develop a node within the greater superset that is can allow the firing of the "neurons" as it is working to deal with the new inputs to still get to the desired output (win the game). This is an extremely high level description of what is happening, it is picking up input information of what it is encountering and is developing new nodes to develop a reaction based upon the information to get to a desired output and this is done on the fly by the system.

So this is AI in a very dumb sense of the word from our perspective but is phenomenally advanced from technology even 10 years ago. The other thing that they don't talk about is how many "Neural Nodes" this AI is utilising and judging by Agent State t-SNE embedding they are dealing with millions if not billions of neurons. And actually does lend a bit to my description that some parts of the web are for specific things:


I might actually download the paper and have a thorough read.

But yes, what I'm getting at is that this is "Artificial Intelligence" which is still pretty dumb and linear from our perspective but it could be supposed that its intelligence is getting more and more dynamic and is probably starting to surpass the small mammal stages.

And also, be careful of the term Machine Learning, it is not always the best descriptor of the actual "Intelligence" used and is very different from a Deep Learning AI as it's "fixed output" vs "desired results". Unfortunately, Machine Learning is the term used within a Deep Learning set so it can get annoyingly confusing.

I hope that makes sense a little bit

[Ttaskmaster]

If the AI was a camping sniper than it would easily thrash anyone!!

In a fair fight, perhaps...
But against artillery? Nope!!

We had a saying - "Ya can't outrun the 81"... and searching fire with an 81mm mortar usually sorts out snipers quite tidily!

[johnroe]

I think the way the public views AI is through films. I do think that 'artificial intelligence' can be perceived wrongly if it's being compared to human or animal intelligence. But it is 'artificial' and it is a form of intelligence.

The fact is with growing populations and our aspirations to explore the planets, we as a race are going to have to solve many complex problems. So computers and AI are nothing more than tools at our disposal; military use for logistics and strategy planning, cctv recognition on an international scale(being tested in China), banks use AI to monitor unusual traffic through accounts, and hospitals use AI to speed up scanning of images of brain for cancer. I imagine that the military would be interested in the game bots strategies, and learning.

Here's a story from the Guardian which shows AI's current acceptability. 'companies fake it until they make it, telling investors and users they have developed a scalable AI technology while secretly relying on human intelligence'. from> https://www.theguardian.com/technolo...tech-companies