AI algorithm for Tacticool game bots

Type Machine learning algorithm
AREAS OF EXPERTISE Machine learning (reinforcement learning), ML-Agents, Unity 3D
TEAM 1 machine learning expert, 1 analyst, 1 project manager

The project was developed as a preparation and verification phase to confirm the hypothesis of reinforcement learning being able to make the existing behavior pattern of game bots more natural and human-like, specifically in an on-line tactical team game for mobile devices. The project was done for the company Panzerdog that develops computer games.

The main goal was to teach the specific type of game bots (cars) to orientate in the map so that the bot adequately reacted to the dynamic changes of the game such as emerging artificial obstacles on the route. In other words, if the bot sees the obstacle on the route written in the scenario and it can be gone around, it must go around it changing the previous route, but still following the aim. 

The Customer provided us with the game scene developed with C# for framework Unity 3D, a map for resting, game bot and the example of the ML-Agents library usage. Also we had the files of configurations, scripts, program code of server part of ML-Agents with the specific version of Unity 3D and pack of ML-Agents that algorithm must use.

  1. The Customer’s code had a lot of interconnections with the neuronet and the system so it was quite hard to implement our code. Also Customer’s code supported only the old version of the library ML-Agents.
  2. Transporting the information about the nearest objects to the car. Is used to eliminate the determination in the Ray Perception Sensor approach.
  3. The bot needs to orientate in the field of the game map. The solution was supposed to be not overloading Customer’s server powers.
  1. The original code was analyzed and rewritten considering further development of the product and support of the new version of the library ML-Agents.
  2. In order not to use “Ray Perception Sensor” to identify the nearest objects to the bot, we used an alternative approach based on the system of tags. The model was given the data about N nearest objects. The data was the distance and the angle to the object.The approach failed as the bot didn’t learn from it. One of the reasons could be the small size of the model and the shortage of the learning period compared to the whole task time.For the local orientation we decided to have the Ray Perception Sensor approach, but to make the detection better we widened the radius of it. Unfortunately, it affects the speed of model learning, so it is not worth making the radius too wide.
  3. The attempt to teach the model without interim checkpoints showed that the car couldn’t reach the destination point. We decided to use the structure of data NavMesh to overcome interim checkpoints and more effective movement around the map.However, the approach with NavMesh was declined by the end of the project in order to use the alternative approach of manual 2D map drawing based on 3D map, as the Customer noticed the problem of servers heavy load while using NavMesh. For global orientation of bots on the map Algorithm A* was chosen as the most classical one.



As a result the behavior of the bot on the demo game map became more logical and closer to the human-like. The bot following the route could change it according to the detection of some objects (that were out of the field of vision before) on the way and successfully go around moving objects (imitating players movements) still reaching the destination point.

At the moment the model is being tested by the Customer team for the ability to use it on the other maps.

Business value

The main business task of the project was the prove of possibility to improve the existing game process using Machine Learning.

We plan to discuss new tasks for Machine Learning connected with the improvement of the game process and balance control in the game.

A mobile app for tracking vehicle fleet of the coal mining companies
A mobile app for tracking vehicle fleet of the coal mining companies
AI-enabled mobile app to recognize fingerprint patterns
AI-enabled mobile app to recognize fingerprint patterns