Software Pirates, Bullion Style!

Paul discusses the updates and improvements to the AI since the last blog post...

It's been just over a year since the last posting about the AI. Since then we've learnt a lot more about how Unity works; we've had several planning sessions about how we want the AI characters to react to in-game events; and we've substantially rewritten the player framework.

During the events we demonstrated at during Autumn 2017 (Play Manchester and Reading Comic-Con) it became clear that we needed to have the player AI ready in time for GEEK 2018. This was partly so that single players could enjoy Bullion, but also so that the game could demonstrate itself to people walking round the event rather than being just a title screen, or needing us to play it. People are attracted to activity and want to see what the game is like before sitting down to play: crowds would gather to watch other players but not to watch the title screen. Having the game play itself would attract people whlist leaving us free to talk to expo guests, press, and anyone else interested in the stand.

Show me the way to go home...

An early casualty of our learning experience with Unity was the AI navigation subsystem. We were in any case having some difficulties with the grid-cell-based algorithmic approach outlined in the previous blog post - it looked slightly unnatural. Shortly after writing the post, we discovered Unity's NavMesh and NavMeshAgents which made our code somewhat irrelevant! So part of the player character refactoring was needed to introduce NavMesh behaviour into the AI player characters and the enemy characters. The net result was smoother behaviour from the AI players and smarter rerouting when obstructions such as chests popped up into the AI player's path.
This highlights one of the benefits from our agile development approach - we learn as we go, prototyping new behaviour and swapping in new components as needed or as discovered. (For example, we're making a similar change now to use the ReWired controller manager rather than Unity's own controller mechanisms).

Different engines for different behaviours

Another change we made was to separate the enemy and player AIs. We decided that the range of behaviour for enemies was significantly different to that of the players, so splitting them out would avoid compromising one for the other. Perhaps we will re-integrate them in the future, particularly when we start considering the behaviour of the Island Gods characters in the boss battles.

How an AI pirate thinks...

Enemy AI is quite simple - choose a player to target, chase them down, and kill them, then re-target. The choice of target can be based on proximity, on which player has the most kills, or on which player has the most treasure; each enemy type will use a different blend of these parameters. With the newer enemies on Salty Swamp we're also introducing a "run away" dynamic (if they are ganged up on by the players) - which required the development of a proximity detector that will prove useful for other player AI behaviours. The current enemies are simpler than human players - they don't become ghosts, they can't swim, they don't have Bull Rush, they don't collect treasure or use power-ups. Rebuilding the enemy AI first, using NavMeshAgents for navigation, proved an important precursor to tackling the player character AI.

Player AI developments

Fight, flight or something else?

No code was taken from the GEEK 2017 player AI into the 2017-18 development phase, but the concepts have all carried over. We started with a cleaner codebase rid of the bugs on the infamous "popup of post-its", and freshly built on top of the new player framework. The ghost behaviour is split from the live character behaviour; ghosts have no need for AI behaviour since all they do is return to their respawn point.

So the AI is part reactive and part plan-oriented. Firstly if the player finds itself in the water it will abandon any existing plan and instead get back to dry land (which will need fine-tuning when we introduce environments with multiple islands where the player will need to risk swimming from one to the other). Then, if it has no current plan, it will create one; and it will then follow that plan until completion, until the plan is no longer viable, or until it gets interrupted.

A typical plan involves choosing an appropriate target, moving into proximity of that target, and then (depending on what type of target it is) either collecting it or attacking it, and then finally collecting any treasure released by that target when it was destroyed.

What does an AI want the most?

Targets are managed by the game framework. Anything we want to be targettable - treasure, chests, enemies, other players, power-ups - contains a component derived from a parent 'targettable' interface. We can then find and manage pools of targets of various types. It's then up to the AI to look at these pools of targets, how far away each one is, and assess the risk/reward balance for approaching each. Simplified, the algorithm for ranking targets against eachother is (reward - risk)/(distance + difficulty)

• Reward: The simplest reward is tangible: how much treasure will I gain by choosing this target? This is known for loose treasure; we can guesstimate for players (the richer they are, the more they drop if they die); and we know on average how much we can get from chests and enemies. But there are other rewards. For me personally: Is that power-up valuable given my current situation? And for the team: Is the enemy count building up to an extent that presents a danger of stalemate if all the players die?
• Risk: Should I attack a player or enemy if they have significantly more health, speed or strength than me? Are there other players/enemies guarding my intended target?
• Distance: the longer it takes me to get to a target, the less appealing it should be. Firstly, while I'm walking to a target I'm not collecting treasure; secondly, the target may have moved or been taken by another player by the time I get there.
• Difficulty: picking things off the ground is effortless. Smashing chests takes time, and attacking enemies or other players takes longer still depending on their health and defensive capabilities.

Alpha/Beta testing

We actually developed 2 plan choice algorithms. The first algorithm was fairly simple - it chose the highest scoring plan every time, and it had a strong bias towards targetting richer, weaker players. In initial team testing this felt too difficult to beat (3 AI players all relentlessly tracking and attacking a richer human player), especially given the players at the expo would be new to the game. So we developed a second flavour that would more randomly choose between a range of behaviours, and would try to avoid joining an existing fight. The team played both but eventually decided that the second cut of the algorithm was too 'cowardly', so we took the original blend to the expo. If we'd had more space on the stand we might have run both and taken feedback from the users, but as it was we limited our A/B test to just the team. We're now revisiting the second algorithm and adjusting the balance, as it better lends itself to the player characteristics for the different crew members.

Power-ups

Sometimes, chests contain power-ups - extra speed, more health etc - which affect the gameplay. The AI now spots these on the map and collects them as it would loose treasure. It then has to decide when is the best time to use the power-up without it being wasted. No point triggering the extra strength power-up if we are a long way from our attack target; the extra speed power-up would be more useful in that instance. The extra health power-up shouldn't be used when we're already pretty healthy. Should I save the Voodoo Curse for when I'm under sustained attack from multiple enemies? Do I use the resurrection Water of Life chalice when I'm critically low on health, or should I allow myself to die (so the enemies choose a new target), escape as a ghost, then resurrect myself - or just walk back to the restart point if it is nearby?

No plan survives contact with the enemy

The AI is constantly monitoring its plan to make sure that it still makes sense. If another player takes the treasure/chest we were going for, or if the player/enemy we were chasing is killed by another player, we need to re-plan. Changes in the environment can also affect our plans. If we're going after a chest and a new batch spawns close to us, maybe we should switch to them instead. Similarly if we are culling enemies, a closer target might present itself. Opportunities might open up as we travel across the map - if we are passing close to some loose treasure or a power-up, we should collect it. Or we might be attacked, in which case we should switch plans to either run away or stand and defend ourselves. The best defence of course being offence! So the AI has hooks into events fired by chest/enemy spawners to be told a new potential target (or threat) is on the scene, and oversized colliders envelope the AI player to notify it when potential threats and opportunities have come within range.

Reacting to environment changes is an an area in which we want to make the AI more human-like. Humans take time to notice and to react to changes, which the AI could detect immediately. We want our AI to have more human-natural reaction times so that it plays more like a person would play. To do this, we are using the Unity Coroutines feature - which defers the handling of those incoming events by an appropriate lag period.

Developing a personality

Currently all the AI players perform the same way. They judge threats and rewards using the same algorithm with the same parameters. The differences in actual in-game behaviour arise because of environmental factors - where the treasure and threats are relative to each AI - and due to a small amount of randomisation when choosing which of the high-scoring plans are going to be executed. This works for us since currently all of the AIs and human players are controlling the same character - Captain Long John Silverside - so there's no distinction necessary.

However, work is ongoing in the current development phase to add more of our pirate cast and crew to the game. So we are preparing the AI so that it will take the personalities of the characters into account. Each pirate will have its own metrics - moving speed, attack strength, etc - which should affect the decision-making process: can I outrun, or catch, my target given our relative moving speeds? Who is more likely to do more damage to whom if we fight? Does my character have a grudge against the other character? So we are borrowing 'player stats' concepts from role-playing games, and the AI will 'roll against' those stats when making choices about how it will react to changing events, and when making new plans.

Each member of the crew will have its own special move for attack/defense. They could also all use the signature Bull Rush move to charge and take out a row of targets in front of them. Currently the AI doesn't use either of these game features. We will be adding logic to detect when each of these would be most useful - e.g. do I have multiple targets in the attack zone for my special move or bull rush? This is likely to make more use of Unity colliders. There is also the Block capability for defense. The AI should consider whether Block will help - triggering it before it is attacked so that it is effective, and knowing when to release it to counterattack or to flee.

So there's a lot of work going on in the AI over the next couple of phases of Bullion development - not to mention the boss-battle AI for the Island Gods characters! We hope that all our efforts result in a more enjoyable game for you to play!