Latest Posts

Topic: Improving the AI

Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-22, 14:42

It would be nice if we had bunch of slaves who would play widelands day and night and we would used them for AI training. But still productivity of such slave would be very low. If it takes a human 2 hours to play a single game, AI could play maybe 15-20 games within the same time. And they can play 24/7. So not realistic for now. Not even mentioning paying somebody fair salary for this training.


Top Quote
JanO
Avatar
Joined: 2015-08-02, 11:56
Posts: 177
Ranking
At home in WL-forums
Posted at: 2020-04-22, 21:01

You are right with the need of time. But I think that it might be possible that in your current training runs you might run in circles sometimes. I mean, it is somewhat likely that a set of parameters that favours atlanteans unfavours barbarians and vice versa (for example). As a result parameters are picked, that unfavour both of them.
If I have a look at the general principle of genetic adaption (in nature), in most cases it leads to specialization.

So I guess that even without investing more time in more training runs, AI with tribe specific sets of parameters would improve the performance. At least those annoying deadlocks should become less frequent, hopefully.


Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-22, 21:54

Not only tribes, but we could split maps into few categories:

-early vs. late encounter
-limited vs many buildingspots
-close vs distant mines
-seafaring & colonization (or none)

So I can imagine many specialized AIs.

What about proof of concept - if somebody try to train own branch for specific tribe and present achievements and compare?


Top Quote
JanO
Avatar
Joined: 2015-08-02, 11:56
Posts: 177
Ranking
At home in WL-forums
Posted at: 2020-04-22, 23:08

Are those characterizations true/false decisions or are they gradually? I mean, from the view of map design they are gradually, but would we need to specify parameters for each type of map or is it possible to define only the extrema and interpolate?

I think I have to set up some training environment, too


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-22, 23:10

JanO wrote:

You are right with the need of time. But I think that it might be possible that in your current training runs you might run in circles sometimes. I mean, it is somewhat likely that a set of parameters that favours atlanteans unfavours barbarians and vice versa (for example). As a result parameters are picked, that unfavour both of them.

in theory you might be right if our parameters were both very numerous and very specific. In fact we at least tried to make them generic. this generic approach in doing reasonable things if the tribe specific things are reasonably specified in ai hints has improved with the adaptations necessary to make frisian concept doable for AI and proven by handling amazons nearly out of the box ( only 1 small aspect AI can't handle for amazons). So even in parametrisation of key decisions we carefully chose parameters (if questions) that would apply for all tribes. example AI does not know about specific buildings but it categorises them by their definitions in the lua files. Farms are identified as space consumers with no input. BuildingMatProducers are identified for each tribe if their output is used in construction. After having established basic economy (which is defined to avoid deadlocks) most of the decisions are strongly influenced by economic need of the building. There are already special considerations in which phase or state an economy is e.g we need more space and therefore should expand or we need to boost our economy and need more production and so on for every state of the economy it is favorable to build special category of building, Genetic algorithm is chosen to find the right weights to determine these needs in the current state of the economy to the best compromise for all maps tribes. It is really hard to explain all the things in the code /mechanics in a forum dicussion but I try to do my very best. Anyhow apart from some special constructs used by c++ the code is not that difficult to read and at least the parts provided by Tibor and others are very well commented. Excusations for being lazy on comments (had been the case since my schooltime you could ask my teachers). Currently really the risk of having specialised AI is very low. The biggest problem from my side is there are still bugs in the code that can't be cured by genetic algorithm and we still have big parts were genetic algorithm isn't used at all. So the thing that would help most is trying to understand the AI codewise and provide questions suggestions and changes to get these problems sorted out. Even reporting obviously weird and reproducable behaviour might help as it would enable us to dig into the relevant parts of the code. (examples that got solved are Ai wasn't building near unguarded enemy land now it is. it wasn't conquering mining spots actively, it was dismantling powerful military building s to fast and some others which got fixed) currently open points are it attacks until complete exhaust. it builds to much sentries and others that I can't remember. sometimes with a specific fix another bug gets unmasked and so on. So really still there are some things that sren't to be explained by genetics so these needs to be fixed first. I'll tell you when this would be the case from my point of view. Then we need some expensive intensive training rounds to get an optimized general behaviour. after that we might go the route to either train different general characters (agrressive, economic and so on) ot tribe or map optimized versions. (where I think map influence is greater then tribe influence due to the design of the parametrisation.)
So I would ask everyone to identify weird or not understood special behaviour of the AI that we can either try to explain or to analyse and to improve. Basically I believe in the cases where genetic algorthmis used and the correct parameters are identified the Ai is doing very well with its decisions not optimal but not stupid as well. So the concept has been proven to work in my eyes.
Sorry for the long text

If I have a look at the general principle of genetic adaption (in nature), in most cases it leads to specialization.

But in our case not darwins laws are deciding about survival of a mutation but a mighty god called Developer. face-wink.png so nature can't be compared imho

So I guess that even without investing more time in more training runs, AI with tribe specific sets of parameters would improve the performance. At least those annoying deadlocks should become less frequent, hopefully.

I have not seen any severe deadlocks recently on maps where ressources were reachable in reasonable distance with normal starting conditions allowing for basic economy to be established. If you have special circumstances where this happens, please share.


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-22, 23:21

JanO wrote:

Are those characterizations true/false decisions or are they gradually? I mean, from the view of map design they are gradually, but would we need to specify parameters for each type of map or is it possible to define only the extrema and interpolate?

I think I have to set up some training environment, too

the categories that Tibor suggested are defined by the trainer who evaluates fitness for a specific purpose. So basically they are your own choice. and I believe they are as gradually as you / the trainer defines them. The AI itself does not know about such categories and don't need to do so.
Maybe one word on "training". this is basically a try and error principle of mutating existing definitions of weights and used parameters and test them. then evaluate the result. dpoing this a hundred times from the same starting point and take the four best performing sets, using them as basis for the next generation. and a new set of values entering the game are mostly 4th to 5th generation of such a run.
The art of having specific AI lies within the definition of fitness for the specific challenge. so on the other hand improving definiton of a generic fitness function might help as well. (defining what is good playstyle)


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-24, 09:42

Ok,
short update to those interested. the current attack behaviour isn't controlled by any algorithm. it is just hardcoded. As soon as the AI has contact (sees an attackable site) it will attack as long as it has soldiers to do so. regardless of all the knowledge it has about the enemy. Only thing that is controlled by algorithm is which target to attack.
the mechanism is currently. select best_target and attack it, no matter how bad this target is in reality. I tried to implement some basic evaluation for this however for a reason unknown this doesn't work yet. Furthermore we need to select attacking soldiers to prevent invalides to attack, as currently it just selects soldiers randomly and even the number of attackers is selected randomly.

Edited: 2020-04-24, 09:42

Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-24, 09:55

hessenfarmer wrote:

Ok,
As soon as the AI has contact (sees an attackable site) it will attack as long as it has soldiers to do so. regardless of all the knowledge it has about the enemy.

I dont think so, there is line

441        if (site->second.score > 0) {

Where only enemy sites with good score are considered for attacking


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-24, 10:47

Tibor wrote:

hessenfarmer wrote:

Ok,
As soon as the AI has contact (sees an attackable site) it will attack as long as it has soldiers to do so. regardless of all the knowledge it has about the enemy.

I dont think so, there is line

441        if (site->second.score > 0) {

Where only enemy sites with good score are considered for attacking

that is correct however due to line 437 this will be positvie for most of them. In the old code there was a definition of whether a player is attackable due to comparison of strengths and a counter to not attack one single site to often. And some other stuff related to the evaluation if we should attack at all. Currently I am planning to reinstate these evaluations and making them subject to genetic algorithm.
Weird thing in my trials was: in my logs of AI vs AI on crater (i just included base/log.h to get it working) the score for the attacked site (best_score) was always 4 regardless of the site attacked. Even when I tried to set a threshhold to 5 in the same manner as the if statement in line 499.

so there is plenty of unexplainable behaviour to investigate.


Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-24, 10:54

OK, this was never seriously reviewed by another person, so just carry on.

I only wanted to say that it was not an intention to "always attack"


Top Quote