Latest Posts

Topic: Improving the AI

hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-02-16, 20:05

the-x wrote:

+1

exactly that is it, maybe we need sth like a "hard ai"

well if you know how to make one. --> feel free to tell us how this could be implemented in the code.
Seriously we are working hard to bring the Ai into a state where the weak Ai choice really makes sense. Althouigh we are quite far from it I admit.


Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3324
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2020-02-21, 18:22

king_of_nowhere wrote:

hessenfarmer wrote:

there is already a definition problem: what is a large map? this can't be determined from size. cause if you have only some small islands it is different again and it doesn't need to be water there are other unbuildable terrains as well. So really there can't be any general rule for anything like this.

that's why i suggested that any map has a tag for it. humans themselves can't figure out the big/small map distinction if they don't know the map in advance. and the generic, unmodified algorithm running if there is no tag on the map.

For determining map size, we could use size_t Map::count_all_conquerable_fields() or something similar. This won't solve the computing power problem though.


Busy indexing nil values

Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-21, 17:48

Ok I took some time to dig into the code of the AI warfare. I believe I found some Bugs and some room for Improvement to prevent the Ai from spoiling its soldiers. (basically wrong comparison operators in my view and a Copy past error leading to much to higher scores to attack.
for those interested I beleive the lines around 276 and 280 should have either differnt sign in the score result (- instead of +) or the opposite comparison in the if part.
line 257 seems to be a copy paste error.

Will prepare a branch and test tonight. However these things should be changed best in b21


Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-21, 19:49

these lines are part of genetic algorithm stuff and might or might not be considered based on result of previous training. And any change in this stuff should be followed by training to let AI figure out if these positions in the score are to be considered. But go on if you feel so..

Row 257 is indeed a copy error.. But even this fix should be followed by re-training....


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-21, 21:01

I know they are part of gentics and are not considered if the Neuron is zero at this position. However if considered they encouraged attack when AI is weaker then then the opponent, which I deem wrong.

Bad thiong is even after the first corrections the AI is still attacking until no soldier is left to attack. This will need deeper analysis As i doubt this could be cured purely by genetics.

I agree that any changes would need training. However I am really eager to hunt this behaviour down to an acceptable level which one might call character or aggressiveness and not (extended) suicide.


Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2020-04-22, 07:30

I am willing to run some training after the change will be in the trunk..


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-22, 10:05

Hi, I just pushed a branch and drafted a pr to ease discussion and review. However the visible results are marginal at the best. Problem for me is that the logs in the evaluation of input scores do not work on my computer (or I miss some part to get them) so I am unsure about the values given and if some are just too high so they are masking every other decision.
If someone on linux (I suppose they work there) could provide me a log with these values from let's say an barbarian AI(blue) vs atlantean AI(red) match on crater (together with a replay) this could help a lot.

Edit: forgot to link the PR https://github.com/widelands/widelands/pull/3873/files

Edited: 2020-04-22, 10:05

Top Quote
blind3rdeye
Avatar
Joined: 2020-03-26, 08:47
Posts: 74
Ranking
Likes to be here
Posted at: 2020-04-22, 11:53

It seems to me that it may be problematic if parts of the AI are 'learning' by playing against other AI, but other parts are not. Effectively the AI has some constraints on it, and it is being trained to do its best against another AI with the same constraints. So it may be learning very effectively to play well against other AI opponents, but could fail hard against someone who plays differently.

I keep thinking about why the AI keeps building so many sentries on the front-line. It builds heaps of them, and if I ever try to push my territory forward with a sentry, the AI attacks hard with a lot of soldiers. So then, I imagine if the AI is playing against a different AI to plays similarly, the one who builds the most sentries probably has an advantage. But when I play vs the AI, I instead just build a couple of barriers, and kill any sentries that pop up on the boarder. That means the AI can't really attack me at all because I'm out of the range. I can hold them off effectively even when their army is much stronger, because their mass of sentries are all out of range.

What I'm saying is that AI learning against other AI might get stuck in a local optimum which is actually pretty poor; and it also might be incapable of escaping that local optimum due to constraints on what inputs and outputs they are given.


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2020-04-22, 12:47

Hi,
You are right about the lots of sentries that are build. Another thing to analyse on my list. However I believe there might be a small misconception about our AI training. Basically the AI is not "learning" it is trained. Which means a human decides, based on evaluation or fitness parameters which mutationof the AI did best in a given scenario. this is not dependent on opponent or how an opponent does.
What happens is that in training mode the Ai mutates stronger then normal and changes the weights of some decisions and if the decisions are taken into account at all. What I am trying is to check if all relevant parameters to make a decision are defined and if they are defined correctly. Afterwards in the training runs which are a trial error method. the best mutation using this parameters with the best weights is chosen for the next runs. If you want to get a glue what is going on go to the AI code and search for the term "inputs[" then you get to a part where the training kicks in. have a look how the various inputs are summed (the parameter is taken into account if his bit position in the representation of an integer number is 1). other parts can be found searching for "military_number"


Top Quote
JanO
Avatar
Joined: 2015-08-02, 11:56
Posts: 177
Ranking
At home in WL-forums
Posted at: 2020-04-22, 13:49

I sometimes wonder: Would it be possible (of course, yes, but would it be possible with reasonable effort) to make the AI learning? I mean, let them change within a running game. Surely that would need some hard brainwork to find a way how to evaluate meaningful versus stupid decisions. Probably too complicated.

Nevertheless, I would suggest to split up AI training for different tribes.

Edited: 2020-04-22, 13:49

Top Quote