Latest Posts

Topic: Improving the AI

Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2019-07-28, 18:18

I made changes to scoring used to AI training penalting too many unoccupied productionsites. After newest AI branch (about roads) I will merge it and use for next round of training... I wonder what difference it will make...


Top Quote
WorldSavior
Avatar
Joined: 2016-10-15, 04:10
Posts: 2091
OS: Linux
Version: Recent tournament version
Ranking
One Elder of Players
Location: Germany
Posted at: 2019-07-29, 21:37

Tibor wrote:

WorldSavior wrote:

Looks like it's already implemented. My idea was that each component of the AI can be trained separately.

This is not bad idea, but currently all DNA numbers are messed up together, so it would be quite painfull to separate them into corresponding areas - but possible.

However, I dont see this as a way to speed up training, because it would take the same time and only part of AI would benefit from it...

Would it? That's hard to believe to me. Currently it looks like any progress in one of the 15 areas could result in degeneration in the 14 other areas. So training all 15 areas separately might even be worth the effort. It can also be done better by several trainers at once. Currently a group of trainers can hardly be more effective than a single trainer.


Wanted to save the world, then I got widetracked

Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2019-07-29, 22:06

Yes, you are right at the end. However only about 4 or so areas involve genetic algorithm, there is still so much hand-made code in the AI...

If we had more machines to train on, we could split the training and speed it up that way.... but currently with one machine I am not sure it is worth the effort. But the concept is good.


Top Quote
WorldSavior
Avatar
Joined: 2016-10-15, 04:10
Posts: 2091
OS: Linux
Version: Recent tournament version
Ranking
One Elder of Players
Location: Germany
Posted at: 2019-07-30, 21:04

Tibor wrote:

Yes, you are right at the end. However only about 4 or so areas involve genetic algorithm, there is still so much hand-made code in the AI...

Interesting. Maybe that's one of the reasons why the training is not super effective...

If we had more machines to train on, we could split the training and speed it up that way.... but currently with one machine I am not sure it is worth the effort. But the concept is good.

Thanks


Wanted to save the world, then I got widetracked

Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2019-07-30, 22:01

AI is about 10 000 lines of C++ code, so definitely it is not only genetic algorithm...


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2019-11-29, 10:34

Hi,
I found some time recently to look into the AI code again for making the AI able to play amazons. (Biggest problem currently is the handling of rare tree woodcutters which I used a very dirty hack for)
By this I discovered a potential problem in the check_productionsites function at line 4271 of defaultai.cc where we set the unoccupied time of a site to current gametime which inmy eyes means it would never be taken into account. I tried a fix for this problem in the amazons branch of my fork.
I also introduced some penalty for unoccupied buildings in this code. (Reason for having so much of them is that the difference of target stocklevel and current stocklevel can get quite big and is dellivering a huge boost here. Perhaps we need to limit this boost in some way as well).
I have not tested this yet but this should change behaviour a lot (especially the first fix)
Amazons are not played well yet but they are working better then expected out of the box. Only thing missing is a possibility to check for already trained workers in productionsites which are not needed and shall be expelled. This is needed for expelling amazons master rangers (called jungle master) from their hut to work in a different building which accepts only master workers (rare tree plantation). I will try to implement this as I believe other tribes would benefit as well from this.
Other annoying thing I don't understand so far is that AI is only upgrading two woodcutters to the next level although productivity of both of them was 100% (over 75% overall productivity the next upgrade should be triggered). maybe the culprit for this is we break the productionsite check if one site needs to be actioned on. (line 382 defaultai.cc) I'll try to comment this part out and see how it works.

I would be glad if Tibor or anybody else with some knowledge of the AI could have a look into my changes and confirm whether I am on the right way. https://github.com/hessenfarmer/widelands/blob/amazons/src/ai/defaultai.cc

I will report my own test results anyway though


Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2019-11-29, 11:04

OK, some short comments:

if (!site.site->can_start_working() && site.unoccupied_till == 0) { site.unoccupied_till = game().get_gametime(); }

AI when considering current production performance needs to know if the site has enough time to start properly working - so that means how long it has been occupied. So the change is not good... Maybe we can create new variable unoccupied_since if it is what you have in your mind...

As for:

if (considering_upgrade && !site.site->has_workers(enhancement, game()) && get_stocklevel(en_bo, gametime, kWorker) < 1) {

First which worker it is? Especially if site can have multiple workers... You are relaxing requirements for upgrading this way... But if you thing the overall result is OK..

As for upgrading the sites, first I am surprised it upgrades lumberjacks at all, as this was not ever expected, but OK. Second thing is that upgrading is bit fragile, so probably there is inherent policy "two upgraded building should be enough" under the hood.


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2019-11-29, 12:32

Tibor wrote:

OK, some short comments:

if (!site.site->can_start_working() && site.unoccupied_till == 0) { site.unoccupied_till = game().get_gametime(); }

AI when considering current production performance needs to know if the site has enough time to start properly working - so that means how long it has been occupied. So the change is not good... Maybe we can create new variable unoccupied_since if it is what you have in your mind...

If you have a closer look in the code the unoccupied_till variable -despite its name- is used exactly in this sense of unoccupied since. (e.g. https://github.com/widelands/widelands/blob/997b1a264f4b249b7add66f8b22fb4a7c9bc622f/src/ai/defaultai.cc#L4545)
Problem ist that these checks never will be true if we set the variable to the current gametime everytime we are calling this function. and for this I tried to have it only set after a game was freshly loaded.

As for:

if (considering_upgrade && !site.site->has_workers(enhancement, game()) && get_stocklevel(en_bo, gametime, kWorker) < 1) {

First which worker it is? Especially if site can have multiple workers... You are relaxing requirements for upgrading this way... But if you thing the overall result is OK..

thanks for the comment. I need to specify the upgraded worker for the enhaced site. However this was intended to allow an upgrade if a suitable upgraded worker is available in warehouses rather to wait for the site upgrading its own worker.

As for upgrading the sites, first I am surprised it upgrades lumberjacks at all, as this was not ever expected, but OK. Second thing is that upgrading is bit fragile, so probably there is inherent policy "two upgraded building should be enough" under the hood.

Until now I have not found such policy to limit to two upgraded buildings. However upgrades depend on the definition for upgradeable buildings in lua so I expect the AI to use the lua definitions which are allowed in a proper manner and normally it isn't bad in that. (means it is really tribe agnostic)

Edited: 2019-11-29, 12:33

Top Quote
Tibor

Joined: 2009-03-23, 23:24
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2019-11-29, 15:15

hessenfarmer wrote:

If you have a closer look in the code the unoccupied_till variable -despite its name- is used exactly in this sense of unoccupied since. (e.g. https://github.com/widelands/widelands/blob/997b1a264f4b249b7add66f8b22fb4a7c9bc622f/src/ai/defaultai.cc#L4545)
Problem ist that these checks never will be true if we set the variable to the current gametime everytime we are calling this function. and for this I tried to have it only set after a game was freshly loaded.

I think the code is right. It says: If the site is doing nothing in spite being occupied for more than 6 minutes - let dismantle it...

thanks for the comment. I need to specify the upgraded worker for the enhaced site. However this was intended to allow an upgrade if a suitable upgraded worker is available in warehouses rather to wait for the site upgrading its own worker.

This is good catch, current implementation is quite a shortcut, but it would be right if you iterate over all needed positions in possible enhancment and check if needed worker is in current site or on warehouse. (Note more than one worker of a level might be needed for enhanced site)

Until now I have not found such policy to limit to two upgraded buildings. However upgrades depend on the definition for upgradeable buildings in lua so I expect the AI to use the lua definitions which are allowed in a proper manner and normally it isn't bad in that. (means it is really tribe agnostic)

Oh, you are probably right about no such strict rule, I dont remember it clearly... As for LUA stuff - if AI is upgrading a site than the problem will not be in LUA I would say


Top Quote
hessenfarmer
Avatar
Joined: 2014-12-11, 23:16
Posts: 2646
Ranking
One Elder of Players
Location: Bavaria
Posted at: 2019-11-29, 16:54

Tibor wrote:

hessenfarmer wrote:

If you have a closer look in the code the unoccupied_till variable -despite its name- is used exactly in this sense of unoccupied since. (e.g. https://github.com/widelands/widelands/blob/997b1a264f4b249b7add66f8b22fb4a7c9bc622f/src/ai/defaultai.cc#L4545)
Problem ist that these checks never will be true if we set the variable to the current gametime everytime we are calling this function. and for this I tried to have it only set after a game was freshly loaded.

I think the code is right. It says: If the site is doing nothing in spite being occupied for more than 6 minutes - let dismantle it...

Ok now I got it. Sorry for the confusion and thanks for the hint. It is exactly like you described and I will revert this change. I was thinling in a complete wrong direction here.

thanks for the comment. I need to specify the upgraded worker for the enhaced site. However this was intended to allow an upgrade if a suitable upgraded worker is available in warehouses rather to wait for the site upgrading its own worker.

This is good catch, current implementation is quite a shortcut, but it would be right if you iterate over all needed positions in possible enhancment and check if needed worker is in current site or on warehouse. (Note more than one worker of a level might be needed for enhanced site)

Ok. this sounds reasonable. Will try to do so but I might need your help probably. (at least I would appreciate your review)
EDIT: just had a look in the definition of has_workers and get_stocklevel the iteration is already implemented there so I just had to get the syntax right to make it work.

Until now I have not found such policy to limit to two upgraded buildings. However upgrades depend on the definition for upgradeable buildings in lua so I expect the AI to use the lua definitions which are allowed in a proper manner and normally it isn't bad in that. (means it is really tribe agnostic)

Oh, you are probably right about no such strict rule, I dont remember it clearly... As for LUA stuff - if AI is upgrading a site than the problem will not be in LUA I would say

No as I said it does pretty much what is described in lua. It just can't handle some aspects yet.
But I think we need to find out why only 2 buildings are upgraded although the code says do an upgrade if productivity is higher then 75% which was clearly the case. My first gues is we don't reach the site in check Productionsites as we only change one site in each loop. which might be to less when there are a lot of buildings. ( I havent found the schedule of check_productionsites yet)

Edited: 2019-11-29, 17:24

Top Quote