Topic: widelands and CPU usage

king_of_nowhere
Avatar
Topic Opener
Joined: 2014-09-15, 18:35 UTC+2.0
Posts: 1668
Ranking
One Elder of Players
Posted at: 2015-10-04, 20:20 UTC+2.0

Venatrix wrote:

If you want to check on the CPU usage all the time, why don’t you play windowed? Just to test it.

Good idea. i tried it, and during the loading widelands was using 30-35% of my CPU and virtually nothing else. It also said the program wasn't answering during the loading - in fact, it is impossible to exit from load screen with alt-tab, but if you are already out of it due to not being on fullscreen, then you can use the pc just fine. CPU consumption also wasn't limited by the capacity of my pc. widelands was the only program that was using CPU at the time, so if the slowness was just a matter of making calculations, it could have used 100% of the CPU and loaded the game in 40 seconds. Then hard disk usage spike to close to 100% for a few seconds before it loaded the game - I suppose the game was loading the various data about maps and buildings from memory. But it didn't do it in the previous two minutes. It's like it did nothing for two minutes, then it suddenly loaded everything in a few seconds.

MAybe there is some code that accidentally causes the game to run a lenghty series of operations the longer a game has lasted? Is there a way to modify a savegame file to cheat the computer into thinking the game has been shorter than it actually was? It would include deleting some statistics, but in theory it should not be complex, if one knows where to look.

Then I could confirm that the game takes more CPU when reduced to icon. It run at 10-15% when it was opened as a window, but as soon as I reduced it, it spiked to 35%. And dropped again when I reopened it. I tried a few times just to make sure it wasn't random occurrence. No idea what may be causing that.

I could not check what happened when I exited the game: cliicking on "exit game" caused the game to go in fullscreen again. And I could not alt-tab. But I suppose it would happen something similar to what happens in the loading.

kaputtnik wrote:

You could start widelands from a terminal/console/prompt. In there will be the output of which part loads in which time. So you could compare the output with a short time game.

Alas, that's way above my mediocre skill as computer (l)user. Gathering the data I did so far is pretty much the extent of my capabilities.

Edited: 2015-10-04, 20:20 UTC+2.0

Top Quote
Tibor

Joined: 2009-03-23, 23:24 UTC+1.0
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2015-10-04, 20:32 UTC+2.0

You should give it a try. The output looks like:

Game: Reading Preload Data ... took 2ms
Game: Reading Game Class Data ... took 0ms
Game: Reading Map Data ... Game: Reading Map Data took 1ms
Game: Reading Player Info ... Game: Reading Player Info took 130ms
Game: Calling read_complete()
Reading Elemental Data ... took 2ms
Reading Player Names And Tribe Data ... took 48ms
Reading Port Spaces Data ... took 4ms
Reading Heights Data ... took 13ms
Reading Terrain Data ... took 93ms
Reading Map Objects ... took 1429ms
Reading Player Start Position Data ... took 0ms
Reading Resources Data ... took 61ms
Reading Map Extra Data ... took 1ms
Reading Map Version Data ... took 0ms
Reading Allowed Worker Types Data ... took 1ms
Reading Allowed Building Types Data ... took 1ms
Reading Node Ownership Data ... took 12ms
Reading Exploration Data ... took 75ms
Reading Flag Data ... took 173ms
Reading Road Data ... took 7ms
Reading Building Data ... took 96ms
Reading Flagdata Data ... took 4ms
Reading Roaddata Data ... took 178ms
Reading Buildingdata Data ... took 20ms
Second and third phase loading Map Objects ... took 92ms
Reading Players View Data ... Vision check successful for player 1
Vision check successful for player 2
Vision check successful for player 3
Vision check successful for player 4
Vision check successful for player 5
Vision check successful for player 6
Vision check successful for player 7
Vision check successful for player 8
took 279ms
Reading Player Message Data ... took 24ms
Reading Objective Data ... took 1ms
Reading Scripting Data ... took 4ms
WidelandsMapLoader::load_map_complete() took 3763ms
Game: read_complete took: 3763ms

Note all these miliseconds


Top Quote
king_of_nowhere
Avatar
Topic Opener
Joined: 2014-09-15, 18:35 UTC+2.0
Posts: 1668
Ranking
One Elder of Players
Posted at: 2015-10-04, 21:07 UTC+2.0

as i said, i am unable to produce that output. I have no flippin idea of what to do to get that. but i could send the saved game to someone who can try it.


Top Quote
Tibor

Joined: 2009-03-23, 23:24 UTC+1.0
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2015-10-04, 22:29 UTC+2.0

No big deal, open a terminal or whatever it is called on your os, here is a screenshot:



and put there either full path or relative path to widelands binary. If you are using windows, I think you must add something special after widelands to produce output to terminal, I am not sure...

It is not that complicated as it seems


Top Quote
wl-zocker

Joined: 2011-12-30, 17:37 UTC+1.0
Posts: 492
Ranking
Tribe Member
Location: Germany
Posted at: 2015-10-05, 09:20 UTC+2.0

See this page, point number 4. It tells you how to create a stdout.txt, where you find the loading times.

EDIT: It is now point 6, Reporting a crash.

Edited: 2015-10-05, 20:00 UTC+2.0

"Only few people know how much one has to know in order to know how little one knows." - Werner Heisenberg

Top Quote
king_of_nowhere
Avatar
Topic Opener
Joined: 2014-09-15, 18:35 UTC+2.0
Posts: 1668
Ranking
One Elder of Players
Posted at: 2015-10-05, 19:51 UTC+2.0

no, that's stilll way above what i can do with a pc.

I can open a bug report, if you want


Top Quote
Tibor

Joined: 2009-03-23, 23:24 UTC+1.0
Posts: 1377
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2015-10-05, 21:57 UTC+2.0

It would do no harm if you open a bugreport. But it would look much nicer if you can mention there that 'the step XYZ takes suspiciously long to read/load'...


Top Quote
SirVer

Joined: 2009-02-19, 15:18 UTC+1.0
Posts: 1442
Ranking
One Elder of Players
Location: Germany - Munich
Posted at: 2015-10-06, 09:27 UTC+2.0

I can throw in a theory here, but having a long save game attached to a bug report could make it easier to verify.

Widelands uses pointers to link data structures together. A pointer is a memory address where another instance of an object is defined in memory. For example, buildings keep track of their inhabitants by pointers to the workers, Ships information about their wares and routes and so on. Problem is that memory locations change with each load. So Widelands splits loading into three steps: load what objects are available, load their individual data that is not pointers (for example the number of wares currently in a building, current HP of a soldier), and last: recreate the pointer links for each object.

Each object has links to a couple of other objects, so the number of pointers raises quite dramatically and recreating them will take a non-linear amount of more time. I think that is were we are spending our time mostly during loading. I might be wrong of course.

A better approach would have been to use an arena allocator for the game objects and keep them all around the same size to not waste too much memory. With that, loading would just recreate the arena and dump the memory from the file into the arena (minus some backwards compaibility cruft). Unfortunately, changing the design of Widelands to something like that is really out of the picture - way too much work.

Loading could still be sped up by recreating the links in parallel (on many CPUs), but so far Widelands is single threaded, so that would be a bit of work too.


Top Quote
king_of_nowhere
Avatar
Topic Opener
Joined: 2014-09-15, 18:35 UTC+2.0
Posts: 1668
Ranking
One Elder of Players
Posted at: 2015-10-08, 04:32 UTC+2.0

ok, bug report posted


Top Quote
kaputtnik
Avatar
Joined: 2013-02-18, 20:48 UTC+1.0
Posts: 2702
OS: Archlinux
Version: current master
Ranking
One Elder of Players
Location: Germany
Posted at: 2015-10-08, 08:34 UTC+2.0

king_of_nowhere wrote:

ok, bug report posted

bug report

Edited: 2015-10-08, 08:34 UTC+2.0

Top Quote