Polls

Default Game Speed

Log in to vote!

Latest Posts

Topic: General call for testing

GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3058
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2015-12-06, 20:39

We are currently very short on people who can review C++ code. So, I thought we might take the pressure off by splitting up the work a bit - whenever we review a branch, we also need to compile it and do manual testing, run the regression test suite etc.

People who don't know how to code C++ but do know how to compile could help us with that - if everything was tested, the person looking at the actual code would save the time for compiling etc.

I have added a guide on how to test to the wiki: https://wl.widelands.org/wiki/TestingBranches/

Maybe the best way to organize this would be for a tester to add a comment "will test" to the merge proposal. Then, when the testing is finished, report any bugs in the merge proposal. In case of no bugs, write something like "tested, all clear" and also mention whether you have run the automated test suite.


Busy indexing nil values

Top Quote
kaputtnik
Avatar
Joined: 2013-02-18, 20:48
Posts: 1771
Ranking
One Elder of Players
Location: Germany
Posted at: 2015-12-06, 21:03

Normally it is enough to do

bzr branch lp:~widelands-dev/widelands/branchname

A folder branchname will automatically be created and contains the code. But i am not sure if a launchpad ssh-key is needed to do so.


Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3058
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2015-12-06, 21:06

I have updated the instructions, thanks face-smile.png

Downloading should generally be possible without an ssh key. Leaving out the branchname is syntactic sugar and should theoretically not matter. You will need an ssh key if you want to push to a branch.

Edited: 2015-12-06, 21:08

Busy indexing nil values

Top Quote
Tibor
Joined: 2009-03-23, 23:24
Posts: 1267
Ranking
One Elder of Players
Location: Slovakia
Posted at: 2015-12-06, 21:59

Especially persons who reported a bug would be good testers, because he/she knows what to look at...


Top Quote
einstein13
Avatar
Joined: 2013-07-29, 00:01
Posts: 1095
Ranking
One Elder of Players
Location: Poland
Posted at: 2015-12-07, 01:36

What about bugs that are hard to reproduce? (like soldiers fighting forever or soldiers going out from castle to 0?)


einstein13
calculations & maps packages: http://wuatek.no-ip.org/~rak/widelands/

Top Quote
kaputtnik
Avatar
Joined: 2013-02-18, 20:48
Posts: 1771
Ranking
One Elder of Players
Location: Germany
Posted at: 2015-12-07, 07:54

einstein13 wrote:

What about bugs that are hard to reproduce? (like soldiers fighting forever or soldiers going out from castle to 0?)

This is just a call for testing branches which contain hopefully fixed bugs that are proposed to merge with trunk (and thus get into the game). Not for investigating open bugs. But of course everyone could every time investigate open bugs face-smile.png

GunChleoc: Isn't it a bit dangerous to have the merge command in the wiki? I fear that someone tries to merge a branch into another branch and this isn't what we want here because two (or more) branches get merged and if someone finds another bug it gets unclear which branch causes the bug.


Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3058
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2015-12-07, 10:17

Good point, I have changed the wiki.

einstein13 wrote:

What about bugs that are hard to reproduce? (like soldiers fighting forever or soldiers going out from castle to 0?)

It is always useful to have clear steps in the bug on how to reproduce, and an attached savegame where needed. This will help with bugfixes that we haven't done yet.

ETA: We now have a branch that is already reviewed and only needs testing to go in: https://code.launchpad.net/~widelands-dev/widelands/ai_level_names/+merge/279697

Edited: 2015-12-07, 10:31

Busy indexing nil values

Top Quote
einstein13
Avatar
Joined: 2013-07-29, 00:01
Posts: 1095
Ranking
One Elder of Players
Location: Poland
Posted at: 2015-12-07, 10:56

GunChleoc wrote:

It is always useful to have clear steps in the bug on how to reproduce, and an attached savegame where needed. This will help with bugfixes that we haven't done yet.

I know that, but I can think about some bugs where spotting is rather random (or have very specific situation so I thought it is random and unpredictable):

  • 0 soldiers during defese (all soldiers were going out the fortress and fight and no soldiers stayed to defend the building)
  • fully promoted soldier in trainingscamp (how it is possible? No idea)
  • Missing portspaces (after some hours of the game and few save-load games, some portspaces are missing and you can't build a port there. The problem is with savefile)
  • Ship has target to nowhere (ships are going to a place where is no port and no portspace)

I guess that the bugs above are fixed now, but in most of games I couldn't spot them. In some games they happened and reproducing them is like "random dice": you have to play for hours to see the results and not always you will see them. Those bugs happen and I can't see the way of testing it. They can happen in more than one way. Sometimes we will see them again after a month or two.

Of course those bugs are very specific and those are marginal things. I am not against testing! face-wink.png I am saying only that we can't test everything. There is number of things that we can't do manually in a short time.


einstein13
calculations & maps packages: http://wuatek.no-ip.org/~rak/widelands/

Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3058
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2015-12-07, 12:55

Yes, something will always fall through the cracks - it is important to test for what we can before merging a new branch into trunk. So far, this has been done by both the person developing the branch (and we will of course continue to do that), and then by the reviewer. What I am trying to do is to find a way to help the reviewer save some time, because I have a lot of open branches waiting for review.

And there is an additional advantage to players rather than coders doing the testing: You will click on things that we wouldn't think of.


Busy indexing nil values

Top Quote
DragonAtma
Avatar
Joined: 2014-09-14, 01:54
Posts: 254
Ranking
Tribe Member
Posted at: 2015-12-07, 14:55
I don't know if this is a "bug", but as recently as 7529 (the latest version I have), the CPU doesn't leave extra space around buildings that need them (foresters, farms, etc.). As a result, those buildings aren't as efficient as they should be.

They're also more than willing to put a forester near a farm, thus letting trees grow where the farm wants to plant crops).

Finally, they're also wiling to put a forester in a place where most of the land around them is not tree-friendly -- meaning most of the trees are planted on desert, mountain, etc. and die before they reach pull size.

Now, maybe these issues are just me overreacting (I like to leave plenty of space for foresters, farms, etc., and certainly don't want foresters to overlap with other space-needing buildings), but maybe someone else should take a look.
Top Quote