Latest Posts

Changes in RegressionTests

Editor Comment

Tweaked the documentation & added a TOC.


Revision Differences of Revision 14

When you think your branch is ready for merging, there are two tests you can run to make sure that your code sticks to our coding style, and to test for a bunch of possible regression bugs. ¶

[TOC] ¶

## Running Codecheck Manually ¶


We have codecheck rules that will help you stick to our coding style guidelines. These are run automatically whenever you compile a debug build, but they can be hard to see now, because cmake creates a lot of output around them. However, you can run the checks manually: ¶

`~~~~ ¶
cd build && make codecheck`
~~~~

If you see lots of "see_if_codecheck_needs_to_run_" messages, it will be easier to run: ¶

`~~~~ ¶
cmake/codecheck/CodeCheck.py src/* | grep -v "src/third_party"`
~~~~

Note that this second command won't check the `CMakeLists.txt` files. ¶


## RunningTest RScegnaression Tests ¶

Some tests can be run automatically while compiling debug builds, but not all of them. We have some automated scenarios that will check for regression bugs.

### Full Test Suite ¶

 You can run all of test in them test suite with ¶

`~~~~ ¶
./regression_test.py -b ./widelands`
~~~~ ¶

### Single Test Scenario ¶

#### By Filename

If you only wish to run part of the testsuite, you can filter with regex for the filename, like this: ¶

`~~~~ ¶
./regression_test.py -b ./widelands -r=portdock`
~~~~

Thatis will run all test(s) (*lua files) with a keyword (portdock in this example) in their file path. You can of course use the entire filename to run just one test. Have a look at the tests directory for which tests are available. ¶


#### With Direct Console Output ¶

While debugging something specific or if you want to run a scenario from within a debugger, you can also use a different commands to get the output directly on the console. - nNote that this won't set up the complete testing environment, so you will always need to use one of the above commands to verify when you're done. Depending Yonu whichll tneest,d ytou specan trify bothe following two commands: ¶

`./widhelands --scenario=test/ maps/lu a_nd thest suitpe.wmcif --datadir_for_c test scring=.` ¶
pt:
or:
~~~~
`./widelands --datadir=data --datadir_for_testing=. -scenario=test/maps/ship_transportation.wmf --script=test/maps/ship_transportation.wmf/scripting/test_rip_second_port_with_worker_in_portdock.lua
~~~~ ¶

Note that `lua_testsuite.wmf` needs special treatment - run it with:

~~~~ ¶
./widelands --scenario=test/maps/lua_testsuite.wmf --datadir_for_testing=. ¶
~~~~ ¶

### Help ¶

For **help**, run ¶

`~~~~ ¶
./regression_test.py -h`
~~~~

or

~~~~ ¶
./regression_test.py --help.
~~~~

### Required pPython vVersion###

Also note that CodeCheck.py and regression_test.py **require python 2.7** or 3.4.x (tested with 3.4.4), so if you get errors, ¶
check your version and then try to force the correct version like this ¶

`~~~~ ¶
python2.7 ./regression_test.py -b ./widelands`
~~~~
`~~~~ ¶
python3.4 ./regression_test.py -b ./widelands`
~~~~

providing you are using **linux** or similar system. Chances are high that python2.7 is already installed on your box, if not use your package manager to install it. ¶