Latest Posts

Topic: Rating system

einstein13
Avatar
Joined: 2013-07-29, 00:01
Posts: 1118
Ranking
One Elder of Players
Location: Poland
Posted at: 2019-07-03, 00:14

Two more things from my side:

  1. I think that we need one major thing in the game: surrender button.
  2. Team games can be treated as 1 vs 1 player with the values taken as averages.

AD 1.: it will help not only rating, but also other games. Now we have to defeat completely players to reach win message window. But according to many replays from tournaments it is not the only case when we say "I win/ lost". But this button should also affect replay! There should be a message window that somebody just surrendered.

AD 2.: the average has not to be simple average A = (P1+P2)/2, it can be any other proper average. I am not familiar too much with rating systems, but I assume that if there are 3 players with numbers: P1=1300, P2=1400 and P3=1500, then the difference on power between P1 and P2 is not the same as P2 and P3. But it is not the problem. We should only find the proper equation for average numbers. And if you want to stick to Gicko, Standard Deviation value can be easily calculated with known rules (I can help you with that calculations if needed). So after all we will know the change and put the change to players' rating numbers.


einstein13
calculations & maps packages: http://wuatek.no-ip.org/~rak/widelands/
backup website files: http://kartezjusz.ddns.net/upload/widelands/

Top Quote
trimard
Avatar
Topic Opener
Joined: 2009-03-05, 22:40
Posts: 230
Ranking
Widelands-Forum-Junkie
Location: Paris
Posted at: 2019-07-03, 12:11

@kaputtnik

aaaah, ok thanks this makes more sense. It seems to me, we should rather try to do our own system rather than implement an old outdated one. I don't think the difficulty is there anyway. More in the communication between players and the sever

@Worldsavior

If you really want you could start with only one rating, and as soon as it seems pointful to create a second rating number for another win condition one could still split that.

Yes, I think that's what we might end up doing. Easier to put into place. But I would still like to motivate players to do even map at 4 or 6 players. Those are so fun! See einstein's idea.

Why changed? [selected maps]

  • Because that would motivate map creator do make more maps.
  • Because it avoid too much repetitiveness for regular players.
  • It makes it easier to change the map or the type of game if we see it's needed.

The rating system could encourage more playing. And if tournaments get ranked, it's already a good number of games ...

Yes but I think even big AAA games have troubles finding games for 2vs2, 3vs3 etc.

Before I forget it: It can also happen that somebody who is about to lose quits the match and never continues it. After a while this should be considered as loss...

Yes, we need an "abandon" button as Einstein suggested. And maybe a set time to finish a game? Or maybe we simply don't allow games to be reported? Only like 1 hours for connections problem and the likes?

Age of Mythology used a rating system which also worked for 2vs2, 3vs3 and so forth, this is really not rare. But I don't know how it works...

Most likely either a own rating per "couple" in 2vs2 games or simply a rating per person using the rules proposed by einstein

It may be possible to implement the rating system first and to implement anti-cheat-measures afterwards.

Yes!

@einstein13

surrender button.

Yes yes yes yes yes!

Team games can be treated as 1 vs 1 player with the values taken as averages.

Yes. If we take 2vs2 and 3vs3 as separate. But I just don't see a way to add all these game to the same ranking and still be relevant. I simply don't think the level of a player in 2vs2 is a good indicator on its level on 1vs1.

Now if we take the 2vs2 games as separate. I'm not sure a direct average is pertinent:

  • P1 = 500, P2=1000, P3=1500, P4=2000.
  • P1 + P4 vs P2 + P3. Same average number of point. But.
  • If P1 + P4 wins. What does this mean? For P4, it shows a good play, because he most certainly carried the game against two relatively good players. For P1, it might no be such a big achievement. Or should we consider the opposite? P1 stood forth against better ranked players whereas P4 simply stood against lower ranked player...

Reasoning is even weirder for the standard deviation. I think the bigger the difference between the two players playing together, the bigger the deviation should be. i.e bigger deviation for P1 & P4 when they win than if P3 & P2 win. You say you know some good rules for that one?

Now average would be much easier to implement. And maybe becomes pertinent after a while? I have trouble making my mind on that.

Maybe we implement what seems best (i.e what you described) and adapt the calculus when we see the need?

Edited: 2019-07-03, 12:12

Top Quote
king_of_nowhere
Avatar
Joined: 2014-09-15, 18:35
Posts: 1668
Ranking
One Elder of Players
Posted at: 2019-07-03, 15:33

well then I think we could start by implementing a surrender button. that won't hurt, and it will make things easier in the future.

trimard wrote:

Team games can be treated as 1 vs 1 player with the values taken as averages.

Yes. If we take 2vs2 and 3vs3 as separate. But I just don't see a way to add all these game to the same ranking and still be relevant. I simply don't think the level of a player in 2vs2 is a good indicator on its level on 1vs1.

While it is easy to come up with examples where the system does not work, it does work unless you cherry pick a special case. League of legends does it for matchmaking purpose, and it works perfectly fine.

Also, the information is not a secret, so I think if someone logged in their forum and asked about the algorithm, they could get some good information.


Top Quote
trimard
Avatar
Topic Opener
Joined: 2009-03-05, 22:40
Posts: 230
Ranking
Widelands-Forum-Junkie
Location: Paris
Posted at: 2019-07-03, 16:31

Also, the information is not a secret, so I think if someone logged in their forum and asked about the algorithm, they could get some good information.

That's a good starting point, but I think there are many places we could ask. I'll start on some reddit sub, cause I'm more used to it.

well then I think we could start by implementing a surrender button. that won't hurt, and it will make things easier in the future.

Yes I'll add as a feature request on launchpad. BTW I just post a bug and an administrator will mark it as whishlist right? Or is there a dedicated page?


Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3324
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2019-07-03, 17:18

I don't have much time right now, so just skim-read this thread.

0AD has a rating system. From their experience

  • It is immpssible to distinguish people quitting the game when they are losing (cheat) from internet connection lost (no cheat)
  • If players lose their internet connection, they are asked to start a new game and resign to give the other player their points
  • Moderation is still needed for disputes
  • Players decide at start of match whether it will be ranked.
Edited: 2019-07-03, 17:18

Busy indexing nil values

Top Quote
trimard
Avatar
Topic Opener
Joined: 2009-03-05, 22:40
Posts: 230
Ranking
Widelands-Forum-Junkie
Location: Paris
Posted at: 2019-07-03, 17:28

Awesome, thanks GunChleoc, it motivates to even more simplify the idea!

  • It is immpssible to distinguish people quitting the game when they are losing (cheat) from internet connection lost (no cheat)
  • If players lose their internet connection, they are asked to start a new game and resign to give the other player their points Moderation is still needed for disputes

Why not get even more simple and just say, if you quit the game it equals to losing? Wouldn't that be too harsh? But you seem to say, that's how they do it in your game. So why not face-smile.png ?

Moderation is still needed for disputes

Yes, and I think we would need to create an interface for that too

Players decide at start of match whether it will be ranked.

Yes, hard to verify though face-tongue.png


Top Quote
einstein13
Avatar
Joined: 2013-07-29, 00:01
Posts: 1118
Ranking
One Elder of Players
Location: Poland
Posted at: 2019-07-03, 22:41

trimard wrote:

Why not get even more simple and just say, if you quit the game it equals to losing?

Because there are some situations when you don't want to say "I lost", for example continuing the game in the future. So from my point of view, for rank games there should be another type of exiting the game, which will indicates if the game counts to rank directly or wait for other ending of the game. And the thing is complicated now... face-sad.png

trimard wrote:

If P1 + P4 wins. What does this mean? For P4, it shows a good play, because he most certainly carried the game against two relatively good players. For P1, it might no be such a big achievement. Or should we consider the opposite? P1 stood forth against better ranked players whereas P4 simply stood against lower ranked player...

First, I don't know if lower means better or worse in Glicko system.
Second, I haven't told that simple average is the best here. It can be any type of average, including our own, based on Glicko formulas.
Third (and probably it is the most important), teams means team work. If strong player wins over two weaker ones and the weakest didn't help, that is the problem of their team, not the rank - that is my opinion. And probably if you ask king_of_nowhere, he will admit that even killing one good soldier can sometimes say "yes" or "no" to win. So even weak player can help the strongest of us. That is how I understand the situation.


einstein13
calculations & maps packages: http://wuatek.no-ip.org/~rak/widelands/
backup website files: http://kartezjusz.ddns.net/upload/widelands/

Top Quote
king_of_nowhere
Avatar
Joined: 2014-09-15, 18:35
Posts: 1668
Ranking
One Elder of Players
Posted at: 2019-07-04, 01:00

GunChleoc wrote:

  • It is immpssible to distinguish people quitting the game when they are losing (cheat) from internet connection lost (no cheat)
  • If players lose their internet connection, they are asked to start a new game and resign to give the other player their points

well, there could be a system where you have some time to reconnect. So when you crash, you have, say, 10 minutes to come back in lobby and rehost from save. if you do, the game continues. if you don't, you lost. if you host and your opponent does not show up, your opponent lost.

or, since the replays are saved, we could also save replays of ranked games and look in case of disputes.

but at least at first i would not worry about cheaters too much.

Edited: 2019-07-04, 01:03

Top Quote
GunChleoc
Avatar
Joined: 2013-10-07, 15:56
Posts: 3324
Ranking
One Elder of Players
Location: RenderedRect
Posted at: 2019-07-05, 09:15

We should think about cheaters, because that will also inform how the programming will be designed. It does not have to be perfect at first go, but the less redesigning we have to to later, the better. We will definitely need the possibility of uploading replays.

We could encode into the game/replay whether it was set to be ranked, this should be no problem.

Do we want to adjust the score accorting to starting condition (headquarters, fortified village etc)?

Some interesting threads in the 0AD community:

Edited: 2019-07-05, 09:15

Busy indexing nil values

Top Quote
WorldSavior
Avatar
Joined: 2016-10-15, 04:10
Posts: 2091
OS: Linux
Version: Recent tournament version
Ranking
One Elder of Players
Location: Germany
Posted at: 2019-07-05, 10:33

trimard wrote:

The rating system could encourage more playing. And if tournaments get ranked, it's already a good number of games ...

Yes but I think even big AAA games have troubles finding games for 2vs2, 3vs3 etc.

I don't think so.

Before I forget it: It can also happen that somebody who is about to lose quits the match and never continues it. After a while this should be considered as loss...

Yes, we need an "abandon" button as Einstein suggested. And maybe a set time to finish a game? Or maybe we simply don't allow games to be reported?

Reported? Do you mean something else, like "paused"?

Only like 1 hours for connections problem and the likes?

Or ten minutes like king_of_nowhere suggested

GunChleoc wrote:

I don't have much time right now, so just skim-read this thread.

0AD has a rating system. From their experience

  • It is immpssible to distinguish people quitting the game when they are losing (cheat)

This is a mistake in the system and not a cheat.

from internet connection lost (no cheat)

  • If players lose their internet connection, they are asked to start a new game and resign to give the other player their points

This is really harsh because it doesn't allow a simple immediate reload of the savegame. One just has to make impossible that the losing player pushes for reloads all the time

And it shows even more that someone who refuses continuing a match should be considered just as player who lost, and not as a cheater.

king_of_nowhere wrote:

GunChleoc wrote:

  • It is immpssible to distinguish people quitting the game when they are losing (cheat) from internet connection lost (no cheat)
  • If players lose their internet connection, they are asked to start a new game and resign to give the other player their points

well, there could be a system where you have some time to reconnect. So when you crash, you have, say, 10 minutes to come back in lobby and rehost from save. if you do, the game continues. if you don't, you lost. if you host and your opponent does not show up, your opponent lost.

This is so much better than the system of 0AD (if one cannot rehost more than a few times, let's say 2 or 3, to avoid abuse of that rule). I can not understand how there can be such a mistake in the system of 0AD...

At the other hand one could also think about using the system of your tournament which allows bigger breaks than 10 minutes (rescheduling).

GunChleoc wrote:

Do we want to adjust the score accorting to starting condition (headquarters, fortified village etc)?

This might be really complicated. For example a FV is a much bigger advantage on a small map than on a huge map, where it doesn't matter that much.

So it's easier to start rating system for headquarters only, isn't it?


Wanted to save the world, then I got widetracked

Top Quote