Upvote Upvoted 5 Downvote Downvoted
a good competitive balancing system?
1
#1
tf2pickup.org
0 Frags +

hey guys, i think it's a good place to ask, sooooooooooooooooo

we said we want to set up a [hl.]tf2pickup.eu in the future and one of the biggest struggles we have is that our EU community is basically fucking huge (at least 1k+ gamers for 6v6, probably a half of it would be highlander gamers (it's a crazy estimate btw)), compared to the national pickups where local communities have something between 50-200 gamers

at the current state of the tf2pickup.org project the site admin staff sets up "skills" which are reflection of the player's skill on a certain class, one value per class where off-classes on 6s aren't counted and it's a moderate task for a group of 5-10 people to balance the skills for players in order to make pickups balanced and enjoyable
these groups are i'd say mostly built from higher tier gamers + a few low gamers in order to know how to balance gamers from low-open

i think it's safe to say that balancing games even for a group of 20-30 people while having 1k+ people community would be a nightmare, so we thought of some prediction system which could help admins balance pickups easier where the prediction system is supposed as a help for an admin and where the admin can override some prediction "skill" values if they think it can be adjusted better

mały's initial idea was to make some sort of AI which could learn player's performance from all his logs (from the gamemode they play) and predict his performance on certain classes...but i'm not quite sure if that would be the best idea

another idea on the other hand would be utilizing trends.tf if they had some sort of api letting it check the player ratio and stuff, trends.tf in general is a piece of good statistical work

the simpliest system for balancing i could find (and the most broken one) is the one which is used in the tf2center chrome plugin/tf2stars where it checks the player's ETF2L/UGC current division or division from the last season and based on that gives an assessment if the player is good or bad on let's say 10 point system where one star out of five is 2 points

one huge note: we ain't making any captain based system https://docs.tf2pickup.org/docs/faq#why-no-captain-based-system so don't ask for it
reasons? people getting constantly fatkidded and simply preventing from having circlejerks destroying the site

ok and now it's a time for my question: what do you think would be the most reliable balancing system? is it one of these i mentioned above? or maybe you have a better idea? please tell us what would be the best idea in your opinion so we could start tf2pickup.eu and hl.tf2pickup.eu in 2022

hey guys, i think it's a good place to ask, sooooooooooooooooo

we said we want to set up a [hl.]tf2pickup.eu in the future and one of the biggest struggles we have is that our EU community is basically fucking huge (at least 1k+ gamers for 6v6, probably a half of it would be highlander gamers (it's a crazy estimate btw)), compared to the national pickups where local communities have something between 50-200 gamers

at the current state of the tf2pickup.org project the site admin staff sets up "skills" which are reflection of the player's skill on a certain class, one value per class where off-classes on 6s aren't counted and it's a moderate task for a group of 5-10 people to balance the skills for players in order to make pickups balanced and enjoyable
these groups are i'd say mostly built from higher tier gamers + a few low gamers in order to know how to balance gamers from low-open

i think it's safe to say that balancing games even for a group of 20-30 people while having 1k+ people community would be a nightmare, so we thought of some prediction system which could help admins balance pickups easier where the prediction system is supposed as a help for an admin and where the admin can override some prediction "skill" values if they think it can be adjusted better

mały's initial idea was to make some sort of AI which could learn player's performance from all his logs (from the gamemode they play) and predict his performance on certain classes...but i'm not quite sure if that would be the best idea

another idea on the other hand would be utilizing trends.tf if they had some sort of api letting it check the player ratio and stuff, trends.tf in general is a piece of good statistical work

the simpliest system for balancing i could find (and the most broken one) is the one which is used in the tf2center chrome plugin/tf2stars where it checks the player's ETF2L/UGC current division or division from the last season and based on that gives an assessment if the player is good or bad on let's say 10 point system where one star out of five is 2 points

one huge note: we ain't making any captain based system https://docs.tf2pickup.org/docs/faq#why-no-captain-based-system so don't ask for it
reasons? people getting constantly fatkidded and simply preventing from having circlejerks destroying the site

ok and now it's a time for my question: what do you think would be the most reliable balancing system? is it one of these i mentioned above? or maybe you have a better idea? please tell us what would be the best idea in your opinion so we could start tf2pickup.eu and hl.tf2pickup.eu in 2022
2
#2
tf2pickup.org
10 Frags +

The easiest solution would be to build an elo system.

Let's take an example:

A random user logs into tf2pickup.eu, and this magic service will look at all (configurable?) leagues and see what their average skill level is. This skill level would just be some number, maybe 1 for open and 10 for invite. Calculate the average skill by weighting the most played division by that factor. We would end up with this base line rating to what a skill level would be, and it's factored based on the overall scores of both time played and highest division played in.

You could then go into more detail on games played, where their internal score (essentially elo now) is based on number of games won (or won in a row). You can then deduct a maximum percentage based on the current score for each loss.

Basically, just write an elo system. That's all.

The easiest solution would be to build an elo system.

Let's take an example:

A random user logs into tf2pickup.eu, and this magic service will look at all (configurable?) leagues and see what their average skill level is. This skill level would just be some number, maybe 1 for open and 10 for invite. Calculate the average skill by weighting the most played division by that factor. We would end up with this base line rating to what a skill level would be, and it's factored based on the overall scores of both time played and highest division played in.

You could then go into more detail on games played, where their internal score (essentially elo now) is based on number of games won (or won in a row). You can then deduct a maximum percentage based on the current score for each loss.

Basically, just write an elo system. That's all.
3
#3
tf2pickup.org
-10 Frags +
24Basically, just write an elo system. That's all.

ELO system eventually makes gamers throw their games in order to have lower ELO and, in consequence, easier games.

[quote=24]Basically, just write an elo system. That's all.[/quote]

ELO system eventually makes gamers throw their games in order to have lower ELO and, in consequence, easier games.
4
#4
12 Frags +

ban people who throw?

ban people who throw?
5
#5
tf2pickup.org
3 Frags +
maly[/quoteELO system eventually makes gamers throw their games in order to have lower ELO and, in consequence, easier games.

While yes, people do throw games to get lower elo and thus easier games, if done right, the "formula" for elo should bounce it back.

I'm unfamiliar with how pugchamp did their elo system nor do I know much about if this is a real impact for pugchamp, but that's where I'd start: learning what pugchamp did and hear feedback from pugchampers about their games.

[quote=maly[/quote]
ELO system eventually makes gamers throw their games in order to have lower ELO and, in consequence, easier games.[/quote]

While yes, people do throw games to get lower elo and thus easier games, if done right, the "formula" for elo should bounce it back.

I'm unfamiliar with how pugchamp did their elo system nor do I know much about if this is a real impact for pugchamp, but that's where I'd start: learning what pugchamp did and hear feedback from pugchampers about their games.
6
#6
7 Frags +
supra
mały's initial idea was to make some sort of AI which could learn player's performance from all his logs (from the gamemode they play) and predict his performance on certain classes...but i'm not quite sure if that would be the best idea

if it doesn't take into account what level they play at it would be incredibly unbalanced, with players who stomp tf2centers rated way above players who only scrim in higher divs and can't put out the same stats.

[quote=supra]

mały's initial idea was to make some sort of AI which could learn player's performance from all his logs (from the gamemode they play) and predict his performance on certain classes...but i'm not quite sure if that would be the best idea

[/quote]

if it doesn't take into account what level they play at it would be incredibly unbalanced, with players who stomp tf2centers rated way above players who only scrim in higher divs and can't put out the same stats.
7
#7
7 Frags +

For the 6s version you could bascially just take the elo from every national site, balance it out so the ratings are adjusted to the different sites settings and assign the players the adjusted elo.

Obviously a lot of people would be missing (mainly brits i guess) but you could always just have a team of a few knowledgable people adjusting their elo. Also helps that basically every div is represented on the national sites so you kinda know which elo to assign to the players.

We also have a bot that shows us the winrate on each class for the german pugs so the admins could use that bot as a tool to balance it out incase 5-0s happen constantly.

For the 6s version you could bascially just take the elo from every national site, balance it out so the ratings are adjusted to the different sites settings and assign the players the adjusted elo.

Obviously a lot of people would be missing (mainly brits i guess) but you could always just have a team of a few knowledgable people adjusting their elo. Also helps that basically every div is represented on the national sites so you kinda know which elo to assign to the players.

We also have a bot that shows us the winrate on each class for the german pugs so the admins could use that bot as a tool to balance it out incase 5-0s happen constantly.
8
#8
tf2pickup.org
2 Frags +
LoudFor the 6s version you could bascially just take the elo from every national site, balance it out so the ratings are adjusted to the different sites settings and assign the players the adjusted elo.

Obviously a lot of people would be missing (mainly brits i guess) but you could always just have a team of a few knowledgable people adjusting their elo. Also helps that basically every div is represented on the national sites so you kinda know which elo to assign to the players.

We also have a bot that shows us the winrate on each class for the german pugs so the admins could use that bot as a tool to balance it out incase 5-0s happen constantly.

i like that idea a lot, brits aren't a problem, their skills could be assigned manually

[quote=Loud]For the 6s version you could bascially just take the elo from every national site, balance it out so the ratings are adjusted to the different sites settings and assign the players the adjusted elo.

Obviously a lot of people would be missing (mainly brits i guess) but you could always just have a team of a few knowledgable people adjusting their elo. Also helps that basically every div is represented on the national sites so you kinda know which elo to assign to the players.

We also have a bot that shows us the winrate on each class for the german pugs so the admins could use that bot as a tool to balance it out incase 5-0s happen constantly.[/quote]
i like that idea a lot, brits aren't a problem, their skills could be assigned manually
9
#9
0 Frags +

AI would only classify skill/balance properly if the dataset you're using (logs.tf in this case) isn't a complete mess on a player per player basis. You'd have to prepare and clean up that data and afaik there's roughly three million logs so good luck with that. There's people with logs for dumb maps like 2fort upsidedown, mess around pugs, ultiduo bball HL 6s Prolander 4v4 you name it. Some of these will be from scrims, offis, TF2C, pugs or other stuff. It would also need to consider individual classes, other individuals on the team you're playing to really make sense I think. Probably do-able with something like Keras/TensorFlow, but sounds like a pain to train a model for that with all the variables you have.

Not only that, but you'd have to feed it data it can actually understand. Just throwing 30GB of raw text logs at AI isn't going to do much so you'd probably end up making several monster CSV/JSON files first (parsed from those 30GB logs).

Using some sort of elo score or assigning a score based on ETF2L/RGL division sounds a lot easier. If you want to go the extra mile you can check amount of matches played on ETF2L, how many won/lost etc. But if your goal is to just make the life of a handful of people easier, I think that'll go a long way.

If you like what you have with trends.tf you could just scrape the data rather than use an API. You should probably ask the creator if they are fine with that though.

AI would only classify skill/balance properly if the dataset you're using (logs.tf in this case) isn't a complete mess on a player per player basis. You'd have to prepare and clean up that data and afaik there's roughly three million logs so good luck with that. There's people with logs for dumb maps like 2fort upsidedown, mess around pugs, ultiduo bball HL 6s Prolander 4v4 you name it. Some of these will be from scrims, offis, TF2C, pugs or other stuff. It would also need to consider individual classes, other individuals on the team you're playing to really make sense I think. Probably do-able with something like Keras/TensorFlow, but sounds like a pain to train a model for that with all the variables you have.

Not only that, but you'd have to feed it data it can actually understand. Just throwing 30GB of raw text logs at AI isn't going to do much so you'd probably end up making several monster CSV/JSON files first (parsed from those 30GB logs).

Using some sort of elo score or assigning a score based on ETF2L/RGL division sounds a lot easier. If you want to go the extra mile you can check amount of matches played on ETF2L, how many won/lost etc. But if your goal is to just make the life of a handful of people easier, I think that'll go a long way.

If you like what you have with trends.tf you could just scrape the data rather than use an API. You should probably ask the creator if they are fine with that though.
10
#10
3 Frags +

If anyone wants to write an ELO system, let's talk.
arcadia#0001

If anyone wants to write an ELO system, let's talk.
arcadia#0001
11
#11
tf2pickup.org
2 Frags +
Poptobobban people who throw?

it's a bit more complicated than you think.
based on our experience from national website and our own playground which is tf2pickup.pl and hl.tf2pickup.pl we've had a lot of trouble with it because it's just not that easy to catch besides obvious cases where someone play some junky shit like direct hit and that person direct misses everything, unwilling to change the weapon therefore throwing for their team

for cases like this on 6s we've banned several weapons, on highlander we've banned for example all demoman shields, i know a lot of 6s sites have done it already, for instance market gardener is banned on tf2pickup.pl, that specific ban is due to a current ETF2L whitelist and so far i think people don't complain much about it in Poland

however this was the easy part of kinda obvious cases, the not obvious cases are where people just play carelessly and it's really hard to assess if someone is trolling, throwing or just bad (like me in 6s),

also mały had in mind people who rather to sandbag than get more experience and play better and again, it's not bad that someone is mediocre in game like me and does not feel keen to get better because obviously it's just a game and not everyone's top life priority, but what really bad is that some people even on the ETF2L try their best to get into the lowest division possible just in order to have easy game and i know mały wants these people out of the site

however by these arguments ELO sounds like a not bad idea, probably worth considering

[quote=Poptobob]ban people who throw?[/quote]
it's a bit more complicated than you think.
based on our experience from national website and our own playground which is tf2pickup.pl and hl.tf2pickup.pl we've had a lot of trouble with it because it's just not that easy to catch besides obvious cases where someone play some junky shit like direct hit and that person direct misses everything, unwilling to change the weapon therefore throwing for their team

for cases like this on 6s we've banned several weapons, on highlander we've banned for example all demoman shields, i know a lot of 6s sites have done it already, for instance market gardener is banned on tf2pickup.pl, that specific ban is due to a current ETF2L whitelist and so far i think people don't complain much about it in Poland

however this was the easy part of kinda obvious cases, the not obvious cases are where people just play carelessly and it's really hard to assess if someone is trolling, throwing or just bad (like me in 6s),

also mały had in mind people who rather to sandbag than get more experience and play better and again, it's not bad that someone is mediocre in game like me and does not feel keen to get better because obviously it's just a game and not everyone's top life priority, but what really bad is that some people even on the ETF2L try their best to get into the lowest division possible just in order to have easy game and i know mały wants these people out of the site

however by these arguments ELO sounds like a not bad idea, probably worth considering
Please sign in through STEAM to post a comment.