Welcome to the Forum Archive!

Years of conversation fill a ton of digital pages, and we've kept all of it accessible to browse or copy over. Whether you're looking for reveal articles for older champions, or the first time that Rammus rolled into an "OK" thread, or anything in between, you can find it here. When you're finished, check out the boards to join in the latest League of Legends discussions.

GO TO BOARDS


[Discussion] The Mathematics of Elo Hell

Comment below rating threshold, click here to show it.

MathMage

Senior Member

12-08-2010

I see pockets of mathematical analysis scattered throughout the Elo Hell QQ threads. Typically this will be one person informing the world that your bad teammates are just as likely to be bad opponents next game; that 4 chances for your team to have a leaver and 5 for the other team leads to your net benefit; that probability contradicts Elo Hell. All well and good in the abstract. But the League of Legends solo queue ranked playerbase is a complex system of some 100,000 units being measured individually according to their performance on a 5-man team. One calculation of expected values is not going to illuminate much of anything about such a system; it's far too simplistic. In this thread I hope to spur an informed investigation into the mathematics of Elo Hell, to find out whether the numbers show a real problem, and whether we can mitigate that problem by tweaking the system and changing the numbers.

Let's start by noting down some basic things about the Elo system in solo queue.
-Everyone starts at 1200 Elo. This is the original average of the playerbase.
-A match is made by finding a group of 10 people who are close in Elo, and arranging those people so that the match's outcome is 50-50, according to each person's Elo. After the match, winners gain Elo and losers drop in Elo according to how many recent matches they've played, as well as other factors that are presumably Riot-proprietary.

  • Duo queueing combines and inflates the Elo of the duos when grouping players up.
  • The selectivity of the matchmaker decreases as queue time lengthens.
-Any given match should not change the average Elo. There are exceptions--I'll note the ones I'm aware of:
  • Queue-dodging reduces the Elo of the dodger, without adding to anyone else's Elo, hence a drop in the average.
  • Inactivity leads to Elo reductions; as far as I know, it's 2 weeks for 50 Elo if you're above 1400. If that is the case, this should be relatively insignificant.
  • Leavers may face additional Elo penalties. Whether this is compensated for in the other 9 players' Elo changes is unknown.
  • Loss forgiveness leads to Elo inflation. Since this occurs only during server instability, its impact should not be large.
  • Accounts with low Elo are probably more likely to be abandoned, which may lead to Elo inflation.

Now, to analyze Elo Hell, we need a definition. Elo Hell occurs when a player's Elo remains significantly below his actual level play over a number of games, typically because his teammates are worse than his opponents (whether it be raging, leaving/AFKing, griefing, or simple poor play that causes this). I hesitate to attach exact numbers, but we can probably use 150 Elo and 50 games as first approximations.

The first obvious conclusion is that Elo Hell is a function of probabilistic outliers. This is why a straight calculation of expected values does not make sense. Were the player's score to follow expected values, he would not be in Elo Hell in the first place. We need analysis of the possible deviation from our expected values. What follows is a slightly better, though still extremely simplistic, calculation of how likely it is that someone who has fallen far below their 'true Elo' would remain there after 50 games.

Assume that any given player may weight their team's chances to win at their Elo by one point--0 to 1. Let us further assume that any given game you play in is 'fair'--no leavers, everyone plays to their weighted score, and the greater number wins. The last assumption is uniform distribution between 0 and 1 for the playerbase at any Elo.

Let's look at a player whose Elo is already far below where it should be, trying to play his way back up the ladder. This player should have a solid influence on any game--say, 0.8. In chess, a 0.75 chance of winning is represented by a 200-Elo difference; not to say that this applies exactly to LoL, but that's to put the number 0.8 in perspective.

So your team's score varies from 0.8 to 4.8, and the other team's score varies from 0 to 5. This gives you a 56% chance of victory in any given match. In the long run, that's to your benefit; but how long does it take for the law of large numbers to come into play?

Typically, after about 50 games, about 24% of such players will go no better than 25-25. That's a pretty significant percentage of players that will feel they're being kept down by their teammates. The consolation is it's hard to fall further: less than 2% of such players will go 20-30 or worse.

I don't pretend that this is a very realistic representation of the playerbase. For example, I by no means expect a uniform distribution of player skill between 0 and 1; more likely one would encounter a normal distribution with mean 0.5 and a standard deviation of 0.3 or so, and a spike at -3 for leavers. I simply want to demonstrate how the expected value is by no means the be-all and end-all of these probability calculations.

This is not because MM is broken and Riot sucks. It's simply the nature of trying to measure individual ability in a 5v5 game with pseudorandom teams. There may be other factors that exacerbate the problems with the Elo system, that can be mitigated; Elo deflation is an example. But the mere existence of outliers is not something Riot can fix by waving a magic wand.

I welcome critique of my math thus far, but what I really hope for is that other people will contribute their own analysis. In the best case, a red post contribution with some insight into any thought Riot has put into this problem would be invaluable.

tl;dr: No, dammit. If you want to skip over the math, you shouldn't be reading a thread about math in the first place.

(For those who are curious about my numbers: I calculated the probability of winning 25 games exactly (.44^25 * .56^25 * 50!/(25!25!)), then 24 games, then 23 games, and so on down to 15 games, at which point the cumulative probability was only increasing by a fraction of a percentage point over each iteration, so I stopped. The odds of winning less than 15 games out of 50 are infinitesimal. Yes, that was a laborious process. I'm glad Google works as a calculator.)


Comment below rating threshold, click here to show it.

MathMage

Senior Member

12-08-2010

Highlighting interesting ideas/potential investigations here. My ratings are speculative opinion and should not be taken as the final word, especially in the (noted) cases where I am evaluating my own ideas (or at least, ideas that I contributed to this thread; they're nothing especially original). A high rating means I think it's a promising idea that is likely to become something Riot could implement; a low rating means I think it's impractical, high research-to-reward ratio, or simply a bad idea.

  • Ladder System--2/5. It's another possible system, with its own advantages and disadvantages. Unlikely to be implemented.
  • Margin of Victory--4/5. Using some of the wealth of information available to refine simple W/L statistics, without breaking it down by individual, could definitely improve the system, if only by weighting Elo changes by a few points either way.
  • Player Role Tendencies--2/5. Improves the baseline of pick coordination. May inhibit creative team comps, and most solutions to that involve user-side complications, though it could be argued that weird comps belong in arranged queue anyway.
  • Concealed Algorithm--3/5. Theoretically sound, but unlikely to be implemented because the playerbase will QQ about transparency.
  • Modifying Starting Elo--5/5. I think factors that lead to Elo inflation/deflation are a systematic problem that need to be accounted for, so that the starting Elo and average Elo stop drifting apart. Disclaimer: I previously wrote a thread (http://www.leagueoflegends.com/board/showthread.php?t=357528) urging similar changes.
  • Team gold as weighting metric--4/5. Technically a branch of the Margin of Victory idea.
  • KDA-weighted Elo changes--1/5. I have yet to see a proposal for how to determine a good weighting for individual statistics, only guesstimated ratios based on minimal theorycrafting. And the potential for abuse is huge.
  • Reversing the Matchmaker--4/5. Elo is fundamentally not a system designed to force 50/50 matches. Making teams with smaller Elo spreads should result in more accurate ratings. However, matches between unequally rated teams will become more common, by design. Disclaimer: My idea.
  • Redistribute Elo loss to weight leaves without deflation--5/5. A clean and simple way to discourage leaves, forgive the teammates of leavers, and avoid Elo deflation in the process. No substitute for a strict and comprehensive leaver ban system, though. Disclaimer: My idea.

Ladder System
Quote:
Wankeroo:
ELO HELL: Constantly dealing with new players. The new players introduced into the ranking system constantly pour into the games of people who are anywhere between 1100 and 1400 rank.

The problem is the ranking system is not a "ladder" where there are low-skilled players at the bottom and high skilled players at the top and 50% win ratio players like 2/3 of the way up the ladder where they will be matched with other players of their ilk and they will not constantly be dealing with new players in their matches.


Margin of Victory (http://www.leagueoflegends.com/board/showthread.php?t=385895)
Quote:
rawpower405:

I do not think Riot should assess individual performance in a game. However, they can take into account the teams final K/D/A/Towers and develop a margin of victory function. The whole point is to develop additional information from taken from the matches that are played to produce a better way of interpreting the outcome.

We've all played close games where the final scores were like 50-48. But we've also all played games where the final scores were like 3-25 with a surrender at 25 minutes. Both of those games SHOULD NOT be treated equally since the final scores are so vastly different. One game was won much more conclusively than the other.


Player Role Tendencies (credit to jalb3rt for original idea)
Quote:
Solberg:
Player Role Tendencies would be great statistics to match making.

If you think about it in plain english, it's really easy to see why. Having a good team comp for both teams automatically decreases the snowball effect of the game (variance of the game) so that from champion select, one team is not already at an advantage.

From a player perspective, the question is a simple one. Would you want a player that's 100 elo higher but is just gonna compete with your solo lane champion while the other team's jungler ganks you lane with double buffs while having two jungle to work with? It doesn't matter how skilled you are at techniques at that point when their ganks are 3 lvls higher than you. And from that point, it just snowballs and your lane gets called a feeder.


Concealed Algorithm
Quote:
Solberg:
Elo as a way of ranking players is fine. I'm of the opinion that the true match making algorithm never be revealed to the players because

1) It removes any effect of players gaming the system
2) It allows riot to implement randomized test to find continue to find a better algorithm
3) I'm of the opinion that the true algorithm should use machine learning techniques to constantly improve according to a learning algorithm so that it has the ability to change just like the players and the metagame.


Modifying Starting Elo (http://www.leagueoflegends.com/board/showthread.php?p=4544556#post4544556)
Quote:
jalb3rt]1. Keep track of every point forfeited by queue dodging. When that number hits the number of active ranked players, give every active player a point.

2. Players playing their first ranked game are given a rating equal to the average Elo currently in the system

3. Give players entering Ranked an initial rating weighted by their normal Elo. Obviously not fully weighted, as normal Elo is probably not an exact predictor of ranked performance, but placing players on a bell curve with a +-150 pt spread from the normal start based on how they fall on the bell curve of the Normal Elo when hitting 30 will get players sorted a lot quicker, and small errors will work themselves out as fast or faster than they do now.[/quote]

Team Gold as weighting metric
[QUOTE=deciett
:
If you took into account a state-of-win multiplier to the elo loss and gain from a match (something between 1 and .5) that varied depending on match conditions, some complex algorithm taking takedown ratios, cs ratios, and objective ratios into account, you could probably arrive at an Elo system that is both a more accurate representation of your true skill at the game AND a more viable system of matchmaking.

For practical applications, the gold earned ratios of both teams would largely be a good state-of-victory decider. The team that is winning will get more time to farm, the one that is losing will get less, and the team with the upper hand will be controlling more objectives. Special weight would be applied to important objectives that have higher-than-gold value for their team (IE, blue buff, baron), but on the whole a team that has had a more dominant game inarguably earns more gold overall. There will be the occasional match where the winning team will have such a ratio as <1, typically a sign of very good teamwork, which could be considered grounds for INCREASING the elo gained and lost by both teams.

This would solve two problems: Elo gains and losses would be based on the team's performance during a game. Close games would be awarded more elo points because they decisively portray the expected performance of both teams. Inversely, snowball games would have less impact on Elo, helping to remove outliers from the elo system.


KDA-weighted Elo changes
Quote:
EasymodeX:
Remember, the actual KDA numbers don't matter. What matters is their value relative to the team average in order to find outlier players in the game. KDA is honestly fine. If you want to try harder, you can do KDA with a factor of CS. Towers are probably a bad idea overall, since there are few towers and the credit can be inconsistent.

If your KDA is abysmal (by some evaluation) compared to your team's average, then your win is diminished (no adjustment to losses). If your KDA is super compared to your team's average, then your loss is diminished (no adjustment to wins).


Reversing the Matchmaker
Quote:
MathMage:
Ordinarily, the matchmaker finds 10 people close in Elo and tries to create a 50-50 game between them. The result is that one team may have a large Elo spread, and determine the game between the feeder and the carry. Teams with a large Elo spread are generally frustrating all around--4 people get annoyed at the feeder, and the other team gets annoyed about the carry.

Consider a system where the matchmaker tries first to find 5 people whose Elos are close together, and make a team of that. Then it tries to match two teams together, trying for an even match, but accepting more unequal matches as time goes by. In short, equal intra-team Elo is prioritized over equal inter-team Elo. Obviously, unequal games would result in a small Elo change if the 'better' team wins, and a large change if the 'worse' team wins.


Redistribute Elo loss to weight leaves without deflation
Quote:
MathMage:
There is one obvious situation where one person can be blamed more than the rest: a leave. There's also an obvious solution: Weight the Elo loss more heavily on the leaver. For example, have the leaver take 4x the normal loss, while everyone else takes 1/4. Or have the leaver take 3x and everyone else 1/2. Any ratio that comes out to the same total Elo loss, so no Elo inflation/deflation occurs. It's both a fast way to sort persistent leavers from the rest, and a severe disincentive to leave.


Comment below rating threshold, click here to show it.

Pete the Puma

Senior Member

12-08-2010

Impressive work, if I may suggest a clear conclusion sentence/paragraph.

I take it only 24% of people who have an "underrepresented Elo" (Elo number lower than true skill) will go worse than 25-25 W/L due to sucky teammates.

Also here is more math for you Math pleasure: (from another post)

Quote:
There is no such thing as Elo hell...

what people fail to realize is that the wast majority of players are between 1040-1360 Elo. In fact ~86% of players are in this bracket. And everyone believes they belong > 1500.... only the best 2% will make it there...

So let us please stop complaining about "Elo Hell"

L'enfer c'est les autres.

Statistics found here:

http://www.leagueoflegends.com/board...d.php?t=269189 (http://www.leagueoflegends.com/board/showthread.php?t=269189)

P.


Comment below rating threshold, click here to show it.

Vecuu

Senior Member

12-08-2010

Good read.


Comment below rating threshold, click here to show it.

Ellomdian

Member

12-08-2010

I respect your initiative and theory. A Lot.

I feel that the issue is not actually one's ELO ranking - e-peen be ****ed. It is the likelihood of encountering one of your outliers (feeder/leaver/Terribad player) in "ELO hell"

I don't care if I play against or with players slightly below or above my skill level. I care when a single player is disruptive enough to pre-determine the outcome of the game. I would have no issue spending all my time in 9xx ELO if there were no leavers or intentional feeders.

While I feel your theory is solid, you by default have to dismiss outliers because without that data from RIOT (and for the record, if someone doesn't get a daily *(HOURLY?)* report of recorded leavers in a particular elo bracket, they should be,) you model is impractical. Odds are (THEORY TO FOLLOW) people in lower brackets tend to leave more games, therefor poisoning your model.


Comment below rating threshold, click here to show it.

MathMage

Senior Member

12-08-2010

Quote:
Pete the Puma:
Impressive work, if I may suggest a clear conclusion sentence/paragraph.

I take it only 24% of people who have an "underrepresented Elo" (Elo number lower than true skill) will go worse than 50-50 W/L due to sucky teammates.

Also here is more math for you Math pleasure: (from another post)


I'll try for a clear conclusion. It's difficult, because I need to (a) conclude my own analysis and (b) encourage other people to post theirs.

Frankly, I'm not very impressed by the logic you present. First of all, Elo is by no means necessarily a normal distribution. Second, I am uncertain where you get your 'number of players ranked', as the number of players above 1200 in solo queue is ~62,250 (http://www.leagueoflegends.com/ladders/solo-5x5?page=2490), the total number of ranked players on ladders is 79,850 (http://www.leagueoflegends.com/ladders), and the total number of solo queue players is not displayed. Third, you do not take Elo deflation into account; 1200 is no longer the mean, so using it as the basis for a mirror image does not make sense. Fourth, and most importantly, Elo Hell doesn't necessarily have much to do with absolute Elo; if you're 1040 Elo and should be 1200 Elo, you're in Elo Hell or close to it.

Just because the Dunning-Kruger effect (http://en.wikipedia.org/wiki/Dunning-kruger_effect) exists does not mean that nobody is actually in Elo Hell. And not everyone who claims to be in Elo Hell thinks they should be 1500 Elo.

Quote:
vonnlol:
It is Riot's fault that they picked a system meant for 1v1 situations and chose to apply it to 5v5,.
It is Riot's fault that they choose to ignore the fact that you cannot carry a bad team in LoL.
It is Riot's fault that they do not respond to Elo Hell threads or seem to care about it at all.
It is Riot's fault that they aren't bothered by the average Elo right now.
It is Riot's fault that they troll 50% of the constructive threads on the forums.

It is not Riot's fault that this system is bad for LoL, it is their fault they chose it.


You're not trying to further the investigation or fix things. As far as I can tell, you're just here to blame Riot. You can do that on another thread, and I'll gladly respond, but don't do it here.

Only one point is relevant: Is Elo an especially bad system for solo queue? The answer is, only if you think judging by wins and losses is a bad system for solo queue. Any system that does that will be subject to the same problems I described in the OP. If you can think of a good way to incorporate individual statistics into the ranking system, I'd be happy to hear more. But be careful of massive complications and potential abuse of the system.


Comment below rating threshold, click here to show it.

MathMage

Senior Member

12-08-2010

Quote:
Ellomdian:
I respect your initiative and theory. A Lot.

I feel that the issue is not actually one's ELO ranking - e-peen be ****ed. It is the likelihood of encountering one of your outliers (feeder/leaver/Terribad player) in "ELO hell"

I don't care if I play against or with players slightly below or above my skill level. I care when a single player is disruptive enough to pre-determine the outcome of the game. I would have no issue spending all my time in 9xx ELO if there were no leavers or intentional feeders.

While I feel your theory is solid, you by default have to dismiss outliers because without that data from RIOT (and for the record, if someone doesn't get a daily *(HOURLY?)* report of recorded leavers in a particular elo bracket, they should be,) you model is impractical. Odds are (THEORY TO FOLLOW) people in lower brackets tend to leave more games, therefor poisoning your model.


This is definitely true. One of the factors my model hasn't accounted for is players who have a disproportionate influence on the game, whether by leaving or by carrying. I think ultimately simulation, rather than probability theory, will provide the best model of the current Elo system--in part because it's easier to represent such influences via simulation.


Comment below rating threshold, click here to show it.

EasymodeX

Senior Member

12-08-2010

Quote:
If you can think of a good way to incorporate individual statistics into the ranking system, I'd be happy to hear more.


The game should assess your performance versus that of your team, and if your performance is inverse of the result of the game, your elo adjustment is diminished.

For example, your team wins, you win 20 elo. Oh wait, you're 1-10-1, and the average KDA of your team looks like 5-5-5. Your +20 elo is diminished to +10.

In reverse, your team loses, you went 34-5-4. Your team's average went 2-14-4. Your -20 elo loss is diminished to -10.

Your team had a player with <80% connection time, your -X elo loss becomes -X/2.

Your team had a player at < (average_team_level / 2, your -X elo loss becomes -X/2.

At the end of the day, Win/Loss should dictate your basic elo adjustment -- this is a team game, and the goal is for 5 to win versus 5. Individual performance is strictly a secondary consideration. However, currently individual performance is not directly factored into the elo adjustment at all (or is it? Riot only knows).

Edit:

The only cases where your individual performance has a disproportionate impact on the result of win or lose is when the individual feeds/fails like whoa, or afks, or leaves, or simply d/cs.

Super-carrying in LoL is non-existent in the general sense of what you could pull of in DotA. You have to be seriously smurfing to make the case where YOUR over-capability carried the team to a victory (and not the opponent's incompetence).

E.g., getting fed on Master Yi does not mean your skill carried the game. It's more likely that the enemy team's incompetent enabled you to win the game.


Comment below rating threshold, click here to show it.

Pete the Puma

Senior Member

12-08-2010

Quote:
MathMage:

Frankly, I'm not very impressed by the logic you present. First of all, Elo is by no means necessarily a normal distribution. Second, I am uncertain where you get your 'number of players ranked', as the number of players above 1200 in solo queue is ~62,250 (http://www.leagueoflegends.com/ladders/solo-5x5?page=2490), the total number of ranked players on ladders is 79,850 (http://www.leagueoflegends.com/ladders), and the total number of solo queue players is not displayed. Third, you do not take Elo deflation into account; 1200 is no longer the mean, so using it as the basis for a mirror image does not make sense. Fourth, and most importantly, Elo Hell doesn't necessarily have much to do with absolute Elo; if you're 1040 Elo and should be 1200 Elo, you're in Elo Hell or close to it.

Just because the Dunning-Kruger effect (http://en.wikipedia.org/wiki/Dunning-kruger_effect) exists does not mean that nobody is actually in Elo Hell. And not everyone who claims to be in Elo Hell thinks they should be 1500 Elo.





Elo will definitely follow a normal distribution and that was proven at some point by a guy who posted graphs here of the "positive" part of the curve (Elo >1200 by number of players) wish I could search for his post.

Number of players ranked was found at that date by taking number of players ranked 1200 and above for solo queue (49305 if I remember) and multiplying by 2 (98610 if I remember). Not perfect but certainly a valid approximation. Those number have definetly changed by now but I would believe the STD to stay about the same.

You are correct that I did not take Elo deflation into account but I also ignored Elo inflation (from loss forgiveness and the probable bias of more <1200 Elo players leaving the game than >1200 players), as mentioned in the original post. Although you are probably right that because Elo deflation> Elo inflation the mean is probably slightly lower than 1200 it can not be very far.

We do have slightly different definitions of Elo hell, where your definiton is someone who's Elo does not reflect their true skill whereas I'm refering to all the people who blankly refer to Elo 1100-1300 as "Elo Hell". I for one dont believe in "Elo Hell" and think: "L'enfer c'est les autres" definitely applies here.

Off to searching for those graphs of number of players/Elo.

P.


Comment below rating threshold, click here to show it.

Wankeroo

Senior Member

12-08-2010

ELO HELL: Constantly dealing with new players. The new players introduced into the ranking system constantly pour into the games of people who are anywhere between 1100 and 1400 rank.

The problem is the ranking system is not a "ladder" where there are low-skilled players at the bottom and high skilled players at the top and 50% win ratio players like 2/3 of the way up the ladder where they will be matched with other players of their ilk and they will not constantly be dealing with new players in their matches.