Which matchmaking 'issue' is the most important to you? (See post for more details!)

1) AFKs in Champion Select Lobby 4,875 36.22%
2) Duo-Queue Elo Disparities in Ranked 1,008 7.49%
3) Skilled Ranked Players in Normal Modes 666 4.95%
4) Premade Matching 672 4.99%
5) Transitioning from Normal to Ranked Mode 1,347 10.01%
6) Free to Play Champions in Ranked Mode 802 5.96%
7) Random Champions in Ranked Mode 647 4.81%
8) Provisional Matches in Ranked 723 5.37%
9) Duo Queue Prevalence in Ranked 422 3.14%
10) Level Disparities 652 4.84%
11) Team Margin of Victory 1,645 12.22%
Voters: 13459. You may not vote on this poll

After Hours with Matchmaking and Lyte

First Riot Post
Comment below rating threshold, click here to show it.

Purgation

Senior Member

09-29-2012

Quote:
Originally Posted by Lyte View Post
I've always said that for a small (<1%) of our playerbase, it will take thousands of games to reach their "true Elo." This is due to a number of factors such as noise, statistical probabilities not falling in their favor, etc. Does this mean those players are in Elo Hell? Possibly. For the average player though, it takes about 200-300 games. We do agree that 200-300 games is too long to reach their true Elo, so we're brainstorming ways to reduce this time-to-true-Elo dramatically.

As you brainstorm, please look at the math of your team ELO system considering two factors:

1. How you estimate and distribute ELO changers to teams
2. Significant churn in the population, particularly large influxes of new players and significant player drop-off at lower ELOs.

In tek speak- you might find that convergence is even slower or never happens in the zone between 1200 and some floor because the influence of newer players with highly unstable ELO ratings in that part of the ladder.

In LoL English - ELO hell is real.


Comment below rating threshold, click here to show it.

Purgation

Senior Member

09-29-2012

Quote:
Originally Posted by Yegg View Post
Goumindong is correct. We ran this analysis awhile back and players who are new to ranked finish their provisional matches with an average rating that's very close to 1200 (and it included some players with sub-500 ratings because of heavy dodging). Now that players don't lose Elo from dodging I expect it would be even closer.
I appreciate your reply, but I think it is the wrong analysis. The correct analysis would be the ELO distribution of players (or we should really say "accounts") after say, 100 games.


Comment below rating threshold, click here to show it.

Purgation

Senior Member

09-29-2012

Quote:
Originally Posted by Goumindong View Post
The perception of ELO hell is almost assuredly some form of observation bias.
What this means is that there are a large number of factors which effect most people that create the perception of ELO hell when ELO hell doesn't actually exist
As it turns out, I am well aware of these issues. This is why I don't post (or pay any attention to those who do try to prove by anecdote)

It also turns out that I am pretty good with the mathematics, and the implementation of ELO in LoL has some poor characteristics.

Finally, the real problem is a terrible player experience. Maybe I suck - - that's just fine, so long as I am playing games with players of a similar skill level. When I am playing a huge proportion of games where either a single player on EITHER team is either so weak or so strong that the entire game is defined by them, it just isn't fun.


Comment below rating threshold, click here to show it.

ThirdTimeLucky

Junior Member

09-29-2012

I greatly appreciate and enjoyed your thoughtful response! Thank you!

Quote:
Originally Posted by Goumindong View Post
Yes and no. In that at any particular point things may not be perfect, but that they are always moving towards it. This isn't a problematic issue and exists in all rating systems which continually add people because adding people adds uncertainty, at the very least, to those people who recently enter. Less so to others if the population is large, which it is.

We can see how it adds less uncertainty by pretending that your ranking wasn't a number, but rather a partial order with each person lined up in order of ranking. If there are three people in line and one person enters and then find their proper spot, the total order has moved significantly relative to the middle point. The middle person, who was previously ranked in the middle, is no ranked either the second or third quartile. If three hundred people are in line the total order has moved very little relative to the middle point.

Since ratings are more or less enforced to average to 1200 adding new people doesn't do much to increase uncertainty in your ELO.

That is to say that new entrants are only problematic in that they consistently make it more difficult to predict the games which have the new entrants.
And if the number of "new" entrants are a sufficiently large relative to the active base, particularly in certain ranges, it becomes a big problem. Given slow convergence properties (estimated 200 games on average) that's a very big problem.

Quote:
Originally Posted by Goumindong View Post
ELO, like true skill, is a heuristic to achieve Bayesian updating. ELO is just a common name for it. There are limits to how fast we can become more certain of a value with Bayesian updating which is a function of the amount of information contained in the newly added data points. Less information relative to the amount of information currently possessed means slower updating. Adding players to a game reduces the amount of information that we have available on the skill level of players. Such there is no system which "converges well" for multiplayer games where the players are not static. This is not a function of the ranking system you use, it is a function of the information contained in the data points.
Fair enough, but there are certainly ways to make the problem worse - implementation details matter.

The exact method used to allocate ELO gain or loss to individual players could matter a great deal.

Think about it this way - let's assume that we created a model to predict the impact each player of higher and lower skills scores would have on a game. (and we need such a model for accurate matchmaking) - - will the model simply be the average of the scores? I submit that it is far more likely to have meaningful features - - e.g. an over-performing low score or a under-performing high score have a much larger impact on the modeled probability - - and as a result allocating ELO gain or loss irrespective of this fact is introducing more noise into an already noisy system.

Quote:
Originally Posted by Goumindong View Post

Many people think that because the original ELO system did not have a system to measure uncertainty that it is not a heuristic bayesian updater, which is wrong, it simply requires less computations and few more assumptions. It also is not likely as swift in updating as others.
Which is in fact, a major problem.

Quote:
Originally Posted by Goumindong View Post
There is one interesting thing that TrueSkill does, but its not actually pertinent to how it matches [well, it may be, but probably not]. In that it lists your skill not as the mean, but as the mean minus three standard deviations. I.E. Your "True Skill" rating is the systems "I am 99% confident you're better than this" point.
Quote:
Originally Posted by Goumindong View Post
Actually in League "new" players to ranked are pretty average. Its one of the advantages of having to play 200+ games and own 16 champions before starting ranked. There are no "newbies"
I will admit that I find this a bit difficult to believe - - although the qualification of "new to ranked" makes it far more plausible. As mentioned elsewhere, I'm not sure that average ELO after provisional is the right statistic.

Look at it this way - - either a) you might as well stop playing after 200 games, because you aren't going to get any better and the quality of games you experience will not change or b) experience DOES matter even after 200 total games and failing to consider this factor in matchmaking will lead to poor matchmaking.

Quote:
Originally Posted by Goumindong View Post
If newer players experience an unfair downdraft then they're more skilled than their teammates and the probability of them winning matches increases. ELO hell does not exist. More trolls and AFK's will exist on the other team, creating a net advantage for the player who is legitimately more skilled than his ranking suggests
Think about flows. Use your own example of the rank ordering. If there is a constant flow of players into the middle of the distribution, and you are both below that middle, but you should be moving up... the constant flow into the middle above you will prevent you from moving above the midpoint.

This wouldn't matter if those flowing in truly were distributed the same as those already in the population... but that would be truly remarkable... a game with no experience curve.

Now, if the total population was relatively static, this might still work out... as the accumulation of players below our hypothetical person in ELO hell would eventually start to "push" them up... but in LoL it is possible and likely that below a certain threshold players simply drop out of the game - which makes the flow of players from the midpoint downward the dominant feature.

Quote:
Originally Posted by Goumindong View Post
No. The probability of seeing a 14 game losing streak is not .0006%. The probability of seeing a 14 game losing stream in any particular 14 games assuming that the probability of winning the game is stable at 50% is about .0006%. But... and this is a big but. The probability of seeing at least one 14 game losing streak in a large number of ranked games, making the same stable win probability assumption is actually quite high. I don't have numbers since IIRC doing this requires simulation and I am lazy. But I can tell you that the probability of seeing 9 heads or 9 tails in a row if you flip a coin 1000 times is about 80%, 10 in a row is about 60%.

Since large streaks can "feed" on themselves due to poor play from psychological factors its not really a surprise for this to be seen in "large" numbers.

It would be interesting to know if large streaks were common or uncommon [i suspect large streaks of losses are more common than large streaks of wins] but I am not sure that knowing that we see large streaks of losses more often than large streaks of wins tells us anything particularly interesting about our match making system.
You are right, of course. I don't get to see the data, and streaks are not really the most important feature... but it's one of the few features I can see that hints at whether or not matchmaking is actually working.

In the end, it is encouraging that Riot is looking at these issues seriously. As I have said before, it's really the experience of it that is important to me. I don't have any illusion that some change in matchmaking is going to suddenly vault my ELO.

What I do hope for is something (anything!) that allows for more actually challenging and competitive games, where one player on EITHER team doesn't have such a disproportionate impact on the outcome of the game. (And unfortunately, despite what some from Riot have suggested, my experience is that the impact of Feed >>> Carry, not the reverse)

Ironically, I actually put a beginning of the game AFK well below a severely outclassed player in terms of their decisiveness to a team

ELO hell does exist, it is an experience that your players are having of feeling that the quality of their own play doesn't matter in games, and in lower ELOs that particularly blatant and egregious behavior (e.g. afks) is the main factor.

Maybe it's just a psychological artifact of people not being able to perceive how their play increased the probability of wining by 3% (I estimate that even in a best case scenario, a player who for whatever reason is at the "wrong" ELO level can expect at best to experience an excess win rate of 55% or so - - in other words in 20 games they will lose win 11. )

Or, maybe, there is something about either the structure of the game itself, or the matchmaking and ELO system that causes individual players to have a greatly disproportionate impact on a game's outcome.

Either way, it is a miserable gaming experience... and causes players to be frustrated, leading in turn to raging and trolling and other toxic behaviors.

Sounds like hell to me.


Comment below rating threshold, click here to show it.

ncp5000

Junior Member

10-07-2012

This account is an account I leveled with my wife when I was getting her to play League of Legends. My main account is DirtyNate which is plat ranked. I was thinking about getting this account up to plat too, but right now leveling is a pain simply because my queue times are getting ridiculous.

I understand there are not many plat ranked players in the 25-29 range. But if I recall correctly there was a recent change to matchmaking so that if you weren't lvl 30 you hardly ever get queued with lvl 30s. Which is fine for the majority of people, but is there any way that I can opt in to playing with lvl 30s so that my queue times come down from 15 min? Queue times of 10-15 minutes just aren't fun when you're trying to level an account.


Comment below rating threshold, click here to show it.

Asko

Senior Member

10-08-2012

Hi Lyte,

how is the point
2) Skilled Ranked Players in Normal Modes [7/10/2012]
advancing?

I just had a game on the EUW servers in which the elos of the teams were

1751
1535
1874
unranked
1566 (me)

vs the enemy team consist of:

2210
2043
1468
1506
1902

in ranked soloQ elo.

This kind of games just suck the will to play the game out of one, sorry to say. Out of 5 matches I play a day 2-3 have similar elo differences and the result of the game most often reflects that.
Are you planning to make normal elo correlate with top ranked elo? for me that seems to be the most obvious and simplest solution.

cheers,
Asko


Comment below rating threshold, click here to show it.

CupcakeTrap

This user has referred a friend to League of Legends, click for more information

Senior Member

10-08-2012

Lyte, this might sound like a very dumb question, but are you noticing a lot of ... shake-up ... lately?

I feel like I've run into a lot of people who have found their usual Elo bracket feels a lot different recently.

Are people surging into Ranked at higher rates as the Season draws to a close?


Comment below rating threshold, click here to show it.

Goumindong

This user has referred a friend to League of Legends, click for more information

Senior Member

10-08-2012

Quote:
Originally Posted by CupcakeTrap View Post
Lyte, this might sound like a very dumb question, but are you noticing a lot of ... shake-up ... lately?

I feel like I've run into a lot of people who have found their usual Elo bracket feels a lot different recently.

Are people surging into Ranked at higher rates as the Season draws to a close?
Likely in order to give a go at the 1500 ranked reward [Janna Skin]


Comment below rating threshold, click here to show it.

Goumindong

This user has referred a friend to League of Legends, click for more information

Senior Member

10-08-2012

Quote:
Originally Posted by Purgation View Post
I believe that ELO hell is real, not because I think I'm great... or even above average... but because of the huge skill disparities I see in many games. I am not looking for a fix because I think it will "prove" I'm a better player - - I'm looking for a fix because I'm tired of games where one player on either team (either because they are so unskilled OR too skilled) basically makes the rest of the fairly even match-ups completely irrelevant to the outcome of the game.
A lot of the skill disparity in League is due to the snowbally nature of the game.

Here is an anecdote from a recent game of mine, where i was destroyed as Kennen top against Jax. I made a misplay early and was killed by a mid eve counter gank, which allowed jax to farm the lane and develop a level advantage over me... which got me killed again.

After that, i could not stay in lane(so i went elsewhere). I wasn't less skilled than him(indeed, i was able to see what had happened and respond to the situation optimally when he did not realize what had happened, maybe then I was a better player than him?). I knew why I was losing and what happened. But at that point in the game it sure looked like i was "just bad" if you looked at the score and didn't understand how the game progressed to that point.

The next game I played the opposite happened. I was playing support as Sona and Ezreal and I walked all over their Sivir and Maokai, After the first engage they could not really fight us in lane and we were both geared to fight in lane. Were their Sivir/Maokai "just bad" or did a high harass lane just snowball? (Hint: It was probably the second)


Quote:
Originally Posted by ThirdTimeLucky View Post
Think about it this way - let's assume that we created a model to predict the impact each player of higher and lower skills scores would have on a game. (and we need such a model for accurate matchmaking) - - will the model simply be the average of the scores? I submit that it is far more likely to have meaningful features - - e.g. an over-performing low score or a under-performing high score have a much larger impact on the modeled probability - - and as a result allocating ELO gain or loss irrespective of this fact is introducing more noise into an already noisy system.
How do you know Riot doesn't do this?


Quote:
Which is in fact, a major problem.
The short answer is "No". The long answer is that confidence levels create just as many problems as they solve, and can increase the time it takes to converge. The reason for this is simple, there must be a "minimum confidence" or the system cannot account for skill. That is, the system will get so confident in your current skill that any win won't change your ELO. So if you're ranked 1200 with 1000 games and go play 100 games of normal and get better you would come back to 1200 and cream people and your ELO wouldn't go up, because the system cannot distinguish between the last win and the first loss.

Once you get to that minimum confidence a system with confidence levels behaves exactly like a system without confidence levels. Optimizing converge requires a balance between allowing for skill changes and reducing the amount of ELO change per loss.

Quote:
Look at it this way - - either a) you might as well stop playing after 200 games, because you aren't going to get any better and the quality of games you experience will not change or b) experience DOES matter even after 200 total games and failing to consider this factor in matchmaking will lead to poor matchmaking.
Doesn't matter so long as the change is close enough. There is also significant empirical evidence for this [we can see that the center of the ELO distribution is 1200, and we also know that if new entrants have the same shape of the distribution then the distribution will center at the average ELO of the new entrants]


Quote:
Think about flows. Use your own example of the rank ordering. If there is a constant flow of players into the middle of the distribution, and you are both below that middle, but you should be moving up... the constant flow into the middle above you will prevent you from moving above the midpoint.
This is now how orders work. If X>Y and Y>Z then X>Z. You do not have to play everyone above you in order to move over them in the ranking. This means that your actual ranking will only change in relationship to new players if you're actually worse than those new players.


Comment below rating threshold, click here to show it.

Maxmike

Member

10-11-2012

Dear Lyte,

I created this account to help introduce friends to the game in early season 1, it sat around level 10-14 for a while. One of the friends I introduced advanced to level 30 and I dedicated this account to duo queues with him.

So, this account has ONLY EVER played ranked with Kodoscopy in duo queue. Every single game we queue together.

Weird things:

1) There is 20 elo difference between us even though we both hit level 30 in the same couple days and we only ranked queue in the same games together. Our win/loss ratio is identical. Why the disparity?! (Currently 1760 elo vs 1740 elo.) The only difference is that his account was created in Season 2 and mine in Season 1 and our normal game history is not identical.

http://www.lolking.net/summoner/na/19954989 - Maxmike
http://www.lolking.net/summoner/na/31371273 - Kodoscopy

2) There is a DISPROPORTIONATE number of times where we are last pick and second to last pick. Our Elo is very similar and so we are typically right next to each-other, but since at least 1450 elo it seems like 9 out of 10 games I am second last and he is last pick. It is extremely rare, but can happen such that I am third pick and he is fifth, and maybe one time in the past 30 games he was not last pick (there was one person under him.) Why? I would expect occasionally (1/5 games) get first pick, but this is never the case. This is a long standing trend.

3) Everyone in every game is really heavy. We have to carry very hard. Give us better teammates and enemies please. :B