Which matchmaking 'issue' is the most important to you? (See post for more details!)

1) AFKs in Champion Select Lobby 4,866 36.22%
2) Duo-Queue Elo Disparities in Ranked 1,005 7.48%
3) Skilled Ranked Players in Normal Modes 665 4.95%
4) Premade Matching 670 4.99%
5) Transitioning from Normal to Ranked Mode 1,345 10.01%
6) Free to Play Champions in Ranked Mode 802 5.97%
7) Random Champions in Ranked Mode 646 4.81%
8) Provisional Matches in Ranked 723 5.38%
9) Duo Queue Prevalence in Ranked 420 3.13%
10) Level Disparities 649 4.83%
11) Team Margin of Victory 1,643 12.23%
Voters: 13434. You may not vote on this poll

After Hours with Matchmaking and Lyte

First Riot Post
Comment below rating threshold, click here to show it.

Dr Clueless PhD

This user has referred a friend to League of Legends, click for more information

Senior Member

09-04-2012

Quote:
Originally Posted by Lyte View Post
Keep an eye out for threads that announce new Team PB&J features. We typically have the team members that worked on the feature do Q&A and introduce themselves to the community.

In the past couple weeks we've been focused on releasing Tribunal features, so awesome members like RiotAaronMike, RiotNinjaTabby and VonBurgermeister have been presenting work like this: http://na.leagueoflegends.com/board/....php?t=2380577
Completely related: How long are you intending to collect data with the reform cards before working on the analysis (and presumably modifying it or removing it)?

I'm curious, because I'm VERY eager to see what sort of effect, if any, the cards are having (though I won't have a chance to see the data myself, of course).


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

09-04-2012
351 of 362 Riot Posts

Quote:
Originally Posted by Imclueless View Post
Completely related: How long are you intending to collect data with the reform cards before working on the analysis (and presumably modifying it or removing it)?

I'm curious, because I'm VERY eager to see what sort of effect, if any, the cards are having (though I won't have a chance to see the data myself, of course).
It's hard to know the exact time to wait since this is the first experiment of its kind. There's some weird issues like when a player is banned for 2 weeks, we need to wait 2 weeks before we can even start measuring the data for that player. When we actually start collecting data for a player, I'd want to see at least 6 weeks of data to see whether their behavior trends for the better or back towards the Tribunal.

I'm just as excited about the data as you are


Comment below rating threshold, click here to show it.

TheDjinni

Senior Member

09-04-2012

(Most of this post is directed at Lyte, who I hope gives this a look, even though I'm responding to someone else).

Quote:
Originally Posted by isobold View Post
We had so many Elo-Hell-experiments, it's not even funny anymore. No real good player got ever stuck in Elo-Hell, which in return means: if you get stuck there, it's because you don't belong onto a higher Elo.
No, that doesn't follow at all. Just because a 1800 elo player can go from 300 elo to 1800 elo doesn't mean a 1000 elo player can go from 300 to 1000 elo.

And just because one person can do it does not mean others can.

Look, there's two ways to formulate the elo hell problem. The first, which I'm about to explain and justify, is I think a serious issue. The second is a perceived issue which may not actually be one.

The First Problem

A lot of complaints about elo hell start with "I should be a higher elo, but I can't get there". The basis of the counter-argument against it goes something like "well if you should be there then you should win more games than you lose, but because you're not you deserve to be exactly where you are".

This response is flawed, and I'll explain why.

Consider the following definition for elo hell: for a particular player, their elo hell is an elo level at which luck in matchmaking causes their actual win ratio to drop below 50% when their expected elo would suggest that their win ratio should be above 50%. I think that based on this definition, at every elo there is by necessity some small subset of players who should be at a higher elo but can't get there; the question is not whether elo hell exists but how many players it affects and how seriously it affects them.

Obviously if a player is "unlucky" and gets repeatedly matched with trolls and afkers and (actual) intentional feeders (ignore for a moment the idea of being matched with bad players) on their team, they will by necessity win a bit less than they should be and may even go down in rating. It is entirely possible that luck with matchmaking brings their actual success rate below 50% even though it should be above 50%, especially if they're only 100-200 elo away from their "expected elo".

For example, if a player is expected to win 52% of their games given his current elo (say he voluntarily dropped it by 100 points or whatever it would take to have that kind of ratio), but in 8% of his games there's a troll on his team (and he automatically loses these games for the sake of argument) and in only 2% of his games there's a troll on the opposing team (which he automatically wins for the sake of argument), then his expected win ratio drops to 49% and thus despite his elo suggesting he should slowly be moving up he actually ends up on average moving down. This person is, through no fault of his own, stuck 100 less elo then he is expected or supposed to be just based on luck.

You Should Have Less Trolls on Your Team?

Now, the typical response to this is that you will have less trolls on your team then there are on your opponents team (assuming you are never a troll yourself), and therefore even if you're matched with trolls on your team there should be more trolls on your opponents team on average. This is actually not necessarily true for all players.

It's true on average, but perhaps some players are matched with more trolls on their team than other players. I would expect to see a normal distribution of trolls for each player: some players get a bit more than average on their team, some players get a bit less than average, but most players get pretty close to the average. Meaning it could be entirely possible that some small (granted, perhaps even miniscule) subset of players who are definitely not trolls and have never been trolls have just unfortunately been matched with many more trolls on their team than on the opposing team. But no matter how small this subset is, it is entirely plausible for it to exist and they could have a valid complaint: they're just unlucky.

I'd like to emphasize this point: the average is not indicative of what actually happens with "every individual player". Saying you on average you should have more trolls on your opponents team does not mean that after say 200 games played it will happen. It might take an individual 2000 games, or it may never actually happen.

But Wait, There's More

Now, there's three more things to consider in relation to your assertion that elo hell doesn't exist because a couple of high elo players weren't stuck in it: at lower elos there is many more trolls than at higher elos (just based on the fact that if you troll, afk, leave, and feed intentionally a lot you tend to drop in elo quickly). Plus, high elo players will have much higher expected win rates than mid elo players when in low elo settings (plus, they can more easily carry the trolls).

So consider the following scenario; let's say you take a high elo player and stick him in a very low elo, where he would expect to win 90% of the time purely based on skill. Let's say further still that this high elo player is very unlucky: he sees a troll in 25% of his games (because he's such low elo), and in 20% of all games it's a troll on his team that causes him to automatically lose, and in 5% of all games it's a troll on the other team and he automatically wins. Given these numbers he should win 72.5% of the time despite this horrid luck and quickly escape this terribly low elo.

Suppose further still that as he goes up, his expected win ratio goes down to say 70% of the time, but the number of trolls on he sees goes down to say 15% of all games. With these new values his win ratio is 62.5%, so he keeps moving up. As you can see, it's entirely possible for a really good player to overcome bad luck and get out of a low elo.

Suppose that you take another player, under identical circumstances and identical luck, except he's not nearly as good; he should win only 65% of games when he's dropped into the lower bracket where he meets trolls in 25% of all games. In this scenario he has an expected win ratio of 53.75%. As he goes up, his win rate drops to 55% and the troll rate is 15%, meaning his actual win rate is just below 50% and he gets "stuck" here.

But there's more: remember when I asked you to forget claims of bad (not necessarily troll) teammates holding a player back for a moment? Well, now I want you to consider them. Yes, I know people are biased to rate their own performance higher than others. But your elo is an estimation, and just because you're matched with people who are of similar elo to you doesn't mean that they're going to be identically skilled. It is entirely possible that (just as some players get more trolls than other), some players are unlucky and get more bad (for their elo) players than others. Obviously actually measuring such a thing would be nigh impossible, but it's still plausible. These people have a very valid complaint with the elo system and do not deserve to be dismissed on the grounds that their claim is impossible.

Where to Go From Here

So I think a better discussion is this: how serious is the elo hell problem and how many people does it affect?

If it only affects players by about 50 elo and it only affects less than 0.01% of all players, then it's probably not a big issue. That said, acknowledging that it does affect players is important.

First of all, how would you figure out what impact it has?

You could start by gathering data for several elos (say 300 to 1700 in 100 elo increments) on the average number of trolls that caused a loss per game in that elo. And by trolls, that would include all games with a leaver, and all game with a player actually suspended in the Tribunal for intentionally feeding (plus any players with similar scores metrics who weren't reported, for example), and so on and so forth. Once you have an average, you may find a correlation with elo to estimate out the average for all elos in between.

Then you could try getting some data on the distribution of trolls across the entire population for each elo and see if there are any correlations there, too. Maybe the lower elo means the standard deviation from the mean is higher, for example.

Once you have that you can determine what percentage of the population is at a significantly lower elo (say 100 or more) than they should be because of trolls.

The Second Problem

There's a second Elo Hell problem, and it goes something like this:

"A bunch of players at 1100-1300 all suck and I know I could be a higher elo but they all hold me back because I keep getting bad players on my team".

90% of Elo Hell complaints are around that elo range (yeah, I pulled that out of my rear, but whatever). Why? I think the answer is obvious: there's a lot of noise at that elo range, a high variance of skill levels and trolls. I think that at and around that elo range, luck plays an even greater factor: who you get matched with/against and how this affects your elo. I wouldn't be surprised to see the greatest number of complaints coming from players around that range who think they're pretty good but keep getting matched with players who just hit level 30 and simply don't know how to do simple things like last hitting or don't understand how bad leaving the game is.

I think that if you can solve the problem of the noise in that elo range (which you have acknowledged is an issue in this thread) you can greatly decrease the number of elo hell complaints.

Conclusion

Look, I'm not saying that all or even most complaints about elo hell are valid. All I'm saying is that elo hell could be an issue for some players. I have no idea whether it is the all-encompassing problem that people assert it is; I am only pointing out that it is entirely logical to expect it to exist at some level and affect some people negatively.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

09-04-2012
352 of 362 Riot Posts

Quote:
Originally Posted by TheDjinni View Post
Conclusion

Look, I'm not saying that all or even most complaints about elo hell are valid. All I'm saying is that elo hell could be an issue for some players. I have no idea whether it is the all-encompassing problem that people assert it is; I am only pointing out that it is entirely logical to expect it to exist at some level and affect some people negatively.
I've always said that for a small (<1%) of our playerbase, it will take thousands of games to reach their "true Elo." This is due to a number of factors such as noise, statistical probabilities not falling in their favor, etc. Does this mean those players are in Elo Hell? Possibly. For the average player though, it takes about 200-300 games. We do agree that 200-300 games is too long to reach their true Elo, so we're brainstorming ways to reduce this time-to-true-Elo dramatically.


Comment below rating threshold, click here to show it.

Claw of Time

Member

09-04-2012

Hi Lyte.



How do you do?


Comment below rating threshold, click here to show it.

Cyrad

Senior Member

09-04-2012

Thank you so much, Lyte.
1) Duo-Queue Elo Disparities in Ranked is an issue I had to deal with. I only played 11 ranked games, but all three of my losses were from duo-queueing with someone who had no Elo.


Comment below rating threshold, click here to show it.

DirtyNate

This user has referred a friend to League of Legends, click for more information

Senior Member

09-05-2012

Hey Lyte,
Just a question for you about retaliation. I just played a game as Malzahar where it was kind of close at first but we started snowballing in the last 1/4. Once the snowball was in full effect the enemy ziggs said "gg malz get carried". This kind of surprised me since I was 8/3 and had same CS as the ziggs, who was 6/6. So I replied "8/3?". To which he said "you got shat on in lane." Ziggs had killed me once in lane, but I had killed both him and hecarim in a 1v2 when they came for a gank attempt. So I said "you killed me, but I killed you."

Ziggs then proceeded to blame his death completely on hecarim, and kept talking about how bad I was. So after the game in the lobby. I said "lets look up your elo, lol 1477, too pro for me, peace loser". With the exception of the "peace loser" I don't think anything I said was too offensive. Bringing up elo was probably uncalled for.

But my question is how to deal with people that attack you completely unprovoked and continue to attack you? It only seems fair that if he can attack me personally without any provocation, that I should be able to defend myself. Is this justified or is my only option to /ignore and move on?

TL;DR Is defending yourself from attack ever justified?


Comment below rating threshold, click here to show it.

DirtyNate

This user has referred a friend to League of Legends, click for more information

Senior Member

09-05-2012

One more think, I've followed this thread avidly since you put it up. You're probably my favorite rioter now, since I think the only thing this game needs at this point, to be the best game ever, is simply a more civil player base.

Thanks for your work!


Comment below rating threshold, click here to show it.

thenewattacker

Senior Member

09-05-2012

there could be a test you have to complete before you can play ranked


Comment below rating threshold, click here to show it.

isobold

This user has referred a friend to League of Legends, click for more information

Senior Member

09-05-2012

Quote:
Originally Posted by TheDjinni View Post
No, that doesn't follow at all. Just because a 1800 elo player can go from 300 elo to 1800 elo doesn't mean a 1000 elo player can go from 300 to 1000 elo.
I've collected several such experiments where the player doing it was between 1250 and 1400 Elo before he did it. The "worst" player to ever do it I saw was 1250 Elo on his main account, dodged down for season 2 to 0 Elo and went up again in less then 200 games to 1244 Elo again, where he stopped his experiment.

So we have statistical evidence that it even works for the kind of player you are talking about. I just use those experiments to compliment the mathematical prove why every player will eventually reach his Elo: http://lol.noamik.de/No-Elo-Hell.png (this is the dumped down version for school kids why Elo is working).

So while you are right that there could be a chance that this isn't following we have now:
- mathematical prove that Elo-Hell can't exist
- statistical evidence that it doesn't exist (I refer here to the Elo-Graph-Tool I wrote one year and a half ago that has been copied since by various sites like LoL-King)
- experimental evidence (every Elo Hell experiment so far succeeded)

I see very limited room for Elo Hell to actually exist facing this.

Lyte is very polite in his responses not claiming what one can't know for sure. But I'm not a mathematician nor a cognitive science guy, I'm an engineer. When I got a good model for a system, that is proven to work through statistics AND experiments, I won't go for this 99%-stuff Lyte uses. I will just assume the model to be good and to reflect reality. And the model says: Elo-Hell can't exist.


Quote:
Originally Posted by TheDjinni View Post
And just because one person can do it does not mean others can.
You would then have to argue, why a person that can't would actually deserve a better rating? How would you measure a persons "True-Elo" to prove that his actual Elo doesn't match his skill (after say 300 games his recent improvement in gameplay not included) ...


Quote:
Originally Posted by TheDjinni View Post
Obviously if a player is "unlucky" and gets repeatedly matched with trolls and afkers and (actual) intentional feeders (ignore for a moment the idea of being matched with bad players) on their team, they will by necessity win a bit less than they should be and may even go down in rating. It is entirely possible that luck with matchmaking brings their actual success rate below 50% even though it should be above 50%, especially if they're only 100-200 elo away from their "expected elo".
You say: "only". Do you have slight idea of how much difference 200 Elo actually make? If you are 200 Elo from your "expected elo", you are 350 Elo away from the mean Elo you should have, since your "expected elo" is an actual range of +/-150 Elo. That's huge. If you are 350 Elo away from where you belong, you should easily win 6-7 out of 10 games. Sure you can have streaks of bad luck. But the more you play, the more unlikely those streaks become, all the while games are becoming easier and easier to win.

Just to give you an impression: when I'm 300 Elo below my "deserved rating", I will win a 4vs5 in more than 50% of all games. So not only streaks are becoming less likely, their impact has to be more and more negative to bring your win-rate below 50%.


Quote:
Originally Posted by TheDjinni View Post
For example, if a player is expected to win 52% of their games given his current elo (say he voluntarily dropped it by 100 points or whatever it would take to have that kind of ratio), but in 8% of his games there's a troll on his team (and he automatically loses these games for the sake of argument) and in only 2% of his games there's a troll on the opposing team (which he automatically wins for the sake of argument), then his expected win ratio drops to 49%
Now do the math first how likely this is to be happening. And for the sake of the argument, calculate the likelihood for 300 leavers. And then get back to the discussion and tell me, if your theoretic issue looks like a real issue to you ...


Quote:
Originally Posted by TheDjinni View Post
This person is, through no fault of his own, stuck 100 less elo then he is expected or supposed to be just based on luck.
I don't see the "stuck" part. This streak will end eventually and the player will raise easily afterwards. And even more important: you missed the part where the games were becoming easier and easier to win. So eventually the player will even hit a spot, were having the troll in his team won't mean auto-lose anymore. So even with a continuous streak, he won't be dropping any further, all the while his streak will end eventually.

There is only one way for such a streak to be eternal: when the player himself provokes trolling. In this case the troll isn't a matter of luck or bad-luck anymore, but a deserved "punishment" for his behavior. In such a case his Elo wouldn't tanked by trolls, but by his behavior. But that's a discussion about potential then. And potential doesn't matter unless you actually use it ...


Quote:
Originally Posted by TheDjinni View Post
It's true on average, but perhaps some players are matched with more trolls on their team than other players.
No, it's not only true on average. It's also true and mathematically guaranteed for each and every player in the long run. You may have streaks of bad luck, but by playing even more, the chances of a streak to persist get lower and lower.
To give you an impression: the chances of you to have 20 times in a row a troll in your team and not in the enemy team are: 0,000009%

The same is true for ANY external event that isn't influenced by your behavior (be it good or bad for your chances of winning). Which is exactly why GOOD players make you lose your matches (and thus Elo), while bad players make you win matches (and thus Elo).


Quote:
Originally Posted by TheDjinni View Post
I would expect to see a normal distribution of trolls for each player: some players get a bit more than average on their team, some players get a bit less than average, but most players get pretty close to the average.
Yes, this is actually correct, but look at the numbers posted above or do some calculations yourself. Start easy with 300 games with a troll, leaver or afk in each of them and calculate the odds to be hit more often by those than the enemy. And then come back and tell us how many members of the entire community you expect to be hit. You will be surprised by the answer ...


Quote:
Originally Posted by TheDjinni View Post
I'd like to emphasize this point: the average is not indicative of what actually happens with "every individual player".
Sure, but fortunately math not only give us the tools to calculate this average. We can also calculate the odds of outliers ...
Try it for yourself, you will be surprised.

Or as an engineer would put it: there is no such thing as an Elo-Hell. The odds of even one player being hit by such a thing are so minuscule, we can outright ignore them.

It's of no use to worry about events which are this unlikely. If you still do, I will implant you a even bigger fear (because I'm a sadist). As of now you shall fear to get smashed against your own ceiling by pure coincidence out of the blue. All it takes for this to happen is that all the atoms you are composed of suddenly swing into the same direction. This is entirely possible. But have you ever seen such an event (be it you, your mum or your desk to be smashed against the ceiling) or heared of anyone observing such an event?


Quote:
Originally Posted by TheDjinni View Post
As he goes up, his win rate drops to 55% and the troll rate is 15%, meaning his actual win rate is just below 50% and he gets "stuck" here.
You made so many assumptions to get here, you forgot to actually look into what happens in reality. I already posted the statistics of my own Elo Hell experiment. In Solo-Queue I'm swinging between 1300 and 1400 Elo, so I'm by no means high elo. Yet I had no trouble to win 66% of my games playing the roles I'm worst at using champs I had no experience with, when I started the experiment. Most of your assumptions are simply false (starting with trolls being auto-lose, or high-elos being able to carry trolls in only 25% of all games) thus making the numbers you put up worthless.

Also more trolls in low elo means more chances to have trolls in the enemy team, rendering bad-luck-streaks even more unlikely ...


Quote:
Originally Posted by TheDjinni View Post
It is entirely possible that (just as some players get more trolls than other), some players are unlucky and get more bad (for their elo) players than others. Obviously actually measuring such a thing would be nigh impossible, but it's still plausible. These people have a very valid complaint with the elo system and do not deserve to be dismissed on the grounds that their claim is impossible.
Again, do the math and come back when you're done with those. What you say is entirely valid for this one game the player might play. And they can't be put aside, if you look at 10 games. Now look at 300 games, calculate the odds, you will be surprised ...


Quote:
Originally Posted by TheDjinni View Post
So I think a better discussion is this: how serious is the elo hell problem and how many people does it affect?

If it only affects players by about 50 elo and it only affects less than 0.01% of all players, then it's probably not a big issue. That said, acknowledging that it does affect players is important.
The cool thing is: we can actually do the math and find out. I did, and I wrote an entire guide about it. Feel free to do the math on your own and make your own conclusion. I'll spoiler you: the odds of players to be more than 150 Elo away from their "mean Elo", thus out of their Elo range, for more than 300 games are so small, that even if LoL had 1.000 billion players, I wouldn't expect a single player to be affected. That's how "serious" the issue actually is.
Oh, and the cool part is: you won't need any statistics to do those calculations.


Quote:
Originally Posted by TheDjinni View Post
I think that if you can solve the problem of the noise in that elo range (which you have acknowledged is an issue in this thread) you can greatly decrease the number of elo hell complaints.
The reason of most of those complaints being of players in this elo-range isn't related to noise, but to 80% of the players actually being in this elo-range.
We have posted the numbers repeatedly:

EU-West 31.08.2011 - 06:25h

Solo-Queue:
  • Total-Ranked-Solo-Queue: ca. 400.000 [100%]
  • players with >=1200 Rating - 128.957 [Top 33%]
  • players with >=1300 Rating - 67.138 [Top 17%]
Due to elo-shifting-effects the median wasn't 1200 anymore. But as you can easily see, most player are between 900 and 1300 Elo.