How many games do you expect to play before you reach your true Elo?

< 10 Games 180 1.63%
11-40 Games 819 7.42%
41-80 Games 1,162 10.53%
81-120 Games 1,736 15.73%
121-160 Games 664 6.02%
161-200 Games 928 8.41%
200-399 Games 1,701 15.42%
400-599 Games 1,082 9.81%
600-799 Games 456 4.13%
800-999 Games 170 1.54%
1000+ Games 2,136 19.36%
Voters: 11034. You may not vote on this poll

Help Riot improve matchmaking! Looking for examples of bad matchmaking

First Riot Post
Comment below rating threshold, click here to show it.

Coltsbro

Junior Member

02-16-2012

Quote:
Originally Posted by Lyte View Post
I'll double check this when I get back to Riot HQ, as I am currently at a convention. I am 90% sure of the answer, but want to double check first!
Have you ever thought of the possibility of switching from an overall ELO system to a per champion ELO system (or keep overall ELO and still implement the per champion ELO)? Realm of the Titans did this while they were testing, and I liked the idea of it. Then maybe in ranked you could give the option to click on another summoner's name and see like their top 3-5 champions and what ELO's they are with them, which would get rid of the "Dude let me Kass I'm really good with him" and someone else says "No, noob i go Morgana mid" Where the Kass would be a 1800 Kass and the Morgana would be a 1200 Morgana. I think this could help out a ton in matchmaking


Comment below rating threshold, click here to show it.

MG z3rg3m3ow

Senior Member

02-16-2012

Quote:
Originally Posted by I Play Snake View Post
i have been matched with people like xpecial, z3rg3m3n, xxphaxen, and im not even lv 20 yet
explain?
O_O

I am sorry.


Comment below rating threshold, click here to show it.

Justiczar

Member

02-16-2012

All credit goes to rawpower405. Brilliant example of how to fix matchmaking in ranked games. This should have been read a VERY LONG time ago.

Quote:
Ive decided to write this post with some results of a preliminary analysis of Matchmaking and would love for a Red to comment on it. Specifically, I've done an analysis of incorporating Margin of Victory in a probability of victory function. I'm including the results here and hope to have an expanded database to work with in the future.

This DOES NOT include:
-Elo Hell QQ
-The Matchmaking Algorithm
-Using Individual Performance Metrics

I believe that the fundamental nature of a dynamic game like League of Legends is impossible to boil down to a simple binary outcome: winning and losing. There are a number of statistics that are already gathered in-game that would help us inform how decisive a victory is. I previously wrote about this at http://community-na.static.leagueofl...d.php?t=383560 but if you don't want to click,
Quote:
Originally Posted by rawpower405
I do not think Riot should assess individual performance in a game. However, they can take into account the teams final K/D/A/Towers and develop a margin of victory function. The whole point is to develop additional information taken from the matches that are played to produce a better way of interpreting the outcome.

We've all played close games where the final scores were like 50-48. But we've also all played games where the final scores were like 3-25 with a surrender at 25 minutes. Both of those games SHOULD NOT be treated equally since the final scores are so vastly different. One game was won much more conclusively than the other.
Football Ranking Systems

I'm going to take College Football as an example of how additional information can be used to INFORM a teams performance. On a 12 game regular season, we could have 4 teams that have the following records:

Quote:
Originally Posted by rawpower405
Team A: 12-0
Team B: 12-0
Team C: 0-12
Team D: 0-12

Simply looking at wins/losses we would conclude that Teams A and B are roughly equal in skill and teams C and D are roughly equal in skill as well. However, football provides us with a final score for all games. The final score provides us with additional information INFORMING us of how each team played in each of their games.

Team A won all of their games by 1 point. Team B won all of their games by 24 points. Team C lost all of their games by 24 points. Team D lost all of their games by 1 point.

Obviously Team B is better than Team A despite having identical records. Why? Because Team B blew out all of their opponents while Team A squeaked by theirs.

We would then re-rank the four teams as follows:

Team B
Team A
Team D
Team C
In terms of football ranking algorithms and predictors, utilizing Margin of Victory has already been assessed multiple times. Jeff Sagarin, one of the computer models used in the BCS formula, produces two rankings. One is Elo based, dubbed Elo-Chess, that DOES NOT take into account Margin of Victory, and one is Elo based and takes into account Margin of Victory. Not surprisingly, Sagarin flat out tells us that his Elo-Chess method is flawed. To quote directly from Sagarin's published rankings:
Quote:
In ELO CHESS, only winning and losing matters; the score margin is of no consequence, which makes it very "politically correct". However it is less accurate in its predictions for upcoming games than is the PURE POINTS, in which the score margin is the only thing that matters. PURE POINTS is also known as PREDICTOR, BALLANTINE, RHEINGOLD, WHITE OWL and is the best single PREDICTOR of future games.
So what does Football rankings and Margin of Victory have to do with League of Legends? Well by using a MOV of some sort in the Elo equation, we should, theoretically, produce a better predictor of matches and thus improve match quality.


Margin of Victory

First we must transform Elo ratings into a Probability of Victory. To do this, we use this function:
Quote:
1/(1+(10^((A-B)/400)))
where A is the average Elo of Team A and
B is the average Elo of Team B.
Team A with an Elo of 1300 against Team B with an elo of 1200 has a Probability of Victory of 64.006%. Teams with Elos that are equal have a POV of 50%.

In order to incorporate margin of victory in the above formula, most sports simply plug the final scores in as A and B with the only thing that changes is the number you divide it by (400 for the case of LOL. College football uses something like 15, NFL something like 6).

I'm hypothesizing that incorporating margin of victory in some way into the probability of victory function will only IMPROVE the predictability of results.


Data Analysis
In order to demonstrate Margin of Victory, I gathered data from lolbase.net. Data included the average elos of each team before the match, the Kills/Deaths/Assists/Towers for each team, and the team who actually won.

I hand gathered all of the data, but hope to have a larger data set to work from in the near future. Due to the time consuming nature, I've only gathered a sample set of 56 matches. While this sample is small, I do feel that the results are encouraging and support the hypothesis from above.

The matches involved Elos ranging from 1193 to 1861. I feel that this is a great range of matches and includes a broad range of players.

The average Elo difference in the matches is 34.65. This represents a Probability of Victory for the higher elo teams of 54.97%. The higher elo teams should win 55 games out of every 100. Very good matchmaking. The closest match in the database had Elos of 1445.2 and 1443.4 (talk about a good match!).

The Probability of Victory was amended to:
Quote:
1/(1+(10^((A-B)/46)))
where A is the Total Takedowns of Team A and
B is the Total Takedowns of Team B.
The reason why the denominator was changed to 46 is because this was the average takedown differential in a game. Meaning the average game was won by 46 takedowns. Games ranged from the closest game having just 2 takedown difference to the largest margin of victory of 91 takedowns.

The Margin of Victory formula predicted that the Probability of Victory for higher Elo teams is actually 63.03% and represents a difference of 8.06% from the Elo POV (54.97%). This means that the Higher Elo teams should actually win 63 games out of 100 instead of 55.

The actual number of wins in the 56 match sample set was 36. So the higher elo teams won 36/56 matches, meaning they lost 20 out of 56. This translates to a 64.3% actual win percentage.

Quote:
The closest match in the database had Elos of 1445.2 and 1443.4 (talk about a good match!).
The actual results were 28 to 72. Representing a Probability of Victory of 90.04% (talk about a stomp....)


If you read nothing else, read the following chart:

Elo POV: 54.97%
MOV PoV: 63.03%
Actual: 64.3%


Conclusion
Margin of Victory can go a long way in improving match ups and the initial results show that it provides a significant improvement for informing the results of Elo.

One of the takeaways from this exercise is that there are statistics in League of Legends that can be readily utilized to improve match quality. They DO NOT have to be takedowns, specifically. However takedowns are already a statistic that is compiled both in the game client and the launcher.

I also think that the "Snowbally" nature of LOL could be partially due to matchmaking. As I demonstrated above, Elo is doing exactly what it THINKS it should be: producing matches that it thinks are relatively even. However, it is not producing even match ups, its producing matches with wins in 2 out of 3 matches instead of wins in 2 out of 4. This is a significant difference.

I leave it up to the game developers to further refine how much Elo the players should get at the conclusion of a match. More than likely through a modification of the K value.



TL;DR

Margin of Victory is a better predictor of match outcomes than Elo.

Elo POV: 54.97%
Margin of Victory POV: 63.03%
Actual win Percentage: 64.3%


Comment below rating threshold, click here to show it.

Levnik Moore

This user has referred a friend to League of Legends, click for more information

Senior Member

02-16-2012

Quote:
Originally Posted by z3rg3m3n View Post
O_O

I am sorry.
GGG Z3rg.

Kicks your a**, apologizes


Comment below rating threshold, click here to show it.

ncrwhale

Senior Member

02-16-2012

Hey Lyte, first, thanks for your hard work.

I have some more anecdotal evidence for you. I'm 200-700 elo above all my "rl" friends. Any time we play normals with 3+ I MUST play AP/AD or we lose very badly. Really the only time we win and I'm not in one of those roles is if we just lost 3-5 games in a row.

Similarly for ranked, when I queue with friends who are much lower, I find the games to be very difficult. This difficulty is typically alleviated if there is a similar duo queue on the other team.

Recently, when solo queuing, I've had a few games "ruined" a few times by plat smurfs; they just dominate their lanes.

All in all I've had a good experience with MM. First and second seasons I've got up to 1600 (which I think was my "true elo" at the time) in around 40 games, with some being un-winnable and some being really easy, but for the most part being fair.

I hope you can understand that math paper you bookmarked a lot better than I!


Comment below rating threshold, click here to show it.

calicfer

Senior Member

02-16-2012

Quote:
Originally Posted by Lyte View Post
Streaking is an interesting phenomenon. For example, everyone considers a coin toss as an event that should result in 50% heads and 50% tails over a large number of events; however, in the short-term, if all 30+ million League of Legends players started flipping coins, lots of individual people will see streaks of 5, 6, 7, even 8 or more of heads or tails!

What I am trying to illustrate here is that even if matchmaking was perfect (and I don't think any matchmaking system can truly be perfect), it will still have streaks and lots of them. However, perhaps we can think of creative ways that will help reduce a players' chance of getting 'extreme' losing streaks that are emotionally demoralizing.
It is an interesting idea. I think you risk throwing people into games they aren't prepared for. You also risk giving games that they will stomp in.

Maybe give players a choice about what win % they want.
Most players would pick 50%.
after 6 losses, many players might pick 75%
People who want to get better would always pick 25%. They would be guaranteed hard games. They would learn how to play when outmatched.

I'd only be afraid of the 75% que-times getting long as people want to win more than they lose. Conversely, people want short que-times and will balance the desire to win with the desire to get in game.

maybe replace the fixed %ages with ranges... 25%-50%, 40%-60%, 50%-75%

It would also be nice to have the predicted win % displayed at some-point. That way we could know if the elo predictor failed or if the matchmaker failed.

go given a set of 6 games at .5 win...
you have a 1.6% chance of winning all 6 (.5^6)*100%
I've played close to 1000 games. So roughly multiplying which is incorrect, I should have had 16 win streaks and 16 loss streaks.

I think I've had a lot more "random" win streaks than loss streaks. I think this has to do with learning new champions. I play with a new champion about once a month. I play them for many games in a row. Many of these games would be losses. (it was worse when I was newer to the game)... Trying a new champion would give me 2-4 expected losses in a row. My coin flip on my main champions is probably around 65% right now.

It would be nice to not impact matchmaking when trying new champions by allowing players to have 3 normal elo's. "Hardcore, relaxed, and learning." Have them be at least 80% independent (eg. people who always play hardcore with 1400 elo, but want to go for learning would start at 1240).
^ because this algorythm is playstyle agnostic, people could use these 3 rankings for
"Main role", "Off role", "whatever role"
or
"Main champ", "Off champ", "new champ"
or
"Carry", "Troll build", "Experimentation"


Comment below rating threshold, click here to show it.

calicfer

Senior Member

02-16-2012

Quote:
Originally Posted by Lyte View Post
Glicko systems have a 'confidence rating,' which is an extra parameter that represents how confident the system is that a player is currently at his true skill level. I think Glicko tries to solve the concerns you have raised.

Elo systems have K-factors, which are similar, but basically represent how important a match is. Early in a player's career, a K-factor starts high because the first few matches are important for determining the relative tier a player should be at.

League of Legends, however, has additional complexity which is why traditional Glicko or Elo systems are not sufficient. For example, a 1800 Vayne player is not necessarily a 1800 Annie player. This means that player performance and consistency varies greatly from game to game, adding a lot of additional variables that are hard to capture with traditional matchmaking systems.

In saying this, there are a lot of smart people at Riot, and we are going to do what we can to make the best matchmaker possible--it will never be perfect, but we will always keep working on it.
We know when we are doing something different.

Let us have multiple elo's. Somewhere between 3 to 5.
As players, we can identify when we are doing something differently and select a different "elo track"


(usefull for learning new champions / using experimental builds / using "troll builds" (fun builds that aren't competitive))
^ you could do these without guaranteeing a loss for your team by doing them on a single "elo track"


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

02-16-2012
52 of 173 Riot Posts

Quote:
Originally Posted by calicfer View Post
It is an interesting idea. I think you risk throwing people into games they aren't prepared for. You also risk giving games that they will stomp in.

Maybe give players a choice about what win % they want.
Most players would pick 50%.
after 6 losses, many players might pick 75%
People who want to get better would always pick 25%. They would be guaranteed hard games. They would learn how to play when outmatched.

I'd only be afraid of the 75% que-times getting long as people want to win more than they lose. Conversely, people want short que-times and will balance the desire to win with the desire to get in game.

maybe replace the fixed %ages with ranges... 25%-50%, 40%-60%, 50%-75%

It would also be nice to have the predicted win % displayed at some-point. That way we could know if the elo predictor failed or if the matchmaker failed.

go given a set of 6 games at .5 win...
you have a 1.6% chance of winning all 6 (.5^6)*100%
I've played close to 1000 games. So roughly multiplying which is incorrect, I should have had 16 win streaks and 16 loss streaks.

I think I've had a lot more "random" win streaks than loss streaks. I think this has to do with learning new champions. I play with a new champion about once a month. I play them for many games in a row. Many of these games would be losses. (it was worse when I was newer to the game)... Trying a new champion would give me 2-4 expected losses in a row. My coin flip on my main champions is probably around 65% right now.

It would be nice to not impact matchmaking when trying new champions by allowing players to have 3 normal elo's. "Hardcore, relaxed, and learning." Have them be at least 80% independent (eg. people who always play hardcore with 1400 elo, but want to go for learning would start at 1240).
^ because this algorythm is playstyle agnostic, people could use these 3 rankings for
"Main role", "Off role", "whatever role"
or
"Main champ", "Off champ", "new champ"
or
"Carry", "Troll build", "Experimentation"
I am interested in the idea of possibly presenting people with messages about their current match-up. I would not want to present people exact expected win rates based off Elos in the Loading Screen; however, there is potential for presenting messages such as, "You are Team Blue, the underdogs in this match!" or "You are Team Blue, the favored Champions in this match!"

The reason why I am hesitant to post exact expected win%s is because it is easy to misconstrue the meaning. If a matchmaker suggests that you have a 85% chance to win but your team loses, that does not mean the matchmaker necessarily failed. Afterall, if we created that specific match-up many many times, you would in fact be expected to lose 15% of those games. It is hard to interpret the 'success' or failure of a matchmaker based off 1 data point.


Comment below rating threshold, click here to show it.

calicfer

Senior Member

02-16-2012

Quote:
Originally Posted by Lyte View Post
Hm, there are some interesting things to think about here. Thanks Mayipora.
^ would be really nice.

Some people carry really well.
Some people know how to be light.

People who carry well do better quing with lower elo people & have a large "carry effect"
People who are light have a very small "feeder effect" and do well when coupled with higher elo players.

also the ability to last pick helps people queing with higher elo players
knowing hard to counter champions helps with first pick.
(while you can trade to make this not effect your performance, you don't usually own all of your duo's champions)


Comment below rating threshold, click here to show it.

NougetTime

Senior Member

02-16-2012

I havent been able to read through alot of this, but at 1800-1900 elo, the disparity of the people it pulls in is too large. For example, a good chunk of the time the team will be
1600 elo
1800
1900
2000
2000

Lets say the opposing team also has the same elo (which doesn't happen much at this elo). What ends up happening is that the 1600 elo will end up going against the 2000 elo, and guess what? You pretty much just lost the game. Its really frustrating. I'm short on time so sorry if this seems rushed, its because it is.