Do you think the Tribunal is effective?

Yes (Please explain why) 347 65.72%
No (Please explain why) 181 34.28%
Voters: 528. You may not vote on this poll

Poll: Do you think the Tribunal is effective?

First Riot Post
Comment below rating threshold, click here to show it.

RivenOnAPrayer

Member

08-19-2012

Quote:
Originally Posted by Lyte View Post
I'm currently working so I can't be as in-depth as I'd like; however, there's interesting discussion here so I'm going to throw in some additional details to clear up some misconceptions.

1) The average Tribunal user actually has really low toxicity compared to the rest of the population. Whether you are looking at Offensive Language, Negative Attitude or Verbal Abuse reports or simply numbers of reports, the average Tribunal user has lower averages than a random sample in the population and way lower than the average toxic player. Saying the "trolls" or "kids" are the ones voting in the Tribunal is just false.

2) Over 50% of players who get a warning from the Tribunal understand what they did wrong and reduce their toxicity dramatically--these players never come back to the Tribunal again.

3) That % that reforms gets lower and lower with multiple punishments in the Tribunal, suggesting that if a player doesn't 'get it' within 1-2 punishments... they are unlikely to ever understand why their behavior is inappropriate in League of Legends.

4) A large percentage of players actually never harass, flame, or use racist/bigotry/abusive language towards other players. Surprisingly, the average player in League of Legends doesn't use this kind of language so saying "it's the internet and everyone is bad" just isn't true.

5) A large number of players actually do not know what the Tribunal is--they don't visit the forums or the website and are not banned by the Tribunal... so it's an unknown entity to them.

I know the Tribunal isn't perfect; but, it does what it's designed to do well which is to identify the most toxic players in the game and either reform or remove them. Should we be designing other systems for the good players and the other 98% of the population that may never interact with the Tribunal? Yes. We know the Tribunal by itself isn't enough to tackle player behavior issues in the game; but, it's one piece of the overall solution.
The problem here isn't percentage of the whole population Lyte - it's several factors.

First, saturation is a major issue. As a player between 900 and 1000 elo, I can say with confidence that there is at least one person per game who does something bannable by tribunal (AFK, verbal abuse, intentional feeding), and I'd be willing to keep a record to prove it. You can point to statistics that say 98% of the whole population of players never get into tribunal, when in reality it *feels* like at least 10% of players deserve it (statistically speaking on a per-game basis). It's a nice statistic you've found to be able to point to, but that statistic alone presents several problems. Is this active accounts only? Or does it include accounts that were made and then abandoned? That's just inflating the population with a demographic that isn't a factor. Abandoned accounts don't join a game, threaten "mid or afk", then proceed to rage, intentionally feed, then leave a game.

Second, Tribunal's effects aren't measurable when banned people can just proceed to make another account. I've noticed when leveling up a secondary account that there was an overwhelming number of trolls/feeders in the first 10 or so levels - I'd seen as many as 4 in a game (I'm not adding people who don't know/are trying to learn the game mind you - this is buying boots of mobility and charging into their tower like no tomorrow).

Third - the number of people I've seen ignore/refuse to report a person clearly in violation of the summoner's code is staggering. The majority of them come from the opposing team to the offender - if team 1 has an intentional feeder giving team 2 a free win, I've seen many times where team 2 just laughs and says they won't report since they got a free win and don't care. If someone from team 1 goes afk, team 2 will laugh and say it didn't matter and that they were going to win anyway. While many people do report and have a sense of justice, just as many people are supportive or that behavior.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

08-19-2012
2 of 11 Riot Posts

Quote:
Originally Posted by TheFatalWound View Post
The problem here isn't percentage of the whole population Lyte - it's several factors.
That's understandable, but I can break down the statistic for any Elo, any game mode, any queue type We have to work with our marketing teams to figure out how to better talk about this data.

Quote:
First, saturation is a major issue. As a player between 900 and 1000 elo, I can say with confidence that there is at least one person per game who does something bannable by tribunal (AFK, verbal abuse, intentional feeding), and I'd be willing to keep a record to prove it. You can point to statistics that say 98% of the whole population of players never get into tribunal, when in reality it *feels* like at least 10% of players deserve it (statistically speaking on a per-game basis). It's a nice statistic you've found to be able to point to, but that statistic alone presents several problems. Is this active accounts only?
I only run analyses on current active accounts.

Quote:
Second, Tribunal's effects aren't measurable when banned people can just proceed to make another account. I've noticed when leveling up a secondary account that there was an overwhelming number of trolls/feeders in the first 10 or so levels - I'd seen as many as 4 in a game (I'm not adding people who don't know/are trying to learn the game mind you - this is buying boots of mobility and charging into their tower like no tomorrow).
Players say this, but re-leveling an account to Level 30 is an obstacle that many players refuse to do again. Do some banned players create smurfs and start over? Yes. Do all of them? No. So the effects are measurable, they just aren't 100% effective against toxic players.

There tends to be too many extremes when talking about a system like the Tribunal. If there's 1 false positive, some players claim the entire system is broken--even if there's 59,999 correct verdicts in the same time window.

If there's 1 toxic player that gets banned and creates a smurf account, some players say the system doesn't actually reform any player. Systems that deal with problems like player behavior never work in this extreme fashion--it's never all or nothing.

When players look at the Tribunal forums and see complaints about their cases every week, they don't see the tens of thousands of players that receive cases and understand what they did wrong and don't make a fuss about it in a public forum.


Comment below rating threshold, click here to show it.

StevieStevinson

Junior Member

08-19-2012

Quote:
Originally Posted by TheFatalWound View Post
The problem here isn't percentage of the whole population Lyte - it's several factors.

First, saturation is a major issue. As a player between 900 and 1000 elo, I can say with confidence that there is at least one person per game who does something bannable by tribunal (AFK, verbal abuse, intentional feeding), and I'd be willing to keep a record to prove it. You can point to statistics that say 98% of the whole population of players never get into tribunal, when in reality it *feels* like at least 10% of players deserve it (statistically speaking on a per-game basis). It's a nice statistic you've found to be able to point to, but that statistic alone presents several problems. Is this active accounts only? Or does it include accounts that were made and then abandoned? That's just inflating the population with a demographic that isn't a factor. Abandoned accounts don't join a game, threaten "mid or afk", then proceed to rage, intentionally feed, then leave a game.

Second, Tribunal's effects aren't measurable when banned people can just proceed to make another account. I've noticed when leveling up a secondary account that there was an overwhelming number of trolls/feeders in the first 10 or so levels - I'd seen as many as 4 in a game (I'm not adding people who don't know/are trying to learn the game mind you - this is buying boots of mobility and charging into their tower like no tomorrow).

Third - the number of people I've seen ignore/refuse to report a person clearly in violation of the summoner's code is staggering. The majority of them come from the opposing team to the offender - if team 1 has an intentional feeder giving team 2 a free win, I've seen many times where team 2 just laughs and says they won't report since they got a free win and don't care. If someone from team 1 goes afk, team 2 will laugh and say it didn't matter and that they were going to win anyway. While many people do report and have a sense of justice, just as many people are supportive or that behavior.
Welcome to the world. If someone breaks open an atm and walks away with it wide open in NYC, people will gladly "clean up" the money lying around. League isn't a utopia, and there is no solid justice system in existence to have what you want and still make this game fun.


Comment below rating threshold, click here to show it.

CarlSagansCosmos

Senior Member

08-19-2012

Quote:
Originally Posted by Donnylicious View Post
Do you feel that reporting players is effective in encouraging admirable gameplay or is it more of a nuisance and simply a tool for reprisal?

Does the Tribunal make players want to be more positive or does it just make players afraid they'll get banned again?

Is the Tribunal fair in accessing the entire picture even though it's based on limited evidence?

Do actions and programs like The Tribunal help promote Riot or just makes them lose customers?

What are your thoughts?
All in all, I think the Tribunal is an incredibly well thought out and ingenious system. Honestly, I'm surprised more companies are not adopting the system, and I would not be surprised if they are not developing a system of their own that reflects this and may one day show.

Do I fear a report? Not really. I have no fear that if I am reported for whatever reason should it reach the Tribunal I will be pardoned. I know I follow the summoners code.

The fears of what if bob and tom are co playing and both report you have been answered time and time again. You have to be reported multiple times across multiple games and I do not believe that would happen as long as you show sportsmanlike behavior. There is no evidence that it does happen and if it does you are found guilty by the tribunal. I think he mentioned a 0.5% "oops" rate, and they have mentioned that that 0.5% many of them are repeat offenders, so it's not like they were innocent to begin with.

Some things to think about

1) Riot has stated they feel the tribunal is TOO lenient. Consider that if they took the charge and removed tribunal they would be punishing a lot of players.

2) People keep throwing out "riot is a company and need to make money all they are doing is losing money". You have to view this on two angles. A hostile environment will make customers leave. They will lose money regardless. In this instance, instead of paying a human to scan through each report they have a basically free system of justice (that is showing to work). Now remember they were be MORE harsh than tribunal which means they would be losing more money by banning more people and losing money while paying a team to sort through reports. So, now they are keeping more customers by making the environment more friendly and saving money on workers.

I also know riot can give players things without losing money. The term "you have to spend money to make it" should be something most people are familiar with. They can promote good behavior by making skins for people who follow the summoners code, give away champs ect...because it makes your fan base more loyal AND promotes positive gaming which means more customers will likely stay and play and thus spend more money. The idea that riot cannot afford to give away things because they are a "company to make money" would only be true if they were on the verge of bankruptcy.

All in all, I think this is something you will see many other gaming companies take hold, It's proving to be effective, because if it was not, they would not use it. They are a "company after all", and if they were losing money they would not continue with it.


Comment below rating threshold, click here to show it.

CarlSagansCosmos

Senior Member

08-19-2012

Hey Lyte, any chance of making the tribunal an App for the iphone/android?


Comment below rating threshold, click here to show it.

RivenOnAPrayer

Member

08-19-2012

Quote:
Originally Posted by Lyte View Post
That's understandable, but I can break down the statistic for any Elo, any game mode, any queue type We have to work with our marketing teams to figure out how to better talk about this data.



I only run analyses on current active accounts.



Players say this, but re-leveling an account to Level 30 is an obstacle that many players refuse to do again. Do some banned players create smurfs and start over? Yes. Do all of them? No. So the effects are measurable, they just aren't 100% effective against toxic players.

There tends to be too many extremes when talking about a system like the Tribunal. If there's 1 false positive, some players claim the entire system is broken--even if there's 59,999 correct verdicts in the same time window.

If there's 1 toxic player that gets banned and creates a smurf account, some players say the system doesn't actually reform any player. Systems that deal with problems like player behavior never work in this extreme fashion--it's never all or nothing.

When players look at the Tribunal forums and see complaints about their cases every week, they don't see the tens of thousands of players that receive cases and understand what they did wrong and don't make a fuss about it in a public forum.
Thanks for responding, but what's your opinion on the issue of saturation of offenders? I can say that I rarely ever play ranked anymore unless it's a 5 man team because the sheer number of people who do egregious offenses (afk, intentionally feeding, etc.). It's become an environment where you can't even enjoy a win anymore because it isn't a real win- it always becomes 5v4, or someone just starts feeding. It's both depressing, and drains any enjoyment out of the game.

That being said, a couple of questions and comments:

First, do you think the categories for reporting could use some expansion? I feel like there are some things that need to be added. For example, if somebody gets mad and proceeds to infinitely sit in a lane lategame regardless of if the team is begging for them to help during team fights. They don't intentionally feed, they don't exactly afk, and their stats look justifiably innocent for any of the existing categories. All that tribunal has to go in is word of mouth/the reports. As a person who has made decisions inside of the tribunal, there is no way to clearly communicate that situation. I know there are other situations like this where there is no clear way to communicate an issue - I can't remember it off the top of my head but I'll gladly readd it when I inevitably run into a similar situation.

I've also got a replay of another game where I truly believe that some bad offenders will not be reported due to no way to communicate it. The other team didn't care enough to report, and they were a group of two, so they'll have 3 reports at best assuming our other teammate reported them. I've got the replay in a LoLReplay file, as I know Riot doesn't enjoy calling names out specifically. Either way, they badly need to be banned and I have a very strong feeling they won't. It's a case that very strongly supports my last statement of having no way to clearly indicate why they need to be banned.

This leads to another question: Do you think there is room to add some sort of replay system or other more clear way to review a game? There were a lot of cases in the tribunal where there was so much gray area that I couldn't tell.

Next, a question: In the tribunal rules, it says someone has right to be pardoned if they leave after having already been wronged. What does this mean? If someone is intentionally feeding, do I have right to leave the game if they refuse to surrender and drag the game out? How far does this rule extend? How many people in tribunal will actually pay attention to this rule? I haven't used this rule yet, as I feel people won't acknowledge it and LeaverBuster is an automated system. It's an interesting rule with some good implications for people being griefed but there's a lot of gray area.

Lastly, I've noticed that when reviewing tribunal cases, there a lot of people in the cases reviewed that also deserve reports and may not have gotten them. I would like to see a feature where you can advocate for that person to be placed in the tribunal as well.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

08-19-2012
3 of 11 Riot Posts

Quote:
Originally Posted by TheFatalWound View Post
That being said, a couple of questions and comments:

First, do you think the categories for reporting could use some expansion? I feel like there are some things that need to be added. For example, if somebody gets mad and proceeds to infinitely sit in a lane lategame regardless of if the team is begging for them to help during team fights. They don't intentionally feed, they don't exactly afk, and their stats look justifiably innocent for any of the existing categories. All that tribunal has to go in is word of mouth/the reports. As a person who has made decisions inside of the tribunal, there is no way to clearly communicate that situation. I know there are other situations like this where there is no clear way to communicate an issue - I can't remember it off the top of my head but I'll gladly readd it when I inevitably run into a similar situation.
Actually, we're doing the opposite for a few reasons. There's too many categories with too much overlap and each report has a specific definition that is hard to understand for many players. For example, what the difference between "Intentionally Feeding" and "Assisting Enemy Team?" One could argue that intentionally feeding is assisting the enemy team. Expanding reports to even more categories just increases the potential confusion and overlap between reports.

In the future, I'm going to try to reduce the reports down to just the basics. Perhaps we'll require players to put in a detailed report reason, perhaps not. We have to see.

Quote:
I've also got a replay of another game where I truly believe that some bad offenders will not be reported due to no way to communicate it. The other team didn't care enough to report, and they were a group of two, so they'll have 3 reports at best assuming our other teammate reported them. I've got the replay in a LoLReplay file, as I know Riot doesn't enjoy calling names out specifically. Either way, they badly need to be banned and I have a very strong feeling they won't. It's a case that very strongly supports my last statement of having no way to clearly indicate why they need to be banned.

This leads to another question: Do you think there is room to add some sort of replay system or other more clear way to review a game? There were a lot of cases in the tribunal where there was so much gray area that I couldn't tell.
A lot of players ask for replays in the Tribunal, but it's not a great solution. For example, the vast majority of cases already have enough evidence to render an accurate verdict--we have done numerous analyses on this. Secondly, very, very few players actually want to load up a client and watch a replay to look for toxic behaviors--the barrier of entry is just so high so why work on a feature that almost no players will actually use?

Then you might ask: well, what do we do about cases where there isn't enough information in a classic Tribunal case and we need replays to tell what's going on? Well, the Tribunal isn't the answer for those cases. The Tribunal is one piece of the player behavior solution and we are constantly working on others.

Quote:
Next, a question: In the tribunal rules, it says someone has right to be pardoned if they leave after having already been wronged. What does this mean? If someone is intentionally feeding, do I have right to leave the game if they refuse to surrender and drag the game out? How far does this rule extend? How many people in tribunal will actually pay attention to this rule? I haven't used this rule yet, as I feel people won't acknowledge it and LeaverBuster is an automated system. It's an interesting rule with some good implications for people being griefed but there's a lot of gray area.
What rule are you referring to? We're in the process of re-doing the guidelines and FAQ and this just sounds like an old rule we missed.

Quote:
Lastly, I've noticed that when reviewing tribunal cases, there a lot of people in the cases reviewed that also deserve reports and may not have gotten them. I would like to see a feature where you can advocate for that person to be placed in the tribunal as well.
We won't be allowing reviewers to 'judge' other players in a given game--this is mainly to avoid double jeopardy. We encourage players to focus on the player you are given in your case because it's likely the other toxic players in the game are already being voted on in another case or even already punished.


Comment below rating threshold, click here to show it.

Whichi

This user has referred a friend to League of Legends, click for more information

Member

08-19-2012

I think the tribunal is actually highly flawed... Riot posts stat's and etc.. but again this doesn't always mean its correct... I've already seen more then my fair share of report cards of people being banned by the tribunal for reasons they shouldnt have.. Riot comes in and says otherwise but without any proof or anything to prove for it..

When i see a players report card say they got reported for making Homosexual Jibes at them without even talking in the entire match then the rioter comes in and says it was pre game post game chat.. not believble.. not even for a second...

The tribunal itself probably works on the majority of the trolls but some players just downright dont deserve pure 100% Permy punishment... its just wrong.. If you want to fix it you'll continue to ban them for a lot longer before you permy ban them.. Players will come back and harass low levels for a while which makes the game very hard and ruins it even more so then if they're harassing higher levels

Seriously explain to me how harassing totally new players to the game is good for the game compared to just continously banning their high accounts every three weeks while they harass higher level players.. It just doesnt make sense... It's totally and completly wrong.. It destroys the invitition of newer players.. ALOT of newer players dont even find themselves wanting to play with real people.. Making themselves only play bots untill 30... Thats a god **** awful way to introduce someone into a moba.. I love Mobas and have a huge passion for them the fact you're destroying new players idea's of moba's simply from the tribunal alone is terrible...

TLDR im perfectly fine with punishment.. Anything.. Player deciding cases however needs to leave.. Now.... Watching people get punished through the tribunal for things they didnt do is becoming quite old.. as i've seen it more than enough times now


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

08-19-2012
4 of 11 Riot Posts

Quote:
Originally Posted by Whichi View Post
I think the tribunal is actually highly flawed... Riot posts stat's and etc.. but again this doesn't always mean its correct... I've already seen more then my fair share of report cards of people being banned by the tribunal for reasons they shouldnt have.. Riot comes in and says otherwise but without any proof or anything to prove for it..

When i see a players report card say they got reported for making Homosexual Jibes at them without even talking in the entire match then the rioter comes in and says it was pre game post game chat.. not believble.. not even for a second...

The tribunal itself probably works on the majority of the trolls but some players just downright dont deserve pure 100% Permy punishment... its just wrong.. If you want to fix it you'll continue to ban them for a lot longer before you permy ban them.. Players will come back and harass low levels for a while which makes the game very hard and ruins it even more so then if they're harassing higher levels

Seriously explain to me how harassing totally new players to the game is good for the game compared to just continously banning their high accounts every three weeks while they harass higher level players.. It just doesnt make sense... It's totally and completly wrong.. It destroys the invitition of newer players.. ALOT of newer players dont even find themselves wanting to play with real people.. Making themselves only play bots untill 30... Thats a god **** awful way to introduce someone into a moba.. I love Mobas and have a huge passion for them the fact you're destroying new players idea's of moba's simply from the tribunal alone is terrible...

TLDR im perfectly fine with punishment.. Anything.. Player deciding cases however needs to leave.. Now.... Watching people get punished through the tribunal for things they didnt do is becoming quite old.. as i've seen it more than enough times now
But I ask you, how many cases are perfectly legitimate and accurate? For every case you see on the forums where you believe the banning was unjust, what if there are 50,000 cases where the banning was just?

Is it possible to have a perfect system where the false positive rate is 0%? What false positive rate is acceptable to players? In medical fields, a false positive rate of 5% is considered standard and a false positive rate of 1% is excellent. We hold the Tribunal to harsher standards, requiring much better than a 1% false positive rate.

As a company, banning players and removing players from the game is the last thing we want to do. We take banning very seriously--we have Player Support reviewing tons of cases to assign punishments and ensure accuracy. We have designers, analysts and executives personally reviewing numerous cases every week, even ones that are just for warnings.


Comment below rating threshold, click here to show it.

Whichi

This user has referred a friend to League of Legends, click for more information

Member

08-19-2012

Quote:
Originally Posted by Lyte View Post
But I ask you, how many cases are perfectly legitimate and accurate? For every case you see on the forums where you believe the banning was unjust, what if there are 50,000 cases where the banning was just?

Is it possible to have a perfect system where the false positive rate is 0%? What false positive rate is acceptable to players? In medical fields, a false positive rate of 5% is considered standard and a false positive rate of 1% is excellent. We hold the Tribunal to harsher standards, requiring much better than a 1% false positive rate.

As a company, banning players and removing players from the game is the last thing we want to do. We take banning very seriously--we have Player Support reviewing tons of cases to assign punishments and ensure accuracy. We have designers, analysts and executives personally reviewing numerous cases every week, even ones that are just for warnings.
Oh this is very true i suppose. I guess i cant say its HIGHLY flawed.. there might be a loose flaw.... oh i dont know.. you got me T_T