@Lyte : The tribunal worries me

First Riot Post
Comment below rating threshold, click here to show it.

tempname950

Member

03-17-2013

bump


Comment below rating threshold, click here to show it.

VanceStubbs

Senior Member

03-17-2013

So yeah, this is the proof of the whole GD conspiracy theory that you'll be permabanned one day, no exceptions.

While we are at it, never drive a car, never use public transport, never go on a plane, never go to beaches, never own gas based heating systems...


Comment below rating threshold, click here to show it.

The Ghost Orchid

Senior Member

03-17-2013

So if you are a genuinely decent person, you have to be falsely reported a huge number of times to ever reach the tribunal. Then, if you are the victim of overly strict of people or auto-punishers (who get weeded out by the system), then you will get a warning. The chances of this happening are extremely low. Does it happen? Yes, but that's only because of the millions upon millions of people playing. Of course when there are so many people playing there will be aberrations.

So let's say you are punished that first time around? What are the consequences? That first time you're punished by the tribunal it's a warning, I believe. Let's say you're a genuinely decent person. Maybe there's something to take away from that. Maybe there are some things you can improve upon.

Then you go on your merry way and continue being that genuinely nice/amazing person you are. You have to have that happen upwards of 7 more times before your ban becomes permanent. And then once you're up for a perma pan, the case is individually reviewed by Riot.

There are just extremely low odds of absolutely everything going against you if you are genuinely a nice person. I keep saying genuine because people aren't good just because they think they are.

Fretting over this will get nobody anywhere. Claiming that the outliers should be what Riot's decisions should be made by is unreasonable. It's amazing to me that so few cases ever reach the forums and often these wrongly judged cases are even overturned by the person that people are trying to blame for the few times the tribunal messes things up.

Numbers and statistical evidence is just so much stronger than peoples 'feelings' and 'worries' on the matter.


Comment below rating threshold, click here to show it.

Amuq the Native

Senior Member

03-17-2013

Quote:
Originally Posted by Montag View Post
We have a fundamentally optimistic view of players. The vast majority of League of Legends players are all right, if not totally awesome. And their Tribunal voting record matches ours pretty damn closely.
You guys keep saying you trust the majority of players, that the community is awesome, etc. etc. Oh, really ? Then start empowering us, right now. And give us vote-kick, which should not raise any problem since a majority of players will not abuse it. Or give us other, real weapons to help us help you eliminate toxicity fast. We want to help.

As long as Riot is not trusting us enough to give us real power against toxic players, I will call total PR bullsh*t on this "you guys are awesome" stance.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

03-17-2013
6 of 14 Riot Posts

Quote:
Originally Posted by Logios View Post
I saw the following quote from you in a thread about Quality Player being banned.

I'm very disturbed that people reviewing tribunal cases are encouraged to consider evidence that is not presented and possibly doesn't exist. It worries that it is acceptable for people to not see any evidence of toxic behavior but click "ban" anyway because there is a possibility that the person was toxic in other games.

I've spent more money on this game than I'm willing to admit and I don't consider myself a toxic player. I've never been banned or warned and even got the coveted Santa Nashor icon for good behavior. But, as we all know, people report for stupid reasons, I was reported the other day for not banning Alistar and last week for "not ganking bot" and "being a noob jungler" (I love bronze division) it scares the hell out of me that all the money I've spent on this game can go down the drain because Riot's social team deems it acceptable for tribunal reviewers to completely ignore the evidence presented to them and click the punish button because just MAYBE, I was "toxic" in another game they can't see.
The average player never has to worry about the Tribunal. Last I checked, only about 0.7% of accounts in North America were eligible for a Tribunal case--you need to receive a serious number of legitimate reports to even receive a Tribunal case. This is also why a high punish rate from the Tribunal does not mean the system is broken--it just means that the criteria to get into the Tribunal are pretty strict.

But, the Tribunal isn't perfect. No system is. A lot of players might argue that if even 1 positive player is banned that the system is broken. This is unreasonable. The reality of the situation is, errors are going to happen--the question we should be asking is, "What is the acceptable error rate?" In the medical fields, a new medicine does not have to be perfect to be a viable treatment for an illness; in fact, 60% success rates are pretty good in medicine. Everyday, many people undergo surgeries that have a 95% success rate, but a 5% chance for a serious side-effect, injury or death--this is considered acceptable and pretty good. Even many birth control methods are 95-99% effective and that's considered acceptable even though the errors there are life-changing.

Because many members of the player behavior team come from these backgrounds, we have forced ourselves to have an even higher standard with the Tribunal. One of the first things we did with the system was optimize it to minimize false punishes. Since the player behavior team formed, we have not seen a positive player that was punished enough times by the Tribunal that they were permabanned. Some neutral or positive players receive undeserved warnings or timebans, but no one has been so absurdly unlucky as to end up in the Tribunal 5 times and ended with a permanent ban.

In saying this, we see a lot of complaints that the Tribunal is broken; if we were really punishing that many positive players, I'd shut the Tribunal down today. I don't need to blindly trust the system, nor does my job depend on the system being used--the player behavior team can implement whatever system we feel has the biggest impact on player behavior. When a banned player goes to the forums and complains about the system being broken, we're missing the counterbalance of that--how many players got a ban that was deserved, but don't go on the forums to post about it? How many of the 99% of players that are never going to see a Tribunal case are posting on the forums with titles like, "Yay, I will never be banned because I'm not a jerk!" Most players here might not know this, but we used to review 100% of cases in the Tribunal. In fact, for many cases we'd assign up to 6 Player Support staff to review them--this was to assess the reliability of Rioter reviews against the reliability of Tribunal judges. We ultimately discovered that Rioters tended to be much, much stricter than the Tribunal. In many countries, we still review 100% of cases. I checked 4 weeks of an International Tribunal where a PS Team reviewed 100% of cases, and do you know how many decisions they overturned? 2.

The Tribunal does a good job reforming or removing the worst players in the game--the bottom 0.7% right now on NA. It's never going to be a perfect system, and development costs are hitting diminishing returns on the system. This is why we're shifting gears on the player behavior team.

---

On the player behavior team, we know that bans are not the complete solution. Some toxic players are deterred by bans, and others simply make new accounts and shift their toxicity. Some of you may have heard of the Account Restriction experiments on PBE, and we're currently testing the first account restriction which is a Restricted Chat Mode. In the future, we'd prefer to avoid as many bans as possible and instead opt for players to remain on their mains but with multiple restrictions on the account. We've decided this effort is a higher priority and more valuable than adding more and more information (like pre- and post-game chat) to the Tribunal. Because account restrictions are less jarring than bans, we can directly impact a lot more players and cast much larger nets than a system that hands out bans.

Secondly, our data shows time and time again that the majority of players are not innately toxic. Due to context (in real life or otherwise), players tend to have bad days and their behavior goes toxic for short bursts. The vast majority of players are pretty positive people--if you think about it, most players say they experience toxicity in a high number of games... but it's usually 1 or 2 people in the game and not everyone in the game. In fact, the majority of our players are neutral or positive and don't say much in the game other than "gg."

Statuskwoh and others have mentioned this already, but the player behavior team has been in the middle of a serious shift. We're going to stop focusing on punishment, and instead focus on features that encourage sportsmanlike behavior in the game. Part of the initial explorations include research on the current problems with Champ Select. We are considering every possibility, and can even completely change how Champ Select works if we believe it's the best thing to do for League of Legends. You can read some of our initial thoughts here: http://na.leagueoflegends.com/board/....php?t=3220746

I've played over 2500 games of League of Legends--90% of those games in Ranked Solo/Duo queue. If you consider the entire Player Behavior team, we've put in some serious time into the game. We're gamers, just like you. If the community wasn't pretty awesome, systems like the Tribunal just wouldn't work. But, the average member of the community is positive. When I first joined Riot, other devs and academics told me that online games and toxic behavior simply go hand-in-hand; they said this was an impossible problem.

I disagree.

When I look at Tribunal data and see that words like c*nt and f*ggot are among the most highly punished words in the Tribunal, I am inspired by the gamers in League and how awesome they are--we just have to figure out how to break the negative perception that weighs us down. To do that, we need to make it easier to focus on positive behavior in our community. We need to make it easier to setup teams for success.

Being a gamer is a culture. Let's not focus on the toxic players that try to ruin our experiences--they aren't a part of our gaming culture. Being a gamer is one of the most amazing and positive cultures out there. Let's work together to showcase that.


Comment below rating threshold, click here to show it.

SimplyAlive

Senior Member

03-17-2013

I support the Tribunal in general...but I must say I feel as it dropped the ball on this case.

I do not think Riot should accept 'a plethora of views with some lacking information coming to the same view as us with the additional information'.
We can only view what is presented to us in a case and can only assume what happened beyond that. That we 'assume' correctly does not solve the problem of the lacking information.

I've been hoping for pre- and post-game chat for multiple years now (no kidding) and I think it would give the community a lot more confidence in the Tribunal as a whole.


Comment below rating threshold, click here to show it.

MyosinFlare

Senior Member

03-17-2013

I stopped doing tribunal a long time ago because I was apparently very bad at it. I was pardoning too many cases apparently.

99% of the cases I saw went as followed

Person A makes a bad play or they just got out played
Person B calls them a noob or insults them
Person A gets frustrated by that and plays even worse and starts retaliating
Persona B continues to insult them even more
Feed back loop of A doing worse because B is insulting them
Everyone on the team reports Persona A for either "intentional feeding" or "verbal abuse"

Person B caused the entire problem, but they were not the one my tribunal case was concerned about.

I think to myself that I can't be the only person who thinks punishing petty arguments like this is excessive, but looking at tribunal statistics, I'm wrong about that.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

03-17-2013
7 of 14 Riot Posts

Quote:
Originally Posted by Amuq the Native View Post
You guys keep saying you trust the majority of players, that the community is awesome, etc. etc. Oh, really ? Then start empowering us, right now. And give us vote-kick, which should not raise any problem since a majority of players will not abuse it. Or give us other, real weapons to help us help you eliminate toxicity fast. We want to help.

As long as Riot is not trusting us enough to give us real power against toxic players, I will call total PR bullsh*t on this "you guys are awesome" stance.
Vote-kick has nothing to do with trusting the community to be awesome.

Vote-kick scenarios have been shown to make positive people into destructive forces. Consider research on the Prisoner's Dilemma, a mini-game about decision making where the best option is to cooperate with your opponent; however, everyday positive people choose to 'destroy' their opponent instead because they lack any information or trust about the people they are in the game with--sound familiar to Champ Select?


Comment below rating threshold, click here to show it.

The Straightener

Member

03-17-2013

Have Riot ever thought about giving us a way to track our reports?
Maybe we can see how many of our reports actually end up on some kind of punishment. This way we can evaluate how are we reporting and do a little bit better.
Some times I feel like people report out of frustration.


Comment below rating threshold, click here to show it.

Relto

Member

03-17-2013

Quote:
Originally Posted by Lyte View Post
Vote-kick has nothing to do with trusting the community to be awesome.

Vote-kick scenarios have been shown to make positive people into destructive forces. Consider research on the Prisoner's Dilemma, a mini-game about decision making where the best option is to cooperate with your opponent; however, everyday positive people choose to 'destroy' their opponent instead because they lack any information or trust about the people they are in the game with--sound familiar to Champ Select?
Does game theory come up a lot when you guys work on player behavior initiatives? It's interesting to see devs actually consulting psychology to make better online experiences.