@Lyte: A small concern about the Tribunal

First Riot Post
1234567 ... 8
Comment below rating threshold, click here to show it.

Wirius

Senior Member

04-05-2013

Quote:
Originally Posted by BrohannesJahms View Post
I think there's a great opportunity here to give Tribunal voters an insight into how much their individual perspectives look like the general consensus. I'm no psychologist, so I can't really comment intelligently on how effective the statistics that we do currently have are.
Wonderful conversation, I wish more forum posts were like this. I believe Dr. Lyte (who has created Mega Man) has the answer and know-how beyond both of us, and I'm glad he came in and answered your post. =D


Comment below rating threshold, click here to show it.

BigPatches

Member

04-05-2013

Quote:
Originally Posted by Lyte View Post
I agree, "Correct" and "Incorrect" aren't the ideal terms to be using in the Justice Reviews, but we had to start somewhere and iterate over time.

In the social sciences, there's a statistic called inter-rater reliability, which basically measures the level of agreement among a number of raters (voters, in Tribunal's case). Conceptually, that's the number we are presenting when we refer to "Accuracy" but we need to work on better ways to define and explain this in Justice Reviews.

I must stress that there's no "wrong" opinion in the Tribunal. Everyone has a unique opinion on what behavior is OK or not OK, and every opinion is extremely valuable to the system and the shaping of online norms over time in the community.
What is the current false positive rate on tribunal? I dunno if you are allowed to disclose that or not..


Comment below rating threshold, click here to show it.

Bellmaker33

Senior Member

04-05-2013

Quote:
Originally Posted by BigPatches View Post
What is the current false positive rate on tribunal? I dunno if you are allowed to disclose that or not..
It's like .4% or something ridiculously small.

Lyte's posted it before, somewhere.


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by Wirius View Post
Wonderful conversation, I wish more forum posts were like this. I believe Dr. Lyte (who has created Mega Man) has the answer and know-how beyond both of us, and I'm glad he came in and answered your post. =D
I think there are lots of good ideas floating around on these forums even if there are a lot of "fluff" threads and some trolls. Being a member of the community, for me, means helping to start good conversations about interesting, relevant topics related to the game.


Comment below rating threshold, click here to show it.

BackdoorPrincess

Senior Member

04-05-2013

Am I the only one to see a glaring problem with this? The fact of the matter is that conceptually, people deciding other people's fate is a good idea, but in reality, the league community is 85% Bronze and Silver, and from my experience, most of them are the type of people to spam Lux laughs, and call other people "noobs" or other derogatory terms when they die, kill them, or just in general feel like it.

While it's a fine and dandy concept to have the community judge the rest of the community, an actual set of rules decided by, say, an organization or company, where they lay down the line and decide whether or not someone is guilty, and have the community go ahead and guess the fate for giggles and pride. Not everyone agrees on what "toxic" behavior is, especially when it is just defined as "toxic".


Comment below rating threshold, click here to show it.

Lvl5syndrome

Member

04-05-2013

lol downvote warriors tried to downvote OP.


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by BackdoorPrincess View Post
Am I the only one to see a glaring problem with this? The fact of the matter is that conceptually, people deciding other people's fate is a good idea, but in reality, the league community is 85% Bronze and Silver, and from my experience, most of them are the type of people to spam Lux laughs, and call other people "noobs" or other derogatory terms when they die, kill them, or just in general feel like it.

While it's a fine and dandy concept to have the community judge the rest of the community, an actual set of rules decided by, say, an organization or company, where they lay down the line and decide whether or not someone is guilty, and have the community go ahead and guess the fate for giggles and pride. Not everyone agrees on what "toxic" behavior is, especially when it is just defined as "toxic".
It's true that not everyone agrees on what constitutes toxic behavior, and that's why it's better to not have a single code of acceptable behavior. The Tribunal lets people with varied ideas about behavior voice their opinions. Some of them get outvoted, which is to be expected.


Comment below rating threshold, click here to show it.

Linaeus

Senior Member

04-05-2013

Quote:
Originally Posted by BackdoorPrincess View Post
Am I the only one to see a glaring problem with this? The fact of the matter is that conceptually, people deciding other people's fate is a good idea, but in reality, the league community is 85% Bronze and Silver, and from my experience, most of them are the type of people to spam Lux laughs, and call other people "noobs" or other derogatory terms when they die, kill them, or just in general feel like it.

While it's a fine and dandy concept to have the community judge the rest of the community, an actual set of rules decided by, say, an organization or company, where they lay down the line and decide whether or not someone is guilty, and have the community go ahead and guess the fate for giggles and pride. Not everyone agrees on what "toxic" behavior is, especially when it is just defined as "toxic".
Some (most?) people want to exclude themselves because it is a flaw, but someone else doing it is toxic. This means most tribunal voters are still voting relatively the same as other players.


Comment below rating threshold, click here to show it.

adolfnipple

This user has referred a friend to League of Legends, click for more information

Senior Member

04-05-2013

Quote:
Originally Posted by Lyte View Post
I agree, "Correct" and "Incorrect" aren't the ideal terms to be using in the Justice Reviews, but we had to start somewhere and iterate over time.

In the social sciences, there's a statistic called inter-rater reliability, which basically measures the level of agreement among a number of raters (voters, in Tribunal's case). Conceptually, that's the number we are presenting when we refer to "Accuracy" but we need to work on better ways to define and explain this in Justice Reviews.

I must stress that there's no "wrong" opinion in the Tribunal. Everyone has a unique opinion on what behavior is OK or not OK, and every opinion is extremely valuable to the system and the shaping of online norms over time in the community.
Do you think that not having a line to cross is a good thing? You have people getting banned saying "Well I'm not sure what i'm doing wrong since there's no definition of wrong or toxic". The boundary is just everyone's opinion so when do you know when overstepping it? Most people love seeing some vilified stranger getting nailed to the stake. It's human nature to burn the witch when 95% of the people around you are yelling "She's a witch!". That's mob theory.

The people who are asking you to post your reform card are usually just interested in what punishment you got so they can troll you on the forums (most have the same exact attitudes that are displayed in these punished games) and tell you that you deserved it.

Most people spam punish too(I've done it to see what happened and i missed one case i believe). Some rioter comes out and says "Well the tribunal is 95% accurate" and the people who use tribunal think that validates what they're doing. They then spout out little snippets like "Chances are you're a bad player if you're in the tribunal because it's so accurate!" and "You had many chances to reform (to what my opinion of an untoxic player is)!" and deflect any real points against the tribunal.

The tribunal is like Communism, it's good in theory but in reality it doesn't work.

TL;DR: You should have rules to follow if you're going to allow a community to regulate itself. Otherwise you end up with burn the witch mentality.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

04-05-2013
2 of 12 Riot Posts

Quote:
Originally Posted by BackdoorPrincess View Post
Am I the only one to see a glaring problem with this? The fact of the matter is that conceptually, people deciding other people's fate is a good idea, but in reality, the league community is 85% Bronze and Silver, and from my experience, most of them are the type of people to spam Lux laughs, and call other people "noobs" or other derogatory terms when they die, kill them, or just in general feel like it.

While it's a fine and dandy concept to have the community judge the rest of the community, an actual set of rules decided by, say, an organization or company, where they lay down the line and decide whether or not someone is guilty, and have the community go ahead and guess the fate for giggles and pride. Not everyone agrees on what "toxic" behavior is, especially when it is just defined as "toxic".
This is a common misunderstanding of the Tribunal. Many players have said, "Well, Tribunal can't possibly work because the players voting are toxic themselves." But, the data shows this is simply not true.

We recently showed some data about this at MIT (the video and slides should be available shortly); basically, the data suggests that the players who vote in the Tribunal are a random sample of the population and the majority of the players are not toxic. In fact, the Tribunal population receives about 3% fewer reports per game than a random sample of the active playerbase, and over 20% fewer reports per game than a sample of players that are punished by the Tribunal.

This data is extremely critical because it suggests that the players that are voting in the Tribunal are your everyday players who want to make a difference in the community. More importantly, when we see data that Tribunal cases with homophobic slurs/racial slurs are among the most highly punished cases in the entire system, it suggests that the everyday players who play League are pretty awesome people.


1234567 ... 8