@Lyte: A small concern about the Tribunal

First Riot Post
Comment below rating threshold, click here to show it.

BackdoorPrincess

Senior Member

04-05-2013

The problem with it is the fact that you offer rewards for participating in the tribunal, even if it is just for getting a case "right". The mere presence of the reward attracts people that don't want to make the community better, they are in it for themselves. For every 1 person that actually wants to make it better, i.e. the people that make threads saying "2000 cases right AMA" or whatever, who actually really want to make league nicer, you get 20 or so people who are trying it to get ip, on it because they are mad at someone from a recent game and want to punish people( some of which honestly don't deserve a ban for their behavior, even if it is for 2 days), or someone just rolling through just pressing punish hoping to get it right.

Statistics talks about a reasonable margin of error (generally <4%) that represents people who lie or do something other than follow the tribunals rules, but when a large percentage of people aren't actually doing the tribunal actively, you get the same voices shouting the same reasoning on cases. We circle back to the fact that people who participate in the tribunal are usually people who actively report any behavior *they* deem inappropriate, and it's true, some are toxic and worth the report and following punishment.

*However*, I've also seen other accounts on the forums where someone gets punished because within whatever time period the tribunal looks for, they were reported by entire pre-made teams for saying some line that said pre-made team was shouting the whole match, but lost and decided to report the person because they won, and since the tribunal says to only look at the behavior of the person reported, they get punished. I think the tribunal is a great first step, but I see many flaws, beginning with the fact that in those cases the player wouldn't have even done something to get reported, if it wasn't for the fact that others had begun being toxic in the first place. It doesn't make it right, but it needs to be taken into account whether a player is just being a straight up jerk, or if the player was giving a rebuttal, or giving a team a taste of their own medicine.

I'd also like to take a look at the whole 3% fewer reports per game, and it honestly shouldn't be that small of a margin. If people who are voting in the tribunal are only marginally better than what a "normal" player is considered, then one has to then take into consideration that the people voting and a person being voted upon for the first time are technically on equal ground, and while it may be true that they are less toxic, there is still some toxicity present, making them not good candidates to judge others.

Lastly, and I know someone will argue it, people can and will get a higher than 50% in the tribunal by consistently pressing punish, just because of the fact that it takes multiple reports over multiple games within a set time period for someone to be eligible to even be in the tribunal, so while the people that are pardoned by them, the few that they are, could stay pardoned, perhaps the ones that are voted to be punished should then go on to an in-house tribunal, made up of actual riot employees, who are perhaps a little more elevated on the subject to decide if they are in fact worthy of being punished, that way you can make sure without a margin of error that the person is in fact guilty, just in case.


Comment below rating threshold, click here to show it.

Tadrage

Senior Member

04-05-2013

You don't get ip for voting in the tribunal


Comment below rating threshold, click here to show it.

Doct0r Phil

This user has referred a friend to League of Legends, click for more information

Senior Member

04-05-2013

Quote:
Originally Posted by Lyte View Post
This is an interesting question, but one that's more difficult to tease out in the data.
Honestly Lyte, I take it as both. Let me explain a little bit.

I work as a mental health counselor for a living. Through school, reading, and mentoring I try to help to cope with whatever life issues that they are currently dealing with. Being a counselor does not pay much but usually you do not get into that profession because you are in it for the money. So, I can say that the main reason I love my job is because I get to be a helpful part of other's lives and watch them grow into the person they desire to become.

My second favorite part from my job is what I learn. Every single client that I meet with always provides with me something I can take further in life. Whether it be a handy phrase, a life example, OR through mistakes they have made. To be honest, there are times in marital counseling where the wives complain about something their husbands did (or are doing).....and I sit there and think to myself, "uh oh, I did something like that to my wife last werk". Also, sometimes I see how bad some marriages are and I it helps motivate me to keep the fire burning in mine. This goes the same for how people handle anxiety, depression, their bosses, etc etc. Pretty much, every person has something they can teach me (whether they know it or not).

I feel this way with the Tribunal. Not only do I feel like I am trying to help build a more constructive community but I also learn what exactly can be toxic behaviors/words/phrases. Sometimes I read what people write and realize that I have said something similar to that in game. Instantly, I feel bad (and somehow wish I could appologise to those people) and it helps me to be better behaved in games. I have also noticed what phrases people "respond better" to when they are upset. Most of the time, if you just say you messed up, they will calm down and say "ok".

Anyways, before I totally get off topic, the Tribunal not only helps the community (and Riot) but it helps me too as a player in game.

So, once again, I believe the answer is both. Yes, there is no actual data towards that, and severely covered with my bias, but if something feels right, appears right, and seems to just "make sense" ....then like my boss at work says, "Go with your gut".


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

04-05-2013
5 of 12 Riot Posts

Quote:
Originally Posted by adolfnipple View Post
Why not use the entire playerbase that is punished by the tribunal instead of a sample against the entire tribunal population? It seems a bit skewed to have a sample population against an entirety population.

Also what are the average games played and type of the normal tribunal user? I'm not trying to be negative, but you can easily shift the bias of statistics to validate your point by withholding certain segments of information or using selection criteria that benefits yourself.
To clarify, it was a large random sample of Tribunal voters compared against a large random sample of the active playerbase, and a large random sample of the players punished by the Tribunal.

It's easy to shift stats to show a biased perspective, but it wouldn't be responsible to do so as scientists. Given that the player behavior team is composed of quite a few scientists with different backgrounds, we hold each other accountable when it comes to data presentation--we're pretty harsh on each other about how data is interpreted.


Comment below rating threshold, click here to show it.

Ryrok

Senior Member

04-05-2013

Quote:
Originally Posted by YoshioPeePee View Post
so it might be a chicken and the egg thing: do people who are less toxic more attracted to doing the tribunal or does doing tribunal make you less toxic? or both?
I can tell you one thing: the mere existence of a Tribunal made me a lot more aware of my conduct toward other players, and awareness led to improvement.


Comment below rating threshold, click here to show it.

Krshna

Senior Member

04-05-2013

The Tribunal is working, and Riot has the metrics to show that. If you are one of the people who says it isn't working, and then links to a reform card that is clearly toxic (to anyone with a decent morality), then you are exposing yourself as toxic.

If you do not think that the toxic behavior that people are being banned for is ban worthy, that means that you think that that kind of behavior is acceptable. You would only find that behavior acceptable if you too, were toxic, or at the least had a skewed moral compass.

No matter how much you people ***** and moan about unjust Tribunal bans, the numbers don't lie, and Riot has them in spades.


Comment below rating threshold, click here to show it.

ShawNuff

This user has referred a friend to League of Legends, click for more information

Member

04-06-2013

If our role as tribunal voters is to shape player behavior policy, why are we graded by how well we fit the existing model (% accuracy, streaks)? A progressive player would likely score lower than a "status-quot" voter, but don't we value the input of both? Likewise with the more lenient voters. AFAIK, the % you get right doesn't affect how valuable your input is to the system, but the reward system is broken to give extra reward the the status-quot voters.


Comment below rating threshold, click here to show it.

Xengre

Senior Member

04-06-2013

What percentage of Tribunal voters retain near 100% weighting (may have to look at this differently if you don't start waiting values at or near max until they prove accurate) in their votes for longer then 2 weeks? How about a month? I ask this because due to how small the total number of cases in the Tribunal is compared to the LoL population this grants significant room for accuracy to easily be skewed by spam punishment even if there were no reward. It also takes a very brief amount of time to vote. This means that even if spam punishers were judged by algorithms and their voting weight was devalued their first initial votes could skew the system enough for it to become a concern if the active and continuing voting base is not large enough and the number of votes on given cases is too few.

This isn't intended to provoke but I believe its an issue worth looking at.


Comment below rating threshold, click here to show it.

Fomorian27

This user has referred a friend to League of Legends, click for more information

Senior Member

04-06-2013

Quote:
Originally Posted by Lyte View Post
To clarify, it was a large random sample of Tribunal voters compared against a large random sample of the active playerbase, and a large random sample of the players punished by the Tribunal.

It's easy to shift stats to show a biased perspective, but it wouldn't be responsible to do so as scientists. Given that the player behavior team is composed of quite a few scientists with different backgrounds, we hold each other accountable when it comes to data presentation--we're pretty harsh on each other about how data is interpreted.
I actually have a question about this. Someone posted implying that your quote

"only about 0.7% of accounts in North America were eligible for a Tribunal case" took into account ALL accounts on NA server, both inactive and active.

Can you comment on this?


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-06-2013

Quote:
Originally Posted by ShawNuff View Post
If our role as tribunal voters is to shape player behavior policy, why are we graded by how well we fit the existing model (% accuracy, streaks)? A progressive player would likely score lower than a "status-quot" voter, but don't we value the input of both? Likewise with the more lenient voters. AFAIK, the % you get right doesn't affect how valuable your input is to the system, but the reward system is broken to give extra reward the the status-quot voters.
This hits pretty well on the core of my concerns, but I think a general rewording would really clear things up. Rather than tracking inter-rater reliability (a term which I might be misusing) and calling it "accuracy", I'd like to see a breakdown of that same stat or something closely related to it showing something like this:

x% - Overwhelming Majority
x% - Strong Majority
x% - Majority
x% - Minority
x% - Very Few
x% - Almost No One

Rather than rating the accuracy of an opinion, this shows a voter how close their views are to everyone else's. Now, it would be really fascinating to see what kinds of behavior generally swing a decision for voters and where those cases end up in the spectrum.