Better measures to ensure Tribunal members are qualified?

First Riot Post
Comment below rating threshold, click here to show it.

Vahlen

Senior Member

01-06-2013

Quote:
Originally Posted by Lyte View Post
The first question I would ask the player behavior team is, "Is there a problem with inaccurate judges in the Tribunal?" If the answer is no, then I'm not sure trying to find a solution for a non-existent problem is the right approach.

Here's an interesting discussion point though: We all see cases that sometimes deserve to be punished, but are pardoned due to a coincidental set of very lenient reviewers. Does that make them wrong? Not necessarily. Every player comes to the Tribunal with their opinion and their beliefs on what is appropriate behavior and what isn't--every vote matters and makes a difference in our community. This is why we don't believe "Accuracy" in the Justice Reviews is the ideal metric, but it works for now. We do value every single vote, regardless of 'agreeing' with the community majority or not.

To us, abusive reviewers are those who vote Pardon or Punish randomly without reviewing the materials, or those who intentionally try to game the system by spamming Pardon or Punish without reviewing the details. Is there a problem with abusive reviewers? Not currently.
What about the few cases that have been posted on these forums about false punishes? I think one even has around 800 upvotes.......


Comment below rating threshold, click here to show it.

GatorHatur

Senior Member

01-07-2013

Quote:
Originally Posted by Titan9900 View Post
"Is there a problem with inaccurate judges in the Tribunal?"

Yes.
- people banned by tribunal don't like inaccurate and inconsistent judges.
- players don't like inaccurate and inconsistent judges. League now has players who refuse to even speak out of fear of being reported.
- Tribunal members don't like inaccurate and inconsistent judgements. It undervalues the tribunal and the decisions of good judges.

I don't think its about "abbusive reviewers" , its about the accuracy of the tribunal. At the moment tribunal is considered a joke. You have players with 100% accuracy rates , inconsistent decisions and people constantly questioning there bans.

I want to see the tribunal in a position in which both banned and current players actually have faith in the tribunals decisions.
A tribunal by definition is a court of justice, do you think criminals have faith in a system that puts them behind bars? No... nor would I expect those who are banned by Riot's system. The rules that are broken to get into tribunal are no different then everyday laws that we are already accustomed to following... why because we know we could be legally prosecuted... The real problem is the general attitude that you should be able to do whatever it is you want on the internet and say what you want with anonymity, without consequence... That's why people have no "faith" as you call it. Its outside the norm for internet culture. The sooner you can see and connect this relationship you will realize the Riot's tribunal is doing quite well and is no more perfect than any other real life justice system.

Bottom line, Internet culture is broken and in bad taste. As soon as they start behaving themselves the sooner there will be less toxicity.

Another note I wonder what the number the % of people in the US are that go to jail but then never go back? or what percent of the population gets perma-jailed(banned) as they would say. Then you can compare real numbers and see how well the system is working.


Comment below rating threshold, click here to show it.

Fudouri

Senior Member

01-07-2013

not a big fan of jail analogy. Pretty sure every toxic player understands the jail time line and would never cross it. In fact, their point would be that they are being punished like being in prison for somethign that would never get them a prison sentence normally.


Comment below rating threshold, click here to show it.

Noblepeasant

This user has referred a friend to League of Legends, click for more information

Senior Member

01-07-2013

Quote:
Originally Posted by Snido View Post
I think the problem with the tribunal is what Riot has told the community about it: That if you are on trial you deserve to be on trial. So the reviewers assume guilt and look for any tiny reason to punish you instead of assuming innocence and trying you fairly based on that. Truthfully I believe only about 1/2 the cases put on tribunal deserve to be punished but we see nearly double that.

And no I have never been punished/warned or anything.
According to Lyte (October 2012) 60% of the cases in NA tribunal are punished.


Comment below rating threshold, click here to show it.

SchierkeWiking

Senior Member

01-07-2013

Quote:
Originally Posted by Noblepeasant View Post
According to Lyte (October 2012) 60% of the cases in NA tribunal are punished.
According to some data I've seen and my own obersvation, the current rate seems closer to 80%.

Of course, it could just be due to the small sample.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

01-07-2013
2 of 2 Riot Posts

Quote:
Originally Posted by SchierkeWiking View Post
According to some data I've seen and my own obersvation, the current rate seems closer to 80%.

Of course, it could just be due to the small sample.
I haven't checked the latest numbers, but if almost all 80% of those cases deserved to be punished, is this still a problem in the Tribunal? Phrased differently, if 99.98% of the cases that are punished are 'deserved' or 'warranted,' what's wrong with the fact that 80% of cases are punished?


Comment below rating threshold, click here to show it.

Arcolyte

This user has referred a friend to League of Legends, click for more information

Senior Member

01-07-2013

Quote:
Originally Posted by Lyte View Post
I haven't checked the latest numbers, but if almost all 80% of those cases deserved to be punished, is this still a problem in the Tribunal? Phrased differently, if 99.98% of the cases that are punished are 'deserved' or 'warranted,' what's wrong with the fact that 80% of cases are punished?
The problem is the disconnect between what people expect and the actual facts of the situation. Intellectually I KNOW that 80% is a huge improvement over previous systems. Accounting for sheer volume at this point. However, I still want/feel that it should be closer to what I perceive to be the 'maximum' level of reasonable punish rate. Which I know will never be achieved(truly) nor would it be possible anyways, especially without increasing collateral damage(false positives.) Yet, it is still something sought after. Especially when the perceived difference is large enough to imply that if we could just cut it down by 25% bringing it to ~85%, then 20% and more and so on. Whittling it down until the person is satisfied(unlikely) or the cost exceeds the benefit(possibly compromised ideals.)

So I can/do convince myself to accept this, but I still feel the occasional twinge of desire.


Comment below rating threshold, click here to show it.

Super Explosion

Senior Member

01-07-2013

Quote:
Originally Posted by Lyte View Post
We all see cases that sometimes deserve to be punished, but are pardoned due to a coincidental set of very lenient reviewers. Does that make them wrong? Not necessarily.
It does quite make them wrong.

Take into consideration how you arrive at your own words: "deserve to be punished"

Tribunal reviewers exist to apply a set of laws, which if the person is actually in violation of, must yield an affirmative ("guilty") signal.

Just about the only other contention is that while the person is in violation of the laws, the laws themselves are suboptimal ('unjust', 'bad')-- which is the implication of agreeing with a 'nullification' ("lenient") signal.


Comment below rating threshold, click here to show it.

Fudouri

Senior Member

01-07-2013

Actually tribunal really is just judging "is this person a jerk".

As such, its very different from laws (which have to be quite concrete specifically for this).

By making the summoner's code pretty abstract, they implicitly have decided that any given case doesn't have a right or wrong.

In real life, that is why you have various degrees of murder etc, because each becomes relatively concrete (though even in those cases, a lot is open to interpretation)

btw, what the heck is a nullification signal? any link i can read up on it?


Comment below rating threshold, click here to show it.

Super Explosion

Senior Member

01-07-2013

Quote:
Originally Posted by Fudouri View Post
Actually tribunal really is just judging "is this person a jerk".

As such, its very different from laws (which have to be quite concrete specifically for this).

By making the summoner's code pretty abstract, they implicitly have decided that any given case doesn't have a right or wrong.

In real life, that is why you have various degrees of murder etc, because each becomes relatively concrete (though even in those cases, a lot is open to interpretation)

btw, what the heck is a nullification signal? any link i can read up on it?
Jury Nullification, Googlable.

The Summoner's Code is a set of laws that can be judged upon using intelligible criteria set to Riot's standard of appellate judgement.

If someone is actually in violation based upon the Tribunal's guidelines, whatever they may be, and reviewers do not apply the law, that is a 'nullification' signal-- a decision explicit or implicit that the law should not be applied in face of the evidence.

This is mostly to the understanding of legal systems and decision making-- "judging "is this person a jerk"" may be fairly accurate.

Nonetheless, if you look more closely, there are criteria and commonalities that go into that judgement.

And thus, understanding that, implications of a particular opinion on whether or not a set of reviewers is 'correct'.