So I started using Tribunal Satisfaction

First Riot Post
Comment below rating threshold, click here to show it.

zliplus

This user has referred a friend to League of Legends, click for more information

Senior Member

09-11-2012

There's no need for judges to consider meta-Tribunal aspects in their decisions. They got pardoned this time, and they'll be back in Tribunal again if their behaviour calls for it (and I'm going to say it probably will). That's just the way Tribunal, and reality, works.


Comment below rating threshold, click here to show it.

LargeSnorlax

Senior Member

09-11-2012

Quote:
Originally Posted by Murasaki Ryu View Post
I'm not sure why winning makes bad behavior ok, but if you say so.
It's a combination of things - Look at the entire game as a whole.

Positives:

Ashe got decent farm.
Same KDA as rest of team.
Obviously tried to play properly up until whatever time her brain broke.
1 game Case.
Won the game anyways.

Negatives:

Sold items for whatever reason (insanity?) at 40 minutes.


Comment below rating threshold, click here to show it.

zliplus

This user has referred a friend to League of Legends, click for more information

Senior Member

09-11-2012

Well, it's the end of the work day, so I'll be a little stubborn and spend more time on this hehe.

I still have to disagree that Ashe had the same KDA as her team. Her ratio is 1.2, much lower than her team (Zyra 2.0, Panth 2.35, GP 3.57, Taric 4.38). Using a half assist value, her ratio is .925 vs Zyra 1.53, Panth 1.675, Taric 2.42, GP 2.61).

That being said, I think I now understand better how a lenient person could pardon though. And as for being a devil's advocate...it's possible Ashe was pretending to be intentionally feeding in order to lure the enemy team into a chasing tactic, thus allowing her team to sneak quick pushes in.


Comment below rating threshold, click here to show it.

LargeSnorlax

Senior Member

09-11-2012

Again, I think what everyone forgets is this - A lot of things are "punishable" - The question isn't whether they're punishable or not, it's whether the slush vote of the tribunal is further towards one side or the other.

Sure, I don't like seeing "gg noobs" - The question is, how many "gg noobs" does it take before the decision is a punish? I don't like spam - How much spam is a punish? I don't like people doing stupid things - How much until a punish?

These cases have several things running for them in order for them to be pardoned (in my eyes):

- Not a significant amount of REALLY bad abuse.
- No racism (At all)
- No afking.
- No feeding (Even in the ashe case, though she apparently went crazy or something)
- Low amount of reports/cases
- Decent scores for all the reported

All of those push it further towards a pardon vote. The more of each of those you see, the more likely it'll be a punish. The more games in a case, the more likely a punish. The more of a person being a dick, the more punish.

If you had plucked cases that had 5 game loads with tons of swearing and abuse I'd agree with you. All this really proved is that the tribunal can be and is lenient on the cases without a lot of weight to them.


Comment below rating threshold, click here to show it.

LargeSnorlax

Senior Member

09-11-2012

Quote:
Originally Posted by zliplus View Post
Well, it's the end of the work day, so I'll be a little stubborn and spend more time on this hehe.

I still have to disagree that Ashe had the same KDA as her team. Her ratio is 1.2, much lower than her team (Zyra 2.0, Panth 2.35, GP 3.57, Taric 4.38). Using a half assist value, her ratio is .925 vs Zyra 1.53, Panth 1.675, Taric 2.42, GP 2.61).

That being said, I think I now understand better how a lenient person could pardon though. And as for being a devil's advocate...it's possible Ashe was pretending to be intentionally feeding in order to lure the enemy team into a chasing tactic, thus allowing her team to sneak quick pushes in.
(Same here) - End of work day.

It's possible ashe was doing that, and it's possible ashe was doing a lot of stuff. If Ashe was deliberately feeding, her team would not have won - And chat seems to reflect that. Even though she went nuts, her team won the game, so she had to have been doing something distracting or otherwise helpful, or the team would've ended the game with a loss.

The ratios are enough to make me think she just died a few more times in a brawlfest game, and who knows what happened at the end? I don't - Hence, gotta go with the information I have, and that isn't enough for me to press punish.


Comment below rating threshold, click here to show it.

ZerglingsForZion

Senior Member

09-11-2012

I'll give my breakdown for each case:


Case1

Quote:
Xin Zhao[00:15:05] :help
Xin Zhao[00:15:16] :noob

Xin Zhao[00:21:40] :BACK
Xin Zhao[00:21:44] :noob

Xin Zhao[00:27:18] :rat
Xin Zhao[00:27:19] :stupid

Xin Zhao[00:29:01] :retarded noobs

Xin Zhao[00:31:20] :this
Xin Zhao[00:31:21] :team
Xin Zhao[00:31:21] :wtf
Xin Zhao[00:31:23] :retarded
Xin Zhao[00:31:24] :noobs
Xin Zhao [All][00:31:29] :mi team never atack
Xin Zhao[00:31:36] :stupid team

Xin Zhao[00:34:45] :noob
Xin Zhao[00:34:49] :kata
Xin Zhao[00:34:51] :wtf

Xin Zhao[00:39:48] :team
Xin Zhao[00:39:49] :stupid
Xin Zhao[00:39:52] :retarded
Xin Zhao[00:39:53] :team

Xin Zhao[00:45:35] :kata
Xin Zhao[00:45:36] :stupid
Xin Zhao[00:45:37] :kata

Xin Zhao[00:54:15] :stupid rat
Xin Zhao[00:54:22] :and kaya

Xin Zhao[00:58:58] :stupid rat

Xin Zhao[00:59:23] :report rat
That's 13 minor infractions in my book from the first game and another 16 in game two. Under 3 in each game might have earned this person a narrow pardon But 29 insults against your team in 90 minutes is 1 every 3 minutes not to mention the spamming.

Case2
Quote:
Ashe [All][00:29:03] :you ruined everything
Ashe [All][00:29:07] :you ruined everything
Ashe [All][00:29:12] :YOU ****ING. RUINED EVERYTHING.
Ashe [All][00:29:15] :YOU PIECE OF ****ING. ****.

Ashe [All][00:50:57] :so bad rammus
Ashe [All][00:50:59] :you suck
Rammus [All][00:51:12] :i own your team hush
Ashe [All][00:51:24] :you got owned, then i put a ward on your dead body, *****.

Ashe [All][00:51:59] :doesn't change the fact that there was a ward on your dick.

Ashe [All][00:52:47] :get waped woob
Ashe [All][00:52:51] :wu wucking waggot
Last quote was clearly get f***ed noob, you f***ing f****t.
Enough there for me not to want to play with this person again if I was in the game so I would punish.

Case3
Quote:
Twitch [All][00:05:30] :noobs

Twitch[00:18:56] :gay

Twitch [All][00:22:16] :noob
Twitch [All][00:22:23] :go die

Twitch [All][00:22:42] :fagreal

Twitch [All][00:24:30] :gg noobs
I am very close to an autopunisher when I see slurs and this game has 2. Plus just bad attitude. Sure the second game was clear but you click punish if you find enough in total not on average. I'm not too worried about this one slipping through, it's clear that this will be back from the 1st game and might even be rehabilitated when they are made aware of their behavior.

I don't use win/loss or even scores when determining a pardon or punish when it's not a feeder report.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

09-11-2012
1 of 3 Riot Posts

Tribunal users tend to be more lenient than Riot staff--this has been shown time and time again in the data When designing and managing a system like the Tribunal, we have to use a common framework used in some academic fields called signal detection theory. What the heck am I talking about?

For any given case in the Tribunal, there are 4 possible outcomes:

Outcomes
1) A "Hit" -- The case was 'true' toxic and was punished by the Tribunal
2) A "Miss" -- The case was 'true' toxic and was pardoned by the Tribunal
3) A "False Positive" -- The case was 'true' innocent and was punished by the Tribunal
4) A "Correct Rejection" -- The case was 'true' innocent and was pardoned by the Tribunal

In a given system, we can choose to optimize along any of these outcomes but they interact with each other--if we focus on Outcome 1, we might lose some fidelity on Outcome 3. At Riot, Team Player Behavior & Justice has decided to focus on reducing the number of "false positives" because our core focus is to avoid banning or inconveniencing innocent players. All of our features and ban sweeps uphold this philosophy; however, a consequence of this philosophy is that we sometimes 'let go' players that may deserve punishment. The players we do punish though are highly likely to deserve it.

However, because the Tribunal is heavily based on frequency and probabilities, toxic players that are 'let go' often are sent back to the Tribunal fairly quickly and are promptly and correctly banned in follow-up visits to the Tribunal. This is why the Tribunal ultimately works, but also why we want to improve the efficiency and accuracy to make the system faster over time.


Comment below rating threshold, click here to show it.

KevinDelMarr

Senior Member

09-11-2012

Lyte, do you think there will ever be a third option, instead of Pardon or Punish it's something like "Player did some negative things that should be noted, but nothing too far out of average?". I know that "Punish" doesn't necessarily mean Ban, but that's how I feel some of the time. Often I'll face the conundrum with the in-betweens Summoners where if I say Punish I will feel like they will get a heavier handed punshiment then what I feel is deserved, but if I say Pardon it will be both too far from the majority to get IP or that the player "got away".

This does call into question the power of Riot in determining punishments and whether players are in the Tribunal for the IP or to bring true justice (I see the IP as a "bonus", and in the end I'm trying to maximize both ends of the IP/Justice spectrum)

Edit: I just read your post after writing this, which does clarify Riot's stance on my subject. I still feel conflicted in some of my decisions because they aren't always so cut and dry


Comment below rating threshold, click here to show it.

LargeSnorlax

Senior Member

09-11-2012

As mentioned - All this really proves is that the tribunal is lenient.


Comment below rating threshold, click here to show it.

Murasaki Ryu

This user has referred a friend to League of Legends, click for more information

Senior Member

09-11-2012

Quote:
Originally Posted by Lyte View Post
Tribunal users tend to be more lenient than Riot staff--this has been shown time and time again in the data When designing and managing a system like the Tribunal, we have to use a common framework used in some academic fields called signal detection theory. What the heck am I talking about?

For any given case in the Tribunal, there are 4 possible outcomes:

Outcomes
1) A "Hit" -- The case was 'true' toxic and was punished by the Tribunal
2) A "Miss" -- The case was 'true' toxic and was pardoned by the Tribunal
3) A "False Positive" -- The case was 'true' innocent and was punished by the Tribunal
4) A "Correct Rejection" -- The case was 'true' innocent and was pardoned by the Tribunal

In a given system, we can choose to optimize along any of these outcomes but they interact with each other--if we focus on Outcome 1, we might lose some fidelity on Outcome 3. At Riot, Team Player Behavior & Justice has decided to focus on reducing the number of "false positives" because our core focus is to avoid banning or inconveniencing innocent players. All of our features and ban sweeps uphold this philosophy; however, a consequence of this philosophy is that we sometimes 'let go' players that may deserve punishment. The players we do punish though are highly likely to deserve it.

However, because the Tribunal is heavily based on frequency and probabilities, toxic players that are 'let go' often are sent back to the Tribunal fairly quickly and are promptly and correctly banned in follow-up visits to the Tribunal. This is why the Tribunal ultimately works, but also why we want to improve the efficiency and accuracy to make the system faster over time.
That's fair. I can understand focusing on minimizing number three to keep innocent players from getting punished. It's probably the best way to go about it, at least I feel it is. Is there anything we can do as Judges to help increase the number of hits?

Also maybe you should not implement a system where people see each case they vote on and the out come. Might cause you more trouble than it's worth. >.>

<3<3<3<3<3