Upcoming Changes to the Tribunal

First Riot Post
Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

09-13-2012
61 of 92 Riot Posts

Quote:
Originally Posted by FlyinTeddyBearz View Post
whats going to happen for the people who used the tribunal before this change? will they still get their ip?
Yes, players will get the IP for all cases done until the end of September.


Comment below rating threshold, click here to show it.

StarBubbles

Senior Member

09-13-2012

Awesome!! Are there gonna be tournaments for the Tribunal ELO? XD In all seriousness, I love the upcoming features.

"Report them for being born." My fav quote of all time.

Oh Tribunal, how I love thee.


Comment below rating threshold, click here to show it.

Bjornsides

Junior Member

09-13-2012

My question has been ignored, so I will re post it:

This is all well and good, but why are you focusing your attention on creating a more accurate tribunal system when you can't report people for trolling pre, and post game? The vast majority of "toxic players" that I see now days only troll at these times, and it seems to me that the system is incomplete, and why would you try to optimise something that you may have to change completely anyway?

If possible I would like a red post reply to this issue because it's something that is of massive concern to me, and the reason that I stopped doing tribunal in the first place.


Comment below rating threshold, click here to show it.

Apollinarius

Senior Member

09-13-2012

Quote:
Originally Posted by McAzazel View Post
The basic economic idea is that individuals care about their material wellbeing (besides other goals). Any activity that is rewarded more highly, will be followed more likely. It then gives that incentives alone wouldn't be enough to motivate blood donations, but might tip the balance by lowering the costs of donating.

There are empirical evidence abound, that incentives can increase prosocial activities -- in particular when it comes to charitable donations. Matching (you donate x and we'll donate x) for instance, will usually show a positive impact; since the effective price of your donation becomes cheaper you should consequently donate more. There have also been field experiments done on whether the chance to win a monetary prize would raise charitable contributions -- which it did.

Upton's studies on blood donations and incentives are marred by a mistake in the selection process. The experiment was conducted thusly:
Individuals were called and asked to sign up for a blood donation. The treatment group was promised compensation if they donated blood while the control group was not. The dependent variable of the study was to then measure whether a donor followed through on their pledge and donated blood.

As you can probably tell, Upton's mistake was in the selection of the treatment group and the control group, because he failed to randomize it properly. Instead of announcing the compensation when recruiting individuals to sign up, he should have recruited all donors without the promise of compensation. One could then randomly select half the participants, call them again, and tell them that, if they follow through on their pledge, they would receive a (monetary) reward. (Upton's study -- flawed as it may have been -- did suggest crowding-out of motivation.)

There have been more recent studies on whether incentives affect donations, but overall they point to either no effect or a small positive effect. The "concern" that financial incentives might attract donors with worse risk factors has not been studied, since all tests (that I'm aware of) have used prescreened blood donors.

People that are and want to appear prosocial (blood drives in the US, for example) react differently. The tests mentioned earlier were all done in relative privacy, but a blood drive have strong social components to them. In these situations, incentives make it attractive for less motivated individuals to donate blood as well; but worse yet, they may attract individuals that simply do it for money, thus further lowering the image reward (intrinsic) from donating blood. Image concerns definitely affect the effectiveness of incentives. If you're interested in reading more about that, I suggest Ariely et al.'s study, "Doing good or doing well? Image motivation and monetary incentives in behaving prosocially."

To cut what could be a long story short: incentives can (temporarily) increase blood donations, especially if they're done in private. In particular, incentives have a significant impact on donors who have a lower propensity to donate. Psychological theories of intrinsic motivation raise the possibility that individuals' motivation can be permanently undermined by the use of incentives, but the one study that tracked this shows no such effect.

Here (in Sweden) you get 50SEK ($about 7.5) when donating blood. However if you forgo the money, the hospital will instead buy a plushie, which is given to the next kid that is hospitalized.
Your suggested method for Upton's study is invalid because your starting pool is tainted with people who would have done it for free in both groups.

In the Tribunal, you are told up front whether there will be a reward or not before you go in (sign up to be part of the study).

Previously, the Tribunal's situation was that everyone was told there would be rewards. People came, some caring about the reward, and others caring about making the community a better place. A large number found it easier to stomach the time required to make the community a better place because there was some reward.

Now, the Tribunal's situation is the opposite. Everyone is told there would be no rewards. Nobody who was doing it primarily for rewards will continue. Those who were only doing it to help the community will remain. Of those who liked the rewards but helped the community too, some will go and some will stay.

Riot isn't trying to increase Tribunal participation. That's a widely held misconception. They have significant overcapacity. They are trying to increase the quality of the judgements so that people who deserve to be punished get punished on their first visit in the Tribunal.

Let's say that the Tribunal has a 70% chance of arriving at a correct verdict. That means that 30% of the people that should have been punished did not. What's worse, it means that 9% of the people that should have been punished in the first group didn't even get punished the second time through the Tribunal.

By removing the voters who were focused more on IP rewards, Riot is hoping to increase the Tribunal accuracy from 70% (my made up number) to 95% (their target may be even higher). This means that only 5% of people that should have been punished on their first visit were pardoned and only 0.25% of people that should have been punished on their first visit made it to a third one before punishment. That's an 80% reduction in false pardons on the first pass and a 97% reduction in false pardons on the second pass.

To put it in real world terms, if you were seeing 30 toxic players in one day before, you would see only 5 after the change, and none of those 5 would be left after their next Tribunal visit.


Comment below rating threshold, click here to show it.

Taldanis

Junior Member

09-13-2012

Quote:
Originally Posted by Bjornsides View Post
My question has been ignored, so I will re post it:

This is all well and good, but why are you focusing your attention on creating a more accurate tribunal system when you can't report people for trolling pre, and post game? The vast majority of "toxic players" that I see now days only troll at these times, and it seems to me that the system is incomplete, and why would you try to optimise something that you may have to change completely anyway?

If possible I would like a red post reply to this issue because it's something that is of massive concern to me, and the reason that I stopped doing tribunal in the first place.
Lyte has answered this question over and over. The answer is that they've done research, and the people that troll in pre-game/post-game chat end up getting reported for stuff in-game anyway.


Comment below rating threshold, click here to show it.

Apollinarius

Senior Member

09-13-2012

Quote:
Originally Posted by Bjornsides View Post
My question has been ignored, so I will re post it:

This is all well and good, but why are you focusing your attention on creating a more accurate tribunal system when you can't report people for trolling pre, and post game? The vast majority of "toxic players" that I see now days only troll at these times, and it seems to me that the system is incomplete, and why would you try to optimise something that you may have to change completely anyway?

If possible I would like a red post reply to this issue because it's something that is of massive concern to me, and the reason that I stopped doing tribunal in the first place.
This has been answered before. While the pre game chat is on their radar (post game not so much, because leaving post game chat is so easy), Riot's experience is that those who troll pre-game troll sufficiently in-game that they get caught anyway. Maybe not as fast as they would be otherwise, but they still get caught and punished.


Comment below rating threshold, click here to show it.

OhBoyItsaMegaman

Senior Member

09-13-2012

Quote:
Originally Posted by RiotNinjaTabby View Post
If your vote matches the final verdict on the case, it is considered correct in terms of streak and accuracy.
Thank you very much for the response, but it doesn't answer my question because I don't know what the term final verdict means. When the Tribunal verdict is Pardon on a miss, and when Riot's verdict is Pardon on a false positive—are both of these considered final verdicts? It's confusing because in one case, the final verdict comes from Riot and is always correct, but in another case it comes from the Tribunal and can be incorrect.


Comment below rating threshold, click here to show it.

Apollinarius

Senior Member

09-13-2012

Quote:
Originally Posted by Maxximuss1011 View Post
I had a question, how often are IP rewarded to ppl who use the tribunal i've been doing it for a week now and was just wondering if i was going to get contacted or what?
It's awarded once the cases you voted on are resolved. Sometimes, that takes a very long time. I hadn't done tribunal in over a month, and before that it was fairly sporadic with a couple of cases a day and then on Sunday I got over 250 IP all at once.

It used to be fairly regular, once a week payouts, but it's become much more of a lump sum affair.


Comment below rating threshold, click here to show it.

Mokkun

Senior Member

09-13-2012

So this is how it's getting implemented for the moment.
I like the concept of seeing our accuracy rates, and the "toxic days" bit is actually pretty cool.
Never done this based on IP either, it's never been enough to care about.

I am not a major fan of a ladder, because this isn't a competition to me. I am both very competitive, and one of those weird people who rarely play ranked, because it might affect my rating negatively. On the other hand, I probably won't make the ladder with my punish rate, so it may not matter.

Which brings me to another point. I've been hitting punish probably 80-90% of the cases I don't skip for the entirety of the time I've used the Tribunal. This would put me at something like 30% off the Tribunal average punish rate. Why has the system never kicked me out as being a "spam punisher."

(I don't believe that I spam punish, I would re-vote 98% of my cases the same way over again, but I also know that I'm not voting the community standard.)


Comment below rating threshold, click here to show it.

Apollinarius

Senior Member

09-13-2012

Quote:
Originally Posted by Had0ken View Post
Thank you very much for the response, but it doesn't answer my question because I don't know what the term final verdict means. When the Tribunal verdict is Pardon on a miss, and when Riot's verdict is Pardon on a false positive—are both of these considered final verdicts? It's confusing because in one case, the final verdict comes from Riot and is always correct, but in another case it comes from the Tribunal and can be incorrect.
It is the verdict Riot acts on. If Riot pardons, the final verdict is pardon. Even if every single person on the Tribunal voted punish but Riot decided to pardon, the "final verdict" is pardon and none of those who voted punish would get IP (old system) or correct judgement points (new system).