Understanding the Tribunal

First Riot Post
Comment below rating threshold, click here to show it.

GON the Mage Guy

Senior Member

10-27-2012

Quote:
Originally Posted by OxfordKiller View Post
Nerd Psychology is something he knows nothing about. Let's not forget the classic 26% line. Statistics...right!
Hello Tribunal forum troll, how are you today?


Comment below rating threshold, click here to show it.

LarcyBrown

Senior Member

10-27-2012

hey do we get to have a faq on the meanings of perma ban until year 2277 and what year means what exactly in general? im just curious on that...really curious


Comment below rating threshold, click here to show it.

CowFriend

Member

10-27-2012

I love you, Lyte.


Comment below rating threshold, click here to show it.

theshim

Senior Member

10-27-2012

Quote:
Originally Posted by Xander756 View Post
Just throwing out scholarly terms such as "crowdsourcing" and "random sampling" is only going to make you sound like you know what you're doing to the untrained ear. Crowdsourcing is not a good system with many critical flaws. You have a Ph.D so I expected you would have known this already, though right?

Crowdsourcing data can be horribly inaccurate for several reasons, one of which is malicious intent of those involved to skew the data. In this case, it would be the intent for Tribunal judges to intentionally punish people because they themselves got picked on.
If this were the case you'd expect far fewer pardons and far, far stricter punishes. The argument (most commonly expressed as "butthurt nerds get angry when they lose and go to Tribunal and punish everyone") would definitively result in punishspam, which doesn't occur in significant amounts, according to Lyte. Given the amount of pardons that have people coming in and saying "Riot how the hell did this guy get pardoned look at the card" - malicious Tribunal voting seems to be having very little effect.
Quote:
Another is that those involved often don't have the requisite knowledge or skill to properly complete the work (one theory as to why this is, is because the people who could actually do this work well don't have the time to commit to work for token wages). This also applies to the Tribunal because you have 12 year olds deciding cases as if they were a Supreme Court Justice!
Ah yes, the 12 year old argument. Summoner level 20 is plenty to have a working knowledge of the game, and honestly do you need any more? Not to mention that there are plenty of older and wiser people in the community. Yes, some 12 year olds may sometimes vote in the Tribunal - but the average 12 year old won't, and certainly won't come back. The people who do tend to be people who actually care, and who take the time, and who read the cases.
Quote:
These have been the problems people have been complaining about all along. Why would someone as highly educated as yourself need to be explained the problems with crowdsourcing on an almost daily basis?
You're misattributing blame here. The question is less "why, if you're so educated, do you have to keep explaining it" and more "why, if people want to know about this, do they not look up what's already been said dozens of times and instead post more and more topics about it?" And that's either laziness or people not knowing that Lyte has posted about them repeatedly - which he has. Seriously, this one's laughable.
Quote:
Another major problem with crowdsourcing is the immorality of having people complete work for token wages. Yet Riot doesn't even pay people to do this work for them. They toss a little bit of IP their way. In fact, I think you even ended that. Now you just have people working for you for free. Gee, I wonder why this company has an F rating with the BBB...?
Because permabanned people complained against the injustice of the system and the BBB doesn't do research on every company to fully understand them? Seriously, read the guidelines, and read the complaints. Riot's not going to unban accounts nor return their RP or money, and people flag these as unaddressed or unsatisfactory.
Quote:
Yes let's teach kids to allow people to bully them without retaliating. This is a great life lesson, Lyte.
Stating "he started it" is a legitimate defense is a much worse one.
Quote:
And claiming "if you retaliate, both of you will be banned" is not a correct or accurate statement. The Tribunal does not have the power to ban other people in the game, only the person who was reported. Therefore, if you were retaliating to someone else and then YOU get reported YOU are the one who is banned. Not them. You then have to just hope the other person also got hit but you'll never know.
But Lyte does. The people who scream and rage at you tend to do it an awful lot! And they tend to have their own Tribunal cases, and tend to get banned because holy hell, they're jerks! Of course, this is all predicated on people reporting them. If someone flames people back in game but never reports, they're both making games highly unpleasant and not helping to clean up the community.

That said a notification along the lines of "Summoner, X people you have reported have received action against their accounts! Thanks for helping clean up the community!" would still be nice and would help soothe some ruffled feathers, so to speak. I'm just not convinced it's a priority.
Quote:
This is the only thing I've agreed with you on thus far. Low skilled players should not get banned. That being said, the Tribunal and reporting system are still broken in this regards. People report players for doing badly for "intentional feeding." Then someone looks at a 2/15 game and says "oh well, they MUST have been feeding" and clicks punish. It seems people do not really understand what INTENTIONAL feeding means.
This has come up a lot. Virtually every post on the Tribunal forums has several replies of people saying they only punish if it is clear and obvious feeding (enemies in allchat talking about it, boots and zeals, 0/28/0). That said, they will also punish if they weren't feeding, but were flaming or abusing their team. It would be nice if people were more accurate with their categories, and Lyte has said he wants to refine them/remove some, so I'm looking forward to that.


Comment below rating threshold, click here to show it.

Xander756

Member

10-27-2012

Wow this is one of the biggest failure of a defense posts I've ever seen. In your fanboyish attempt to defend the Tribunal, you literally started spouting nonsense. You even began to try to argue that there are not flaws in crowdsourcing systems. Why don't you look up "crowdsourcing" on wikipedia. There is a very large section called "criticisms."

Quote:
If this were the case you'd expect far fewer pardons and far, far stricter punishes. The argument (most commonly expressed as "butthurt nerds get angry when they lose and go to Tribunal and punish everyone") would definitively result in punishspam, which doesn't occur in significant amounts, according to Lyte. Given the amount of pardons that have people coming in and saying "Riot how the hell did this guy get pardoned look at the card" - malicious Tribunal voting seems to be having very little effect.
Can't get much stricter than bannings for 1 game. Can't get much more frequent than an 80% punishment rate.

Quote:
Ah yes, the 12 year old argument. Summoner level 20 is plenty to have a working knowledge of the game, and honestly do you need any more? Not to mention that there are plenty of older and wiser people in the community. Yes, some 12 year olds may sometimes vote in the Tribunal - but the average 12 year old won't, and certainly won't come back. The people who do tend to be people who actually care, and who take the time, and who read the cases
This is completely baseless. You have no idea what you are talking about when you make claims that "the average 12 year old won't." Based off what data do you draw this conclusion? I'd love to see the average age of Tribunal judges.

Quote:
You're misattributing blame here. The question is less "why, if you're so educated, do you have to keep explaining it" and more "why, if people want to know about this, do they not look up what's already been said dozens of times and instead post more and more topics about it?" And that's either laziness or people not knowing that Lyte has posted about them repeatedly - which he has. Seriously, this one's laughable.
No, what are you even talking about here? The problems with crowdsourcing are real and extremely well known. As I mentioned before, go look up the "crowdsourcing" article on wikipedia. These criticisms have nothing to do with League of Legends or the Tribunal in of themselves, but the fact is they are the EXACT SAME PROBLEMS that people regularly bring up in the Tribunal forum. The fact that Lyte needs these explained to him when he has a Ph.D is what's laughable.

Quote:
Because permabanned people complained against the injustice of the system and the BBB doesn't do research on every company to fully understand them? Seriously, read the guidelines, and read the complaints. Riot's not going to unban accounts nor return their RP or money, and people flag these as unaddressed or unsatisfactory.
Again, I was explaining well-known issues with crowdsourcing systems. You can't defend them. Stop trying. The fact you are attempting to makes you appear to be nothing more than a Tribunal apologist, eagerly defending any criticism thrown its way, no matter how valid.

Quote:
Stating "he started it" is a legitimate defense is a much worse one.
And yet it is a positive defense in many criminal and civil cases. I wonder why the "he started it and I was defending myself" excuse can be used in a court of law but yet the standards are supposedly too high on an online video game? Hahaha.

Quote:
But Lyte does. The people who scream and rage at you tend to do it an awful lot! And they tend to have their own Tribunal cases, and tend to get banned because holy hell, they're jerks! Of course, this is all predicated on people reporting them. If someone flames people back in game but never reports, they're both making games highly unpleasant and not helping to clean up the community.

That said a notification along the lines of "Summoner, X people you have reported have received action against their accounts! Thanks for helping clean up the community!" would still be nice and would help soothe some ruffled feathers, so to speak. I'm just not convinced it's a priority.
Doesn't matter if we does or doesn't. We're talking about in specific Tribunal cases. You will never know if that other person was disciplined. From your perspective, only you got punished. Maybe the Tribunal should have the ability to punish people who made reports?


Comment below rating threshold, click here to show it.

Kashral

This user has referred a friend to League of Legends, click for more information

Junior Member

10-27-2012

Quote:
Originally Posted by Lyte View Post
Lately, I've been receiving a lot of e-mails about the same questions and complaints about the Tribunal. I'm working with WookieCookie, RiotSeb and statuskwoh to add these to the Tribunal FAQs but we would love to discuss them here first.

1) What's the point of having a language filter if I can get banned for 'adult' language?
a) We aren't out to punish players who use adult language; in fact, very few players are ever banned for saying stuff like "f***, I missed that skill shot" or "damn, f***ing close game!" However, the language filter is not an excuse to verbally abuse someone in the game. When you verbally abuse or harass someone by calling them "f***ing ******" or "n*gger noob", that's when you've crossed the line and will be punished by the Tribunal. I've said this before and I'll say it again, we aren't trying to clean online games of adult language; we're trying to reform or get rid of jerks who like to verbally abuse and harass other players.

2) I only have 1 bad game in my Tribunal Reform Card, why am I banned???
a) The Tribunal verdict is not based on a 'majority' rule; there is also no requirement that all games have to be toxic for it to warrant a suspension. Depending on the case, sometimes 1 game is all that is needed to show that the player deserves a suspension. To Tribunal reviewers, decisions like these are up to you. When you view a Tribunal case, the questions you should ask yourselves are, "Is this player's behavior appropriate in League of Legends?" and "Would I want to play with this player?"

3) This player should have been banned, but he was pardoned. The Tribunal is broken!
a) The Tribunal was inspired by 'crowdsourcing' research. When I was still a student, I worked with labs that developed games such as "Foldit" that took advantage of crowdsourcing to find solutions to extremely tricky tasks--when you read this research you truly appreciate how accurate crowdsourcing systems can be. In fact, the Tribunal is far more accurate than traditional Player Support.

However, the Tribunal isn't perfect. The design is based on statistical probability and random sampling and every system like this has outliers. This means that sometimes, a player has an egregious Tribunal case but is still pardoned because he happened to get a lenient set of voters; however, if the player is truly toxic, he actually is re-sent back to the Tribunal in a handful of games and is usually accurately punished in his second visit to the Tribunal.

Because the Tribunal is designed to identify frequency and consistency of toxic behavior, this is why it doesn't take into account the severity of the offense. If a player is extremely toxic he will quickly get banned multiple times by the Tribunal until he is permabanned; in fact, extremely toxic players end up permabanned significantly faster than players who are 'less severe.' In the future, we may want to brainstorm ways to improve on the Tribunal so that it does take into account severity and adjusts punishments accordingly... but we're not there yet.

4) That player in my game was far worse than me. Why am I banned???
a) I'll keep this one simple: retaliation isn't OK. Reacting to a toxic player by being toxic just incites that player to be even worse--it also ruins the game for up to 8 other players. It is far, far worse to watch two people yelling at each other all game than it is to watch 1 player ranting into blank space. Almost everyone in the game can find someone out there that behaves worse than they do--but so what? Finding someone worse doesn't justify your behavior or justify retaliation. We expect more out of our players. What players often don't realize is... those other players in your game that you thought were worse? Nearly 100% of the time, those players were banned even before you were. If someone is being toxic, just report them and move on. If you retaliate, both of you will be banned. It's really that simple.

5) Low skill players should be banned from the game! The Tribunal should only ban Leavers/AFK and intentional feeders, those are the only types of players that really ruin games!
a) No. Just no. Jerks ruin games. League of Legends is for everyone whether they are 500 Elo or 2500 Elo. It's not a game for jerks.
What about the same people that just keep being banned over and over. I have one person I will not name names but myself as well as my brother always seem to get this one entity, he is always negative, uses foul language and gives our position away to the enemy team.

Every single game we have been in with this player he does this, we have reported him no less than (40) times over the past year and we are still seeing him at least once a week. I am sure he has been banned a lot, he even goes on to say how he is an employee of Riot and it's just one of his many alt accounts.

Also, my brother has been doing the newly formed Tribunal just to test it out; needless to say it seems like it's worse now. He has done about 53 maybe 54 cases, three or four of the cases he punished, rightfully so just for him to be the strong minority and be overruled.

Its like people did not even read the chat logs, whats going to be done about that as well? When you put a system like this in and it has a chance to show a leader you make it a contest, and like all contests people will do anything to obtain first, even cheat. I strongly feel at least 50% of the current Tribunal members do not even read the chat logs.

What are your thoughts on these questions and concerns?

PS, What would you think about the removal of all IP when your account gets suspended? I think it would be a lot better when people actually lose more than just time.


Comment below rating threshold, click here to show it.

theshim

Senior Member

10-27-2012

Quote:
Originally Posted by Xander756 View Post
Can't get much stricter than bannings for 1 game. Can't get much more frequent than an 80% punishment rate.
Where are you getting 80% from? Last we heard it was in the 60% neighborhood. And people still do not get banned for one game. The Tribunal case may only consist of one, but it takes a lot more than one to end up there. That it still forms cases with only one game is a weakness of the Tribunal, I concede, and one that could stand to be fixed - although, interestingly enough, one-game cases tend to be the most pardoned, with a 62% rate according to gnfnrf's incredibly cool topic.

I want to point out that I feel the system has weaknesses, holes, and flaws. I just think I have a pretty good handle on what they are and why they are there, and many are - as an old professor of mine used to say - "features, not bugs".
Quote:
This is completely baseless. You have no idea what you are talking about when you make claims that "the average 12 year old won't." Based off what data do you draw this conclusion? I'd love to see the average age of Tribunal judges.
Based on...a pretty good working knowledge of 12 year olds and teens? But I concede I have no data here, I'm operating from personal experience and knowledge of psychology.

I'm also wondering how we could assemble that information, as there's no age tracking in LoL as far as I know. A survey included in the Tribunal set to randomly pop up when people logged in asking them a few questions would be very interesting, and I think worthwhile.
Quote:
No, what are you even talking about here? The problems with crowdsourcing are real and extremely well known. As I mentioned before, go look up the "crowdsourcing" article on wikipedia. These criticisms have nothing to do with League of Legends or the Tribunal in of themselves, but the fact is they are the EXACT SAME PROBLEMS that people regularly bring up in the Tribunal forum. The fact that Lyte needs these explained to him when he has a Ph.D is what's laughable.
Ohhhhh. See, when I read your post, I thought you were accusing Lyte of being stupid for having to explain these things to people, rather than for having to have people explain crowdsourcing flaws to him. My bad, especially with the laughable comment, and I apologize.
Quote:
Again, I was explaining well-known issues with crowdsourcing systems. You can't defend them. Stop trying. The fact you are attempting to makes you appear to be nothing more than a Tribunal apologist, eagerly defending any criticism thrown its way, no matter how valid.
Go read the BBB page for Riot. It's interesting.

And the fact that crowdsourcing has issues doesn't mean it's useless, or that the issues can't be accounted for in design. Your accusation that the Tribunal is worthless and broken because of crowdsourcing issues - especially with the absolutist "you can't defend them, stop trying" is every bit as imperious and closed-minded as you claimed me to be.
Quote:
And yet it is a positive defense in many criminal and civil cases. I wonder why the "he started it and I was defending myself" excuse can be used in a court of law but yet the standards are supposedly too high on an online video game? Hahaha.
Because there's a difference between someone yelling at you and someone initiating actual physical violence. There really is. If someone actually assaults you, yes, you can defend yourself, because otherwise you're going to suffer physical harm. If someone is a jerk in an online game, it is much more reasonable to expect restraint. If you fight back against the person who assaults you, you're protecting yourself. What are you protecting in League by flaming someone back, besides your ego?
Quote:
Doesn't matter if we does or doesn't. We're talking about in specific Tribunal cases. You will never know if that other person was disciplined. From your perspective, only you got punished. Maybe the Tribunal should have the ability to punish people who made reports?
Punishing reporters is a bit much, though I still firmly believe we could use a flagging function for obviously false or malicious reports to send them to Riot and/or cut down said reporters' credibility. And again, I think even a simple notification like the one I stated in the last post would be helpful. Either way, Lyte's statement was not inaccurate, even if you don't personally know about it.

On the other hand, let me play Devil's Advocate here for a moment - what purpose does that notification serve? I suppose it does grant confidence to people to know that the Tribunal is working, but the way you phrased it made it sound a lot more like you want personal vindication that you got rid of this guy - and I don't think that's really beneficial knowledge.


Comment below rating threshold, click here to show it.

ThereRnoIDs

Member

10-27-2012

Sorry dude, you "can" get banned for 1 game no matter how many times you guys deny it.

If you play a few games and lost a few, there's no doubt a few false reports will wind you in the the tribunal especially solo queue.
Then the tribunals will go, wow this guys a jackass in game 1 only out of his 5. Punished.

Have you seen all those people saying it's wrong to be an ******* even if it's1 out of 50 and deserves a ppunish? No matter how many times I see this, if you say gg noobs or show aa bit of unsportsmanship in a total of 25-50 hours of game play, you can get a banned...

And the tribunal still trying to justify my ban for being negative and miserable.
http://na.leagueoflegends.com/board/....php?t=2712939


Comment below rating threshold, click here to show it.

gnfnrf

Senior Member

10-28-2012

Quote:
Originally Posted by theshim View Post
Where are you getting 80% from? Last we heard it was in the 60% neighborhood. And people still do not get banned for one game. The Tribunal case may only consist of one, but it takes a lot more than one to end up there. That it still forms cases with only one game is a weakness of the Tribunal, I concede, and one that could stand to be fixed - although, interestingly enough, one-game cases tend to be the most pardoned, with a 62% rate according to gnfnrf's incredibly cool topic.
Just so you know, it seems that the actual vote threshold for punishment was moved with the latest Justice Review rollout, so my statistics from that thread may not apply any more. I will run another batch in a bit, once the Justice Review has settled into standard practice.


Comment below rating threshold, click here to show it.

Yeeehawww

Senior Member

10-28-2012

Thank you so much Lyte, I'm glad to see you guys pointedly address that toxic behavior, even in retaliation, isn't ok. I've seen so many people thinking otherwise.