@Lyte: A small concern about the Tribunal

First Riot Post
123456 ... 8
Comment below rating threshold, click here to show it.

Wirius

Senior Member

04-05-2013

This is an intelligent OP. Lets take a second here and probably realize that Riot most likely thought they knew what the majority of the playerbase thought was offensive. Its not rocket science. They want the tribunal to be a teaching tool, and probably thought that the majority would most always be correct, as the majority generally is when it comes to what is offensive to players.

If you are a minority in what is offensive, then generally, you really aren't correct in your understanding of what is offensive. In general, this is probably true, except in borderline cases.

So the idea of implying you are correct would in general be the truth. However, in an outlier, this would seem disingenuous. The thing is, the offensive people of our community are a minority, the target of the tribunal. I believe educating the minority community on what is generally offensive behavior is a net positive, and can only encourage people to follow these general norms. You will always have outliers, but the key is, they are outliers. Which means, they generally don't happen.

Further, it is important to motivate people to actually try in tribunal cases. Having a measuring tool is a great incentive for many people, and many people want to be on the "right" side. As such, I see no problem with the wording, and that it only helps the tribunal ultimately as a teaching tool for young or immature people.


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by Bellmaker33 View Post
@OP - You really just have to take a step back and realize that "Correct" vs. "Incorrect" only means that you correctly or incorrectly voted in comparison to the majority of voters, and the decision that came of the votes.

It's a stretch, I know.

I actually agree with you. I think tracking should be sorted as:

Overwhelmingly Majority > Strong Majority > Majority > all the others, with numbers or percentages below them.

And then there should be the "Punished" by number, "Pardoned" by number, "Toxic Days Prevented" by number.

My thoughts. I kinda disagree, but more agree that it should be reformatted simply based on connotations.
I still believe that a good Tribunal user is one who makes the most honest decision they can, but words have power. I think "correct" and "incorrect" ought to be changed to something else, and I'm not really sure how to handle the Longest Streak stat. Your suggestion is quite good, though. I like the idea of being able to see how my own opinions about behavior either match or don't match the gist of the rest of the voting population.


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by Wirius View Post
This is an intelligent OP. Lets take a second here and probably realize that Riot most likely thought they knew what the majority of the playerbase thought was offensive. Its not rocket science. They want the tribunal to be a teaching tool, and probably thought that the majority would most always be correct, as the majority generally is when it comes to what is offensive to players.

If you are a minority in what is offensive, then generally, you really aren't correct in your understanding of what is offensive. In general, this is probably true, except in borderline cases.
Agreed. Most BM and toxicity is relatively easy to identify and pretty well agreed upon by a wide range of people. There are always going to be grey areas, and the Tribunal does a pretty damn good job of handling those cases, I'd say.

Quote:
So the idea of implying you are correct would in general be the truth. However, in an outlier, this would seem disingenuous. The thing is, the offensive people of our community are a minority, the target of the tribunal. I believe educating the minority community on what is generally offensive behavior is a net positive, and can only encourage people to follow these general norms. You will always have outliers, but the key is, they are outliers. Which means, they generally don't happen.
Not going to go into a discussion of cultural relativism here, but part of the beauty of a system like this one is that more than one perspective is considered in the outcome of a case. That said, no one perspective is necessarily the "right" one for dealing with more ambiguous cases. It's important to steer people away from thinking in absolutes, because absolutes are what the Tribunal's detractors and community trolls fall back on to justify the perception that it's a bad system.

Quote:
Further, it is important to motivate people to actually try in tribunal cases. Having a measuring tool is a great incentive for many people, and many people want to be on the "right" side. As such, I see no problem with the wording, and that it only helps the tribunal ultimately as a teaching tool for young or immature people.
Incentives are great. I just think that another option might be better than using sticky objective wording.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

04-05-2013
1 of 12 Riot Posts

I agree, "Correct" and "Incorrect" aren't the ideal terms to be using in the Justice Reviews, but we had to start somewhere and iterate over time.

In the social sciences, there's a statistic called inter-rater reliability, which basically measures the level of agreement among a number of raters (voters, in Tribunal's case). Conceptually, that's the number we are presenting when we refer to "Accuracy" but we need to work on better ways to define and explain this in Justice Reviews.

I must stress that there's no "wrong" opinion in the Tribunal. Everyone has a unique opinion on what behavior is OK or not OK, and every opinion is extremely valuable to the system and the shaping of online norms over time in the community.


Comment below rating threshold, click here to show it.

Bellmaker33

Senior Member

04-05-2013

Quote:
Originally Posted by Lyte View Post
I agree, "Correct" and "Incorrect" aren't the ideal terms to be using in the Justice Reviews, but we had to start somewhere and iterate over time.

In the social sciences, there's a statistic called inter-rater reliability, which basically measures the level of agreement among a number of raters (voters, in Tribunal's case). Conceptually, that's the number we are presenting when we refer to "Accuracy" but we need to work on better ways to define and explain this in Justice Reviews.

I must stress that there's no "wrong" opinion in the Tribunal. Everyone has a unique opinion on what behavior is OK or not OK, and every opinion is extremely valuable to the system and the shaping of online norms over time in the community.
I would pay money to borrow your brain, Matrix-style, so I can learn all the cool stuff you know.

My weekend will now be spent reading up on inter-rater reliability.

/hug Lyte.


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by Lyte View Post
I agree, "Correct" and "Incorrect" aren't the ideal terms to be using in the Justice Reviews, but we had to start somewhere and iterate over time.

In the social sciences, there's a statistic called inter-rater reliability, which basically measures the level of agreement among a number of raters (voters, in Tribunal's case). Conceptually, that's the number we are presenting when we refer to "Accuracy" but we need to work on better ways to define and explain this in Justice Reviews.

I must stress that there's no "wrong" opinion in the Tribunal. Everyone has a unique opinion on what behavior is OK or not OK, and every opinion is extremely valuable to the system and the shaping of online norms over time in the community.
Do you have any ideas about how you might change things going forward with respect to this issue? I really like the suggestion that I quoted in the OP. It would be interesting to see how my opinion compares to the rest of the voters. I know that number is reflected in part with the Accuracy rating, but as far as I can tell, that number doesn't take into account the size of the majority, does it?


Comment below rating threshold, click here to show it.

Wirius

Senior Member

04-05-2013

Quote:
Originally Posted by BrohannesJahms View Post

Not going to go into a discussion of cultural relativism here, but part of the beauty of a system like this one is that more than one perspective is considered in the outcome of a case. That said, no one perspective is necessarily the "right" one for dealing with more ambiguous cases. It's important to steer people away from thinking in absolutes, because absolutes are what the Tribunal's detractors and community trolls fall back on to justify the perception that it's a bad system.



Incentives are great. I just think that another option might be better than using sticky objective wording.
O.k., so I think you and I agree on almost everything. Your key issue is the absolutist position of claiming, "correct". However, there is also the argument to be made to not go too relativist. In not taking a stand that the majority is "correct" tribunal haters could easily argue that the majority isn't correct, oppression of the minority, etc.

I think it is important to take a stand somewhere in this issue and state, that yes, the majority is in fact "correct" in this case, as what the majority decides, is what happens. This may encourage others to start asking what it is the majority considers. And for the extreme minority that really are in their own isolated bubble, this can be a wake up call to really examine why they're in the minority, and many times, why it is so extremely minor. And isn't reexamining and questioning your own choices ultimately a good thing to ensure you're really making sound choices?

In the end, I don't see a problem with the wording as being too strong, considering the audience and the cases being judged. In fact, I see the wording ultimately being helpful to the community, and expressing a statement that encourages better behavior, and more careful judging.

I guess the question is, if we changed the wording to be more what you liked, would it continue to have the benefits I've expressed? Are these benefits even an issue?


Comment below rating threshold, click here to show it.

BrohannesJahms

Senior Member

04-05-2013

Quote:
Originally Posted by Wirius View Post
O.k., so I think you and I agree on almost everything. Your key issue is the absolutist position of claiming, "correct". However, there is also the argument to be made to not go too relativist. In not taking a stand that the majority is "correct" tribunal haters could easily argue that the majority isn't correct, oppression of the minority, etc.

I think it is important to take a stand somewhere in this issue and state, that yes, the majority is in fact "correct" in this case, as what the majority decides, is what happens. This may encourage others to start asking what it is the majority considers. And for the extreme minority that really are in their own isolated bubble, this can be a wake up call to really examine why they're in the minority, and many times, why it is so extremely minor. And isn't reexamining and questioning your own choices ultimately a good thing to ensure you're really making sound choices?

In the end, I don't see a problem with the wording as being too strong, considering the audience and the cases being judged. In fact, I see the wording ultimately being helpful to the community, and expressing a statement that encourages better behavior, and more careful judging.

I guess the question is, if we changed the wording to be more what you liked, would it continue to have the benefits I've expressed? Are these benefits even an issue?
I think there's a great opportunity here to give Tribunal voters an insight into how much their individual perspectives look like the general consensus. I'm no psychologist, so I can't really comment intelligently on how effective the statistics that we do currently have are.


Comment below rating threshold, click here to show it.
Comment below rating threshold, click here to show it.

Last Spark

Senior Member

04-05-2013

What happens when people that was banned or is in the blacklist reports you? The report will be the same as a "clean" player reports you or has a different impact?


123456 ... 8