Statistical Analysis of Tribunal Cases

Comment below rating threshold, click here to show it.

theshim

Senior Member

08-12-2012

Quote:
Originally Posted by gnfnrf View Post
Part 3 is up in the third post of the thread.
Hmmm...you did racism there, and I know you're probably getting tired of crunching these, but do you think you could add a couple for homophobia? Maybe just "gay" and "f*g", I'd be interested to see how quick the community punishes for those.

I recall Zileas saying the punishment rates were similar between homophobia and racism, so I think it would be interesting to see how strongly it's borne out.


Comment below rating threshold, click here to show it.

FaerellG

Senior Member

08-12-2012

gnfnrf, you are an amazing person. This thread should be stickied. That's all I wanted to add to this thread.


Comment below rating threshold, click here to show it.

Mokkun

Senior Member

08-12-2012

This set is pretty interesting.

I'm randomly curious, did you check spanish/portuguese cursing when you did checks for cursing/swearing/insults?

I will second that I've seen a significantly smaller portion of non-english speakers in my games lately, but if I don't count my premade games, I'd still say it's higher than the % you're showing.


Comment below rating threshold, click here to show it.

NullAurelian

Senior Member

08-12-2012

This is really, really interesting. Kudos for doing this, it is a benefit to the community.


Comment below rating threshold, click here to show it.

gnfnrf

Senior Member

08-12-2012

Quote:
Originally Posted by theshim View Post
Hmmm...you did racism there, and I know you're probably getting tired of crunching these, but do you think you could add a couple for homophobia? Maybe just "gay" and "f*g", I'd be interested to see how quick the community punishes for those.

I recall Zileas saying the punishment rates were similar between homophobia and racism, so I think it would be interesting to see how strongly it's borne out.
What can I say... I meant to, and I was planning to, but when I actually sat down to make the lists, I completely forgot.

If I do go back and add it, and reprocess the data, I'm not sure I want to include "gay" with the other slurs. I think the false positive rate would be too high. Maybe I need to do multiple passes, but I'm really trying not to check every combination of words.

Quote:
Originally Posted by Mokkun View Post
I'm randomly curious, did you check spanish/portuguese cursing when you did checks for cursing/swearing/insults?
I did not. I know very few Spanish curses (and I have no idea which ones are popular on LoL), and no Portuguese curses at all. Also, I'm having some issues handling the diacritics in the non-English text, so I might not be able to search for them without solving that problem (does anyone know how to get unicode string conversion working smoothly in Python 2.7?)


Comment below rating threshold, click here to show it.

SUNDOGG

Junior Member

08-12-2012

Boy. Looking at the Chat log is totally subjective, Why does any one has to listen to one being abusive to others and bragging and downing others.

That totally sucks and everyone is supposed to just take it.

One can be blunt in speech or laconic,but that does not mean they are abusive.

I think people should be banned one day for calling others names.

Immediate sanctions based on fact.


Comment below rating threshold, click here to show it.

iatre

This user has referred a friend to League of Legends, click for more information

Senior Member

08-12-2012

Thanks for this. I have a question about the breakdown by champion. One would expect that there would be some natural spread in the punish rates between various champions on a completely random basis. That is, Irelia might be punished just as often as Leona is as the number of cases goes to infinity, but in your sample, you'd expect a random spread, possibly fit to a normal distribution, with most punish rates clustered near 60, but some random champs are outliers in either direction.

You'd need to perform a Bayesian analysis on the data to see if there is any real effect or just statistical noise.

I don't have my Ph.D yet, so Lyte would probably be able to provide a clearer explanation. :-p


Comment below rating threshold, click here to show it.

gnfnrf

Senior Member

08-13-2012

Quote:
Originally Posted by iatre View Post
Thanks for this. I have a question about the breakdown by champion. One would expect that there would be some natural spread in the punish rates between various champions on a completely random basis. That is, Irelia might be punished just as often as Leona is as the number of cases goes to infinity, but in your sample, you'd expect a random spread, possibly fit to a normal distribution, with most punish rates clustered near 60, but some random champs are outliers in either direction.

You'd need to perform a Bayesian analysis on the data to see if there is any real effect or just statistical noise.

I don't have my Ph.D yet, so Lyte would probably be able to provide a clearer explanation. :-p
I know I haven't been explaining it, but I have been checking my data for statistical significance as I go.

To answer your question, you are correct that I should randomly expect some variation between the top and bottom conviction rate (or any other ordered table by champion) just through chance alone.

There are two ways to test to check how likely this is to be what has actually happened. (Note: This is going to get technical.)

The first way is to build a generalized linear model of the situation, with the reported players as the dataset, case verdict as the response variable, and champion as the explanatory variable.

Because the response variable is a 2 level factor, this model is best built using logistic regression. Doing so produces not only a probability for each champion, but a p-value for each champion; the chance that a result like that (for the individual champion) could have occurred by chance.

Not all 100 champion p-values in this model indicate significance, depending on the relative popularity of the champion and how dramatically different the rate is from the mean; but at the classic p<=0.05 test, 35 do.

Not every champion listed in my top and bottom 5 show statistical significance this way, but most do (Trundle and Xerath do not, which I noted in the post, and Maoki and Skarner are borderline).

If this were a regular linear model, the next step would be to calculate an analysis of variance on the champion variable as a whole, to tell us if using champion to explain verdict is viable in concept. However, those calculations cannot be applied to logistic regressions.

The next best thing, then, is a monte carlo simulation. I can compare my real results to a block of 1000 fake results, where verdicts were assigned randomly (but in the same proportion) to the reported players. If I see similar extremes of value in the fake data as the real, I can presume that there is no real significance.

These results are in logits, not percentages. In my real data, the gap between the lowest conviction rate and the highest conviction rate is 1.23. In my simulations, the gap averages around 0.88, with a standard deviation of 0.19. This puts my real data just inside the classic 95% confidence interval.

If I were writing a biology paper, I would be sad and depressed and wouldn't be able to publish. But I'm not, I'm writing about video games in my spare time. So there you go. Some of my data are on the edge of sketchy.

(Note, I did the first check on pretty much everything I presented (build a model, make sure the terms are significant), but not the second check. Most other questions showed far more significance than this one, though, so I'm not worried.)


Comment below rating threshold, click here to show it.

ZerglingsForZion

Senior Member

08-13-2012

Awesome as always. Keep it up.


Comment below rating threshold, click here to show it.

simpsonboy77

This user has referred a friend to League of Legends, click for more information

Senior Member

08-13-2012

This is AWESOME.

Do you have any plans on releasing your source code? If not would you be willing to compile it to a library for other people to build off of?