@Lyte Referencing "Teamwork OP"

First Riot Post
Comment below rating threshold, click here to show it.

BelligerentGnu

Senior Member

09-12-2013

Quote:
Originally Posted by Riot NeuroCat View Post
However, when presenting our cool findings, one thing we have to consider is the target audience. Coming from academia, I'm all for transparency and clarity of results, but our target audience is players, not academics. What would be more interesting for the average player to consume - a multipage wall of text or a high-quality animated video? Even though we don't deep dive into the stats in the video, we definitely consider the appropriate controls and confounds when running the initial analyses that generate the results.
Maybe. But for those of us who a) find this stuff fascinating and b)want to contribute as we can, a high-quality animated video is just frustratingly opaque.

The teamwork OP vid is designed to change behaviour across the general player population, sweet, I get that. But I don't see why that stops you from posting a multi-page wall of text to the forums and letting those of us read it who wish to.

Actually, I'd lay money on the fact that you've already produced internal memos that would do the job nicely, that you could slap online with minor editing.


Comment below rating threshold, click here to show it.

DarkslayerG

This user has referred a friend to League of Legends, click for more information

Senior Member

09-12-2013

Quote:
Originally Posted by FrozenXylaphone View Post
Did you know the most famous psycological experiment is often falsely reported on? That is the Milgram Experiment. The conclusions reached in psycology books are false conclusions based on a single instance of the experiment.
There is a woman who dedicated her life to researching this and was shocked to find completely different results when looking at the other instances that Milgram conducted his same experiment.
She found that the results where not consistent at all. And yet, this is still taught in psycology classes across the world as if it was true.

http://www.npr.org/2013/08/28/209559...bedience-study

This was on NPR.
Shocking how science can be misapplied to reach false conclusion then become fact albeit not actually being true at all.


"Over 700 people took part in the experiments. When the news of the experiment was first reported, and the shocking statistic that 65 percent of people went to maximum voltage on the shock machine was reported, very few people, I think, realized then and even realize today that that statistic applied to 26 of 40 people. Of those other 700-odd people, obedience rates varied enormously. In fact, there were variations of the experiment where no one obeyed."
? She does not even attempt to explain why she thought the statistics were wrong. In addition, Milgram did performed several variations with different factors and they had different obedience rates. This is common knowledge taught in all intro psych classes. Please do not just read an interview and believe it as fact.


Comment below rating threshold, click here to show it.

Coredor

Senior Member

09-12-2013

Quote:
Originally Posted by Lyte View Post
As we all know, correlations do not imply causation.

When we first worked on in-game tips, we posted a few correlations that were interesting to us; however, the hypothesis was whether reading the tips themselves had a strong impact on player behavior in the game. If you are interested in some of the impact of these tips, you can take a look at our GDC talk here:

http://gdcvault.com/play/1017940/The...Shaping-Player

Since that time, we've done a variety of analyses on teamwork and sportsmanship and how they impact your odds of winning. None of the analyses on their own are 100% conclusive, but together, we felt like it was compelling data.

For example, in one analysis we took a set of players and codified their chat logs using an advanced language model in Champion Select Lobby before games have even started. By coding the chat logs as neutral, positive or negative then correlating these to in-game performance or end-of-game reports and Honor, we can see the powerful effects of sportsmanship-like behavior before the game has even started.

In another example, you can take 5 sets of players that have markers of sportsmanlike behavior such as a low report rate, a high Honor rate, and so on and so forth. You can then compare these sets to 5 sets of players that are considered neutral (or toxic) and see which sets of players tend to win more games.

In yet another example, you can take every behavioral combination of a team and see how win rates differ among them; for example, how often does a team of 5 sportsmanlike players win? What about 4 sportsmanlike players and 1 toxic player? What about 4 sportsmanlike players and 1 neutral player? What about 5 neutral players? You can then look at the behavioral composition of a team and see whether they outperform the matchmaking system. So on and so forth.

When we talk about Champion Select in the future, we'll be digging into some of these analyses in more detail, but we wanted to start the discussion about how OP teamwork is
These results are still bias though. You can manipulate the controls of what chat is required to be considered 'sportsmanlike' so that it matches up with higher wins.

In your last two examples that still doesn't show anything more than correlation. I want to know what controls you used. Because from here it seems like your jobs all depend on your research yielding tangible results and therefore the majority of you are biasing the tests.


Comment below rating threshold, click here to show it.

Zeremist

Junior Member

09-12-2013

@Lyte I watched your video, and what you said about "priming" and the psychological effect of colors got me curious, so i went and googled it to learn more. If red is the color of aggression, and stimulates more negativity, why is all chat red? Is it possible that changing all chat color to something more.... peaceful? would reduce negativity? Maybe something like green, gray, violet, yellow, or brown. Or, put it in the options where you can change the color of your chat.
Wasn't sure which thread to post this in, but i didn't think it was worth making a new thread, so hopefully you see this and respond i guess.


Comment below rating threshold, click here to show it.

Coredor

Senior Member

09-12-2013

Quote:
Originally Posted by Riot NeuroCat View Post
I wish I could help give you a more detailed explanation of these statistics, but I came in towards the end of the analysis which produced the results cited in the video. As Lyte said, we're working on more really cool teamwork-related analyses, so when the time comes, perhaps I can shed some light on those results.

However, when presenting our cool findings, one thing we have to consider is the target audience. Coming from academia, I'm all for transparency and clarity of results, but our target audience is players, not academics. What would be more interesting for the average player to consume - a multipage wall of text or a high-quality animated video? Even though we don't deep dive into the stats in the video, we definitely consider the appropriate controls and confounds when running the inital analyses that generate the results.
There's no reason you can't do both. Posting spreadsheats and things you already have only takes about 20 seconds and provides tangible proof. Making a youtube video may hit a wider audience but all it really gives us is anecdotes and your word that being positive might help.


Comment below rating threshold, click here to show it.

Krest

Junior Member

09-12-2013

Love the Player Behavior team's work. This stuff is as fascinating as League itself! Keep up the good work!


Comment below rating threshold, click here to show it.

Eagleye749

Senior Member

09-12-2013

Quote:
Originally Posted by Riot NeuroCat View Post
I wish I could help give you a more detailed explanation of these statistics, but I came in towards the end of the analysis which produced the results cited in the video. As Lyte said, we're working on more really cool teamwork-related analyses, so when the time comes, perhaps I can shed some light on those results.

However, when presenting our cool findings, one thing we have to consider is the target audience. Coming from academia, I'm all for transparency and clarity of results, but our target audience is players, not academics. What would be more interesting for the average player to consume - a multipage wall of text or a high-quality animated video? Even though we don't deep dive into the stats in the video, we definitely consider the appropriate controls and confounds when running the inital analyses that generate the results.
It's easy for you to call us uneducated because you're trying to prove a point that is already undisputed. Who the hell would say that teamwork is bad for your winrate?

Anyways we just question the numbers you came up with and respectfully believe that teamwork does not make you win THAT much more.


Comment below rating threshold, click here to show it.

thumbnail9

Senior Member

09-12-2013

You see we just want the stats, the data, the meat, not your message. We've heard the message countless times and now we want the proof.

If Lyte ever wanted to publish a paper on his masterful work here done at riot, which I can't see why he wouldn't, it needs to be released anyways. The worst thing that could happen would be an Anova says that it doesn't even make an actual statistical difference.

We're just asking for what any person in the scientific community would ask you to do.


Comment below rating threshold, click here to show it.

FelRain

Senior Member

09-12-2013

Eh, saw the video. It was pretty obvious. Being positive actually doesn't make you win more games. It's just that being negative severely reduce your chances of winning.

If someone on my team is super nice and apologizes for their mistake, it doesn't mean anything. It's not going to make me want to group up or push objectives more or less. The person is just nice. I'm going to play the same. Being positive/neutral makes them less likely to rage though, but you could argue that they're a nice person and being positive is who they are and are less likely to rage because they're a positive person, not because they're positive in game.

If someone on my team is raging? I might stop caring. I might throw the game. I might not team up making our team lose. I might consider our team a lost cause and vote for a surrender. And I'm sure I'm not the only one out there. I want this guy to lose. Also, people pay more attention to the chat than the game at that point.


Comment below rating threshold, click here to show it.

Lyte

This user has referred a friend to League of Legends, click for more information

Lead Social Systems Designer

Follow RiotLyte on Twitter

09-12-2013
6 of 7 Riot Posts

Quote:
Originally Posted by Amuq the Native View Post
Oh please, don't be patronizing. I know you're better than that.
You could make a could video, with links to the academic paper at the end or in the video comments. You're going to write the academic paper anyway (or else the study is a waste !), so why not make it accessible to everyone who wants ? I for one would be very interested in it.
There's certainly interest in writing a paper and if we pursue that path, we'll try to make it easily available. Many of the studies we've been conducting have been collaborative studies with very well-known academic institutions, and they are working on papers on some of this material.

It takes time to write a paper, get it peer-reviewed, iterate on reviews, and finally be published (unless you publish in one of the lesser known journals or go open-access). For some papers it could take 1 year or more--why would we wait on cool content such as the video for over a year, just to tie it to the paper? It's not really debatable that most players want to just enjoy the video content; only a subset of players would be interested enough in the academic paper or raw data. The two things (video and paper) don't need to be tied together, and if papers on League are available in the future, you can bet they will be easy to access.

A lot of players may question the data, and that's very cool. I love being challenged and challenging other scientists to improve the quality bar of research in the video game industry. It's unfair for players to accuse the scientists on staff of fabricating data, or being so incompetent that they don't know the difference between a correlation versus a causal effect--the team here is a strong team, and they are far more critical than the average scientist in academia. In fact, many meetings are held just to debate the rigor of the data and whether we're being responsible and specifically not ever doing simple marketing spins on numbers.

We've shared more data than ever before on systems like the Tribunal. As scientists, we believe in transparency and the ability to replicate our findings; we believe in this so strongly we happily shared our experiments and data at the Game Developer's Conference so that all game studios to join us in the player behavior discussion.

In saying this, we've heard the feedback about wanting some more explanation on stats in videos such as Teamwork OP. We'll see what we can do if we do future videos. Maybe I'll write a design blog to go with the video to explain some of the data (similar to my post in this thread, but maybe with a bit more detail)--we'll have to see if it makes sense.