NA and EU stability update

First Riot Post
Comment below rating threshold, click here to show it.

sonicdeathmonk

Vice President of Operations

03-01-2014
9 of 11 Riot Posts

Quote:
Originally Posted by NightmareVirus View Post
I believe the server move to optimize East coast needs to be priority. I believe that can be agreed on. Not champions. Not skins. Not Events. The general ability to ENJOY THAT GAME period. Needs to be top priority. NO one on the east coast is happy about the move to Oregon. It's caused a general increase in ping for all of us. It's made some of us almost unable to play. Why ruin something that was already working 'decently' alright to where not alot of people were complaining. Now you've stirred up a hornets nest of trouble.

What were the thoughts that made you guys consider moving the servers MORE west and even North some Sonic?
Moving servers more central for all NA players is a priority. As I mentioned earlier, truck loads of servers are purchased, we just need to lock down the data center contract and get the Internet circuits installed. In the meantime the servers near Portland, OR will see connection improvement for many players as the direct connections to the major ISPs and telcos get installed (more are getting hooked up every week). We were maxed out of quality space, Internet bandwidth, and power in the Los Angeles area so Oregon was the best choice at the time to get large amounts of servers up and running (Facebook, Apple and a few other major players are an easy drive away), it is technically only 7-9 msec further away from the LA area (if LA was the better place, it is not for all players) but we need to optimize the routing and circuit mix to really achieve the best possible connection for all players to the Oregon located servers. The team is working long hours to get this done.

In the last few months many players were likely getting first routed to LA then to Portland through San Jose or sometime weirdly through Seattle. Now lots of players are taking a much more direct route to just San Jose and then Portland if you are not in the Southwest USA or you will likely pass through Seattle then to Portland if you are in the Northwest USA or coming out of most of Canada. So ping time for many players should have gone down in the last month (but not for all). This will continue to optimize for more players as we get more telcos and ISPs directly connected.

Another team is focused on getting game servers spun up more centrally which will then allow Oregon to become the fall back site if things go into the ditch in a bad way (something we have not had as option in the past). So net, Oregon is a temporary hop which has allowed us to significantly expand the number of game servers and other resources we have needed to help stabilize NA. Next step is to optimize ping time for as many players as we can in Oregon and then a more central to NA location which we will tell you about as soon as we lock it down.

-sonicdeathmonk


Comment below rating threshold, click here to show it.

sonicdeathmonk

Vice President of Operations

03-01-2014
10 of 11 Riot Posts

Quote:
Originally Posted by FlynnOFlenniken View Post
I'd like to take a moment to recall my experiences for the past few years regarding my internet connection and League.

I've always played with a wired connection. Not wireless.

When I first began playing League towards the end of season 1, I had Verizon internet. Not Fios. Just the high speed internet. Back then my connection to everything, even things not League, was pretty spotty. I'd disconnect frequently whether I was playing League or surfing the web. I normally had around 90 ping in game, but sometimes I'd have massive lag spikes. Been too long to remember exact numbers, but it'd be in several hundreds or thousands.

I believe at some point in season 2 I switched to Suddenlink internet. For several months at least I had pretty great internet. Averaged about 60 ping in League. Websites would load super fast. Started to play a lot more pvp hoping to get better at the game and eventually try my hand at ranked.

Probably around either the end of season 2 or beginning of season 3 I noticed my ping had gone up a little to about 70 ping. Didn't think too much of it.

Add a 2-3 months, 80 ping. Another couple months, 90. Some more, 100 ping. By the beginning of season 4 I was starting to average about 110 ping and the fluctuations were getting pretty bad.

Now I'm getting 130-150 ping with massive spikes to 1000-5000.

What's really weird is that the problem doesn't seem contained to League anymore for me. I'm having a hard time just browsing the web now. Websites take forever to load sometimes, or they'll load improperly and I'll have to refresh once or twice.

I can't tell what the problem is. Not sure if it's my isp or maybe my house has bad wiring? Maybe my isp is getting hit more than most by all the ddos lately? Guess I should probably contact my isp and ask some questions. See if they might be able to figure things out.

My vague location is somewhere between Houston and Dallas in Texas.

Maybe it's paranoia, but I feel like my problem has gradually gotten worse as this reflection ddos business has grown.

Figure any information might help you guys.

Edit: Also, there was a Saturday like one or two weeks ago I had 50 ping. I don't know WHAT happened, but it was pretty amazing getting to play with that. That was the only good day among a lot of bad ones though.
We are concerned that as popular video services like Netflix, Youtube and Hulu have grown the Internet congestion at peak play time (which is also tends to overlap with peak video watching time) has started to interfere with all kinds of Internet traffic throughout the world. In the last 9 months we have been working to build more dedicated links to the ISPs most players are connected to which has taken many months to negotiate and install the required equipment. Just in the last month some players should be seeing significant improvement (though the DDoS pain has not helped along the way). As more connections get put in place every week, we expect more players to see improvement. Lots of work going on in this area and we will share more details as things get turned up.

-sonicdeathmonk


Comment below rating threshold, click here to show it.

Shentok

Senior Member

03-01-2014

My problem is that in 99% of the games I now get connected to the Oregon servers instead of the LA servers. It sucks. I hate having 60~70 ping when I used to have 25~30 ping every match. I know it's great having 60 ping comparatively, but I don't even know how people play on 60 ping. Everything feels delayed and it feels like my reaction times suck because of it.


Comment below rating threshold, click here to show it.

sonicdeathmonk

Vice President of Operations

03-01-2014
11 of 11 Riot Posts

Quote:
Originally Posted by ShikigamiSlayer View Post
Are there any plans to have uplinks to RCN? That's my current ISP in PA.
We are looking at the ISPs and Telecoms that players are connecting from and will be working to directly connect to as many as we can (not all have the ability to directly connect to us). We are also hooking into several peering exchanges in North America which should bring lots more smaller ISPs just 1 hop away from game servers (once the traffic gets to the peering exchange that is). We expect players will see improvement almost every week as your particular ISP gets hooked up, but it will take several more months to cover most players with our new direct connection strategy.

-sonicdeathmonk


Comment below rating threshold, click here to show it.

Sefam

This user has referred a friend to League of Legends, click for more information

Senior Member

03-01-2014

Quote:
Originally Posted by sonicdeathmonk View Post
Moving servers more central for all NA players is a priority. As I mentioned earlier, truck loads of servers are purchased, we just need to lock down the data center contract and get the Internet circuits installed. In the meantime the servers near Portland, OR will see connection improvement for many players as the direct connections to the major ISPs and telcos get installed (more are getting hooked up every week).
You guys keep using the word "Central" and that worries me. I drew on a map to help the centralization efforts.

http://i.imgur.com/XTeZTRI.png

(Okay, that was a cheap jab)

It's like you guys avoid using the word "East" or something. It's fairly laughable when the ping I get connecting to EUW is about the same as what I get connecting to NA. The fact that Oregon is a "fallback" is also pretty darn concerning; ping fluctuates depending on whether or not I'm routed to Portland or LA.

This whole "USEast servers" thing has been going on for a long time and other video games that were released in the 90ies had and some of them still have USEast servers.


Comment below rating threshold, click here to show it.

Valenten

Senior Member

03-01-2014

Quote:
Originally Posted by sonicdeathmonk View Post
Moving servers more central for all NA players is a priority. As I mentioned earlier, truck loads of servers are purchased, we just need to lock down the data center contract and get the Internet circuits installed. In the meantime the servers near Portland, OR will see connection improvement for many players as the direct connections to the major ISPs and telcos get installed (more are getting hooked up every week). We were maxed out of quality space, Internet bandwidth, and power in the Los Angeles area so Oregon was the best choice at the time to get large amounts of servers up and running (Facebook, Apple and a few other major players are an easy drive away), it is technically only 7-9 msec further away from the LA area (if LA was the better place, it is not for all players) but we need to optimize the routing and circuit mix to really achieve the best possible connection for all players to the Oregon located servers. The team is working long hours to get this done.

In the last few months many players were likely getting first routed to LA then to Portland through San Jose or sometime weirdly through Seattle. Now lots of players are taking a much more direct route to just San Jose and then Portland if you are not in the Southwest USA or you will likely pass through Seattle then to Portland if you are in the Northwest USA or coming out of most of Canada. So ping time for many players should have gone down in the last month (but not for all). This will continue to optimize for more players as we get more telcos and ISPs directly connected.

Another team is focused on getting game servers spun up more centrally which will then allow Oregon to become the fall back site if things go into the ditch in a bad way (something we have not had as option in the past). So net, Oregon is a temporary hop which has allowed us to significantly expand the number of game servers and other resources we have needed to help stabilize NA. Next step is to optimize ping time for as many players as we can in Oregon and then a more central to NA location which we will tell you about as soon as we lock it down.

-sonicdeathmonk
Hey there! Thanks for keeping us int he loop on this now I noticed you mentioned Oregon in there for Northwestern/canada areas but what about us in the South Central/South eastern US? We are still expiriencing rather High ping and I was just wondering If yall are maybe looking into getting some kind of hosting service in like Austin or something since its a big tech hub in the south.


Comment below rating threshold, click here to show it.

ObsidianStorm

This user has referred a friend to League of Legends, click for more information

Senior Member

03-01-2014

Quote:
Originally Posted by sonicdeathmonk View Post
Moving servers more central for all NA players is a priority. As I mentioned earlier, truck loads of servers are purchased, we just need to lock down the data center contract and get the Internet circuits installed. In the meantime the servers near Portland, OR will see connection improvement for many players as the direct connections to the major ISPs and telcos get installed (more are getting hooked up every week). We were maxed out of quality space, Internet bandwidth, and power in the Los Angeles area so Oregon was the best choice at the time to get large amounts of servers up and running (Facebook, Apple and a few other major players are an easy drive away), it is technically only 7-9 msec further away from the LA area (if LA was the better place, it is not for all players) but we need to optimize the routing and circuit mix to really achieve the best possible connection for all players to the Oregon located servers. The team is working long hours to get this done.

In the last few months many players were likely getting first routed to LA then to Portland through San Jose or sometime weirdly through Seattle. Now lots of players are taking a much more direct route to just San Jose and then Portland if you are not in the Southwest USA or you will likely pass through Seattle then to Portland if you are in the Northwest USA or coming out of most of Canada. So ping time for many players should have gone down in the last month (but not for all). This will continue to optimize for more players as we get more telcos and ISPs directly connected.

Another team is focused on getting game servers spun up more centrally which will then allow Oregon to become the fall back site if things go into the ditch in a bad way (something we have not had as option in the past). So net, Oregon is a temporary hop which has allowed us to significantly expand the number of game servers and other resources we have needed to help stabilize NA. Next step is to optimize ping time for as many players as we can in Oregon and then a more central to NA location which we will tell you about as soon as we lock it down.

-sonicdeathmonk
From the sound of things, we shouldn't hold our breath for those central servers. Especially if you're mentioning fixes for "the meantime."


Comment below rating threshold, click here to show it.

Thug

Member

03-01-2014

Quote:
Originally Posted by sonicdeathmonk View Post
Moving servers more central for all NA players is a priority. As I mentioned earlier, truck loads of servers are purchased, we just need to lock down the data center contract and get the Internet circuits installed.
sonic can you comment with an ETA on the central servers and do you have any locations in mind yet? 1 month? 3 months? 6 months? it doesn't seem like the data center contract, or the installing of the internet circuits could take more than a month. Thanks for the updates.


Comment below rating threshold, click here to show it.

Thug

Member

03-01-2014

Quote:
Originally Posted by Shentok View Post
My problem is that in 99% of the games I now get connected to the Oregon servers instead of the LA servers. It sucks. I hate having 60~70 ping when I used to have 25~30 ping every match. I know it's great having 60 ping comparatively, but I don't even know how people play on 60 ping. Everything feels delayed and it feels like my reaction times suck because of it.
Lmao.. this is what the east coasters laugh at dude.. you are complaining about 60 ping.
You live 1 state away from the server. We live 10+. We have never played on 20-30, or 60 ping. most of us play on DOUBLE of what you say "feels delayed".


Comment below rating threshold, click here to show it.

daft inquisitor

Senior Member

03-01-2014

SDM, you're my hero!