The battle for bandwidth.
So I saw a post over at Ioshints about bandwidth, how we, as consumers come to expect a certain bandwidth, and how ISP’s are dealing with these issues.
It is a fact that ISP’s greatly oversubscribe their networks, and keep selling more access-lines with an even greater bandwidth cap than ever before.
So lets explore some of the factors in this bandwidth revolution!
The (very) near future:
New types of network devices and services is one of the reasons why people need more bandwidth. Also, traditional types of devices that was not internet enabled are now becomming just that. Think IP-phone, security systems, DVR’s and so on. Some of these are not bandwidth hogs, but they are very intolerant toward jitter and loss.
More and more content requires an even larger pipe. An example of this is the new Xbox360 offering of providing HD movies to the home. A requirement for this is at least 8Mbps connectivity. Thats 8Mbps effective bandwidth. Not what you pay for each month, but what you are actually getting. A real-life scenario of a typical family utilizing this offering, is 8Mbps purely for the movie for you and your spouse, maybe 1-2Mbps for one of your kids playing online games and maybe up to 1Mbps for the other kid to play around with photoalbums on facebook. Thats 11Mbps you need to avoid any bad user experience (choppy video, lost connections to game servers, and long waiting times on facebook).
From this perspective, the day where we absolutely must have 20Mbps to enjoy the offerings of a digital lifestyle is approaching rapidly. So where are we at now?
Current situation:
At home, I have an 8/2Mbps ADSL line. 8 Down, 2 Up. But even under the best of circumstances I never get more than 6Mbps. This is also a line from a very respectable service provider, which is rock solid, great support, and generally a good all-round ISP. Now in my case the difference can be due to simple distance, or bad wire drops somewhere (we all know ADSL’s inherent distance limitation). According to my ISP however, the impedance on the line should provide me with the full 8Mbps. Alas, my line could be experiencing oversubscription on something else than the access-link to the ISP (PPPoA).
Another example of how this scenario plays itself out. My dad’s house is in a fairly large city, very close to the POP of the ISP, and they firmly states they will provide him with 20Mbps down, but no matter what has been tried, nothing above 8Mbps is possible. The tech-support again, says it should be possible to at least double that stating impedance measurements again.
I could name several more cases like this where what you pay for is a factor of 2 more than what you get.
Thats the technical current situation, what about the non-technical side?
Well, this is where it gets pretty bad indeed. Almost all ISP’s im aware of has a large marketing department that is busy getting new customers as well as hanging on to the ones they have. The first part of this is the new users.
New ISP users:
How many of you have missed any commercials stating something along the lines of: “Now with 20Mbps connection, for the low price of X$/month”? Almost all of us have thought that this was a better deal than the one we have now. But as I outlined in the current situation, this is pretty much hot hair by the marketing department. There are so many cases where the small blueprint will tell you that this connection will only be 20Mbps if the copper (a bit more on the media type later on as i have some different experience here) is the best of the best, there are no cable hinderance in any way, the planet is aligned with the right stars on the right time of day and on and on. In reality you are not getting what you are paying for. Thats the hard fact in most cases.
Imagine going to a car dealership and being told that this certain model of a car will go 20Km/Litre, and later finding out it only does 10. How will you feel about this? will you be up in arms complaining? I certainly would. So why is this not happening in the ISP consumer market?
Well, for one thing, we are just now starting to see applications that fully utilize these bandwidth limitations. HD video, photographs all over the place, Web 2.0, streaming back and forth, file access from everywhere, data at your fingertips. Lack of knowledge by the consumers also does not lead to complaining at the ISP’s. If you tell your non-geeky friends to meassure their internet connectivity, do you think they could? I doubt it… Because of the successful marketing campaigns, all the knowledge they have of broadband is that bigger is better. In 7 out of 10 cases users wont attribute a bad internet experience as being the fault of the internet provider itself, but rather as because of the service they are using (facebook, online gaming and so on), or even if they do realize “this internet is slow”, they might think: “well, 20Mbps is not enough, I MUST get 30Mbps”, feeding into the ISP marketing campaigns again.
Current ISP users:
The second part of the marketing raving, is about keeping existing customers. In almost all markets, where competition is fierce you will see that the price-point is where competition really flares up. In the ISP market however, they (as in ISP’s) have managed to make it about more bandwidth for the same price! Again, I have several first hand examples of existing customers receiving a letter stating: “You will now be upgraded to a 2x
These two types of behavior really p*sses me off alot. But its part of the ISP’s toolbox to stay in control.
So technically, whats really going on here?
Well, there’s quite a few things, many concerning bandwidth management and profiling users.
Profiling:
What follows is a guess on what ISP’s do to customers, since I dont have any direct relationships to any ISP’s, nor any experience in deploying ISP networks. I do however read alot about these technologies and about how users are experiencing their service, and from this i can make an educated guess on the technical decisions going on inside an ISP.
First of all, lets talk about idle times and unused bandwidth. Above I wrote alot about the near future, but at the present alot of sold bandwidth is unused. As im writing this, im sitting in a hotel room, using their wireless service. I have been using that for most of the day. I have some network applications open, including a webbrowser (Firefox) connected to about 15 sites, most of them just normal passive webpages, no constant reloading or anything but a few being the famous Web 2.0 are interactive and watching my every move and reloading content and so forth, I also have Microsoft Messenger running, Tweetdeck updating at regular intervals, my mail client (Thunderbird) also retrieving mails every now and again, on top of that Skype is also doing its thing. The only real constant bandwidth usage is from streaming internet radio and my pinging a few internet hosts. This is typical workstation usage for most people most of the time. I am using around 130Kbps a second, which seems to be right since my radiostream is a 96Kbps stream + the other stuff going on.
This “profile” also matches what im doing when im at home on my 8Mbps. My girlfriend is a heavy user of Facebook, always looking at photoalbums from friends and family. Lets say that we as a couple use 1Mbps on average during all daily hours. Lets also assume that my ISP was actually capable of providing me with the full 8Mbps downrange capacity. That still leaves 7Mbps on average to be consumed.
As you can see from this, this is a great opportunity for ISP’s to oversubscribe their networks. Even from the home to the DSLAM you are certainly able to do a 1:4 over subscription just to be on the safe side. This is even before data has ever reached the core of their network. With this sort of profile, ISP’s dont need to have a full 1:1 capacity from the DSLAM’s to their backbone, again giving them some leverage to avoid having to upgrade the line capacity between the DSLAM to the backbone. In alot of ISP scenarios, utilizing existing network infrastructure (translate: copper), is a huge saving compared to biting the bullet and digging down fiber cables.
Here comes the catch. How do you handle “out-of-profile” users? For example a family with a young teenager using P2P sharing programs? This one kid can bring down the “saved up” bandwidth, because he/she will be using the full bandwidth the line can provide. So again, if we use 8Mbps as an example, instead of having a profile stating 1:8, it will now be 1:1. If this happens all over the place, oversubscription is not possible because the ISP’s must try to deliver the CIR (Commited Information Rate), rather than the PIR (Peak Information Rate). In the end, if enough of these out-of-profile users are on the same segment, an investment in infrastructure is unavoidable… or is it?
As it turns out, it is not unavoidable. Technology makes profiling based on your network usage possible, and hence makes it possible to take action against those who you consider out-of-profile, even though they have done nothing wrong in regard to bandwidth utilization. We have seen this on the online newsoutlets of late. ISP’s throttling bandwidth if users are using P2P programs by doing TCP resets, bringing down the used bandwidth to the level they (ISP’s) want, instead of the CIR they sold to the customer.
Lets switch gears, and provide some good news. Fiber to the home. This is becomming more and more popular, especially in Scandinavia. This basically provides unlimited bandwidth from the home to the backbone. This is great right? Well, sort of. A problem arises here as well. ISP’s have put alot of money into fiber backbones, being able to handle alot of user load, but they still must go through other Tier ISP’s, and they must still pay for it. So even though a user can get 1 Gbps of connectivity to the ISP, that ISP cannot provide 1 Gbps to the rest of the internet, so you are scaled down there as well. Good news: You have unlimited bandwidth to the backbone.
To boil it all down, the ISP’s are oversubscribing lines. The customers think they are getting what they are paying for. The marketing departments are making promises they cant keep. The customers keep throwing money at stuff they are not getting.
What im calling for is transparency. Dont screw your customers by selling them snake-oil. Let them know what they are getting, and tell them you are profiling. Dont keep customers based on false promises.
At some point, if this all continues, you will have a bubble that will burst. Customers wants new media, in more ways than ever before. They will be very unsatisfied realizing they cannot get that media, because the bandwidth they thought they had, and have been paying for for years, simply doesnt exist in reality.
Its an undeniable fact that investments in more and more bandwidth is nessecary. It is also undeniable that the cost of these investments are too much to bear for many ISP’s without increasing cost of access. Until then, the ISP’s are profiling. So do the right thing, and let your customers know about it!
These things need to reach some sort of equilibrium to make ends meet. Otherwise we are all doomed!! 🙂
Had to get that off my chest. Thanks!