Netflix and Google and Amazon and Apple should join forces to let Comcast customers know when they're being throttled.
"Your video is buffering because Comcast is slowing your connection down. Click here to contact Comcast customer support. Click here to find Netflix Certified providers in your area."
Netflix and Google and Amazon and Apple all have great brands with high customer satisfaction. They should utilize all those eyeballs to funnel even more hate toward Comcast.
> "Click here to find Netflix Certified providers in your area."
Followed by:
"I'm sorry, there are no Netflix Certified providers in your area."
When Comcast is rated as one of the worst companies for customer satisfaction, customers aren't staying with them because they are ignorant when it comes to the alternatives. They are staying with them because there are no alternatives.
"I'm sorry, there are no Netflix Certified providers in your area."
Seems like a natural place to put "click here to contact your state/federal representative"
While I grant that cynisim is definitely warranted, at the end of the day, these companies have monopoly jurisdiction because we the people allow them to.
Can anyone tell me what would be the problem with just having a law "no law is allowed to be passed preventing any individual or entity from providing broadband service to any other individual or entity"? And how said problem would be worse than the current dismal arrangements?
"Look I'm sorry mate but I'm just running this wire through your house to give your neighbor internet, if you don't let me you're in the wrong"
"Sorry but we need to tear up this highway and disrupt traffic for the next couple of weeks to lay some fibre. I know someone else did the same thing last week but they were another company and it's illegal for the local government body to stop any of us laying stuff on public land when we want"
Obviously that extreme is untenable. You may be after something a bit more useful like unbundling, but in the US you have to explain how forcing a private company to let competitors use it's cable is not outright stalinistic communism.
There's a difference between not allowing ISPs access to public lands and outright prohibiting them from offering their services.
Obviously if they wanted to run wire through a house they'd have to get the owner's permission, or the local government's permission to dig up a road. The thing is, as the law currently stands they're forbidden from doing so; even if every household in they city signed a contract saying "sure, let company X run cable through our yard", company X still wouldn't be allowed to sell them internet over that cable.
Maybe a company could innovate and develop some form of fast wireless mesh network. But even if they had the technology and finances to do so, local laws stating "the only company allowed to sell fast internet here is Comcast" or equivalent prevent them from doing so.
There are two issues with this. There are also MASSIVE incentive issues with this (lobbying, etc) but in this comment I'll only discuss legal issues:
The first is that a legislature can't easily bind its future self - laws passed by a majority can be repealed by a majority or circumvented by a majority in the future. So even if we pass Law 2014-003 saying what you said, there's nothing stopping a 2015 (or 2020 or whenever) law from saying "Law 2014-003 is hereby repealed" or more narrowly "Law 2014-003 doesn't apply to this law". There are some edge cases where you can pull it off but by and large it'd take a state/federal constitutional amendment.
The second issue is that the federal government can only regulate "interstate commerce". While you'd think that anything involving the Internet would automatically apply there, you couldn't stop a state law that covered only a local business dealing with in-state customers. So such a law could be used as a shield for Verizon/Comcast/etc while state laws restricting ISPs would only apply to smaller in-state competitors. My parents and a number of my friends are served by a local Wireless ISP because that's the only alternative in their neck of the (literal) woods. Under your system, their ISP could be subject to innumerable regulations that Verizon/Comcast would be immune to.
We do ... for landline telephone service. The telecomm companies had to put up with that for so long that when other communications services started seeing demand, they did what they could to lobby away similar regulation for internet access.
What it boils down to in most places is that internet access is not (legally) a public utility and therefore skirts public utility regulation.
> Seems like a natural place to put "click here to contact your state/federal representative"
How does that solve anything, though?
Judging by the lack of competition, it seems it's not profitable to compete.
Now, you could force Comcast to lease out its network capacity to competing companies, but this is really a shortsighted solution: the consequence is that potential competitors, who are contemplating establishing competing last-mile connectivity, cannot make a solid calculation to see if it would be profitable, because it might risk being forced to lease out the network capacity it establishes, thus it avoids doing so.
The problem here is simple: digging down cables is expensive. No amount of legislation is going to change that. The real solution is reducing the cost of the last mile connection. This might be possible through opening up legislation (removing barriers to entry), though. Like opening up more wireless frequencies, to carrier last mile traffic to consumers.
>Judging by the lack of competition, it seems it's not profitable to compete.
Nobody is allowed to compete. No company will be afforded the same benefits given to the first cable company that actually layed the physical infrastructure. Local governments give all kinds of benefits to the first company to bring in cable. They won't for the second. They will see it as wasteful duplication of infrastructure.
Companies should not be granted monopolies on the last mile lines that they were granted all kinds of exceptions for. Telephone companies seem to get by without it, why are we tolerating it for cable companies, especially when they are competing with services offered online?
A majority of the people who can pick ISPs (37% of the country) have only two choices. I'd bet in most cases those two choices are Comcast and Verizon. The odds you'll have a neutral ISP as an option are pretty tiny.
It is absolutely true. Even in the middle of silicon valley for many houses comcast is the only viable option. In that regard, internet service is very similar to public utility like electricity, water, sewage and phone service.
Do you actually know what kind of bad shape our electric, water, and sewage systems are in? In many cases, they consist of century old infrastructure with pipes leaching lead into water, sewage systems overflowing raw sewage into rivers, etc. Even with these companies being defacto monopolies in many places, they keep the infrastructure in far better condition and invest much more on upgrades than public utilities do. And I doubt you'd like the result if telecom infrastructure was upgraded on the timelines utilities operate under.
This amusing anecdote belies a key fact: Comcast and other Cable Industry Groups proposed and lobbied for laws making municipal internet services <illegal>.
How is that relevant to my point? If anything, it supports my point. Municipal utilities tend towards monopoly, because private entities can't compete with a universal government funded service. But what makes you think that these municipal broadband networks would be better maintained than our aging and deteriorating power, sewer, and water infrastructure?
Humor me for a second and get past the fact that such a law would benefit cable companies. Is it a bad idea on its own merits? I don't think so. Municipal broadband would crowd out private investment in broadband. Given the state of other municipal infrastructure, there's a high probability it would be underfunded and low quality. Would that be a better state of affairs? Being stuck with an overloaded municipal system that never received enough funding because voters don't really care enough allocate a lot of money maintaining and upgrading it, that at the same time makes it impractical for private systems to operate?
I don't think the solution is to allow municipalities to do more things that undermine competition. The solution is to deregulate at the state level. Force municipalities to make the kind of "hands off" deal Google demands in return for rolling out Google Fiber to any company willing to make the investment.
(Q1) Regulators should encourage competition? True/False
(Q2) All government services are monopolies? True/False.
(Q3) All municipal services are either (utilities, monopolies)? True/False.
These are all absurd formulations.
Public Services are a thing. Some examples:
Ex 1> My public library is an ISP.
Ex 2> My public library is Tax Filing Center.
Ex 3> My public library is a copyright depository.
Ex 4> My public libary is a copy/fax location.
Competitors:
Ex1'> Comcast
Ex2'> TurboTax
Ex3'> Netflix
Ex4'> Staples
I don't think the solution is to allow municipalities to do more things that undermine competition.
Again, this is absurd. Public services do not undermine competition by implication. This formulation is akin to saying: "a good competitor undermines competition by being good." Or, "a new competitor undermines the competition, by entering the competition."... Um, no.
This is not to say single payer healthcare would not change the business/policy landscape in the USA.
But to even suggest that any municipal provisioning of communications services (public radio, public TV, public internet access) would be aking to "single payer healthcare" in scale/scope/completeness etc is simpley a rhetorical exaggeration.
I think its interesting to debate the pitfalls here, butnot with such extreme poles as a starting point.
Limited scope incremental services are fine, IMHO. Feasible, cost-effective, and market & competition extending.
And the existence of national parks does not make the government a "land monopolist" or what not. =D
Government competition is not fair, because governments don't play by the same rules as businesses. They can make money by fiat through taxes. They can and usually do lose money on services they provide. Businesses simply can't compete with the power of taxation. It should come as no surprise that there are no book rental companies when there are libraries.
I'm a libertarian but i believe in public rights of way. I'm finw with a single provider of roads, provided they are not toll roads but rather "dumb" roads that let me go when and where I please. This is in contrast to government-sanctioned but opportunistically managed toll-roads with dynamic pricing that rips me off every time I have an important meeting to get to by turning into a "protection racket" for my important business.
The rest of the stuff about "fair" x or y is not really relevant. Markets are supposed to be "free" and not "rigged", but neither of these two concepts imply "fair" in the sense you are using it. Every company on the NYSE hasi its own cost of capital--is that 'fair'? no. its not fair to the owners. But that does not imply that the market is not free for capiral or that the market for capital is 'rigged'.
There are plenty of ways that government services can be provided to extend markets geographic reach, variety, and product depth without getting into some nasty discussion of socialism/totalitarianism etc. There are some services that are more-efficiently organized through the government. In that respect, a government entity is just like a non-profit that competes with a for-profit or private foundation entity (eg universities). And more ...
Of course government should provide roads. And water, and sewage, and a police force, and fire protection, and libraries, etc. What made you think I didn't agree with that? You are fighting a straw man.
And "fair" absolutely is relevant, from a market-wide POV. Of COURSE some companies have natural advantages over others. That's great. They should win. When the better competitors DON'T win, you have a market fairness problem. If government is providing a service at higher cost to the taxpayer than if a private company was providing it, then there is a problem. And this happens all the time, because the sticker price of government services is always low, or zero, even though the real cost can be very high.
I don't think it's controversial to say that public infrastructure crowds out private investment. It's very hard to "compete" with a service that everyone is obligated to pay for anyway through their tax dollars. A good example is schools. In the U.S., private schools are allowed, but 90% (and increasing) of kids go to public schools. Is that because the public schools are better? No. It's because taxpayers are obligated to pay for public schools anyway.
Furthermore, I don't think it's controversial to say that municipal services tend to be least common denominator affairs. Again, look at schools. I might personally be willing to pay twice as much property taxes to get well-funded schools that hire well-paid teachers, but good luck getting 51% of voters to agree to such tax increases.
Look also at power, water, and sewer utilities. Why are they so badly underfunded, and in a continual state of life support? As a yuppie, I'd be happy if my sewer bill went up by 500% (from $10/month to $50/month) if that meant a high-quality sewer system that didn't dump raw sewage into the river when it rained. I'd be happy to pay a lot more for water that wasn't flowing through old leaded pipes. But good luck getting any of that through a rate setting board!
Public utilities are under political control. Depending on the industry, this can be a good thing or a bad thing. I think for telecom, it's not a good thing, in that it won't lead to an equilibrium that I think techies would be happy with. Say you wanted to build municipal fiber in San Francisco. What's the body politic that will control that system? Do you think it'll be techies? Of course not. It'll be the folks that are out there picketing Google. The discussion will shift from provisioning high speed service to figuring out how to subsidize low-income and high-cost people. Or, take Seattle, which is not very dense and is quite sprawling. Do you think the political consensus will lead to installing fiber in the high-density, high-income, low-cost areas first?
It's also interesting you use public libraries as an example. Municipalities all over the country are cutting library budgets to the bone. Why? Because people don't care about them. I think what you'll find is that municipal broadband will be in the same bucket. People won't care enough about it to appropriate the substantial funds that will be necessary to maintain and continually upgrade the network.
Force municipalities to make the kind of "hands off" deal Google demands in return for rolling out Google Fiber to any company willing to make the investment.
This is really foolish. A company that wants to be a monopoly will provide fantastic service, but a company that has succeeded in establishing a monopoly no longer has any motivation to treat you as anything other than property. Why would any rational citizen let their municipality make such a deal?
But I bet you want to keep your unlimited usage, right? Water, phone service and electricity are metered; sewage isn't because it's just too hard to measure.
If you want to treat broadband as a utility, you have to pay for it as such. Which means metered billing, which I don't think anyone wants.
If metered billing wasn't simply an excuse to gouge me for money then I might be interested. As it is now, we're seeing metered usage tacked on to plans that were sold under the title of "unlimited". In that situation, it feels like the ISP is trying to work around increasing capacity by discouraging usage.
According to Netflix in February, no ISP has ever throttled the service. My understanding is that nobody's intentionally slowed down anyone else, they just failed to allocate additional capital to improve congested parts of their network. Comcast should also still be bound by the "net neutrality" principles they agreed to as a condition of their NBC Universal merger. I'd think if they were singling out any service for throttling, at least one of their 136K employees would blow the whistle to the FCC/DOJ.
This would make sense but for one point. The graph of service quality spiked as soon as Netflix began paying Comcast. If they're weren't throttling then how could such a spike be possible? You can't see those kind of performance gains by throwing a ton of hardware at it. You couldn't even provision the amount of hardware needed to see those gains in that short of a time frame (less than one day by the chart).
It's not a question of hardware but routing. The deal cut out the middle-man between Netflix's and Comcast's networks, Cogent. Because Comcast and Cogent were in a constant feud over their peering agreements (Comcast wanting Cogent to pay more to transit all that Netflix data), service through that link was pretty much at capacity and slowly degrading the Netflix experience for Comcast customers.
The two networks are directly connected now, which eliminates that bottleneck. That's why performance spiked. There wasn't a capacity issue on either side of that Cogent link; Netflix is willing to pay for its bits, and Comcast's network has adequate capacity to deliver them. This routing change actually occurred two days before the deal was announced; you could see through a traceroute already that the route between Netflix and Comcast had shortened and no longer went through Cogent.
> Comcast wanting Cogent to pay more to transit all that Netflix data
That doesn't make sense to me. Comcast's customer isn't Cogent; Comcast's customer is the end user who sends them a check every month for access to the internet, regardless of where it comes from. If Cogent is being paid by Netflix on the other side, then Cogent and Comcast are exactly aligned (each wanting to deliver the Netflix packets because their respective customers are paying them to) and should peer without any money changing hands. If Comcast wants Cogent to pay it so that Comcast's own subscribers can get access to Cogent's customers' content, then they're being shady.
> If Cogent is being paid by Netflix on the other side, then Cogent and Comcast are exactly aligned (each wanting to deliver the Netflix packets because their respective customers are paying them to) and should peer without any money changing hands.
Should Comcast cache my personal website for free so that it's super fast half way across the country? Or am I responsible for making my website faster (by using CDNs and such)? If it's the latter, why should Netflix not have the same experience I have?
Your question confuses me. If your website is cached on a CDN, Comcast users still have to connect to the network that hosts that CDN, and Comcast accomplishes that by using a peering or transit relationship. The CDN won't magically speed up Comcast's network, which is still responsible for getting the content to people's homes. Netflix is already making its website faster by paying Cogent a lot of money to deliver its data to the ISPs and by (clears throat) using and even building its own [1] CDN. Where Comcast comes in is delivering that content to its customers, who, again, pay it to do exactly that.
Yes, and others in this thread believe Netflix shouldn't have to pay to peer with Comcast. What I'm saying is, I too want to peer with Comcast for free. Do you support this as well, or just Netflix?
If you have traffic for Comcast's customers and you bring it to Comcast's facilities, what service do you imagine Comcast is providing that you should have to compensate them for?
It's not transit. They're not delivering your traffic to some other ISP. They're delivering it to their own customers, which is what those customers have already paid them to do.
They're delivering it to some of their customers. Those who are also Netflix customers.
Netflix wants to distribute it's costs back onto everyone else (everyone but themselves). Even though this new arrangement reduces their costs vs. using Cogent.
I thought this discussion was whether it's reasonable to expect Comcast to peer for free, which it is. (AnthonyMouse's answer is exactly right.) But now somehow this discussion is about the distribution of costs across Comcast's customers? Er, OK: Comcast does have a problem there in that it charges its customers flat rates for access instead of charging them per byte. That creates all sorts of weird incentives both for those customers and for Comcast. At some level, that's the root cause of a lot of this; as an ISP its incentive should be to route as much traffic as it can as efficiently as it can, and weird stuff like this comes up because it isn't.
That's messy and Comcast should fix how it distributes the cost of its customers using the internet, but what does it have to do with Netflix paying for anything? Similarly, you don't pay Comcast to route the packets from your website to your visitors, even though only a subset of them use your site, thus asking everyone else to take on those costs. How is this different?
You think metered internet access is a better solution? Seems to align the costs and who pays for them a bit better. So if I'm sitting here watching Netflix 24/7 365 my bill is understandably higher. It'd prolly make me how much I used Netflix but that's understandable.
Any plan where I don't directly pay per byte I download seems destined to run into problems like these - Comcast doesn't want to deal with the increased traffic Netflix is pulling into their network, but really what are there customers paying for but that traffic?
> You think metered internet access is a better solution?
A better solution to what? There isn't a problem to be solved here. There is no danger of Comcast becoming insolvent as a result of Netflix traffic going to Comcast customers. The threat to Comcast of people using too much Netflix has nothing to do with Netflix using too much bandwidth and everything to do with Netflix taking too many customers away from Cable TV. Comcast doesn't seem to have any trouble with bandwidth consumption from people watching their on-demand video services.
The idea that there exists some large collection of people who would prefer metered billing is nothing but a talking point in favor of a policy that damages the ISPs' video service competitors. AT&T did studies in the 1970s that showed that people prefer unmetered plans over usage-based billing even when usage-based billing would save them money.[1] Because it's not worth $10/month to have to incur the mental cost of having to constantly measure your usage.
This is a straw man: Netflix doesn't want to peer with Comcast at all. Netflix would like Comcast to provide their (Comcast's) customers with the bandwidth those customers are already paying (Comcast) for.
But this still doesn't explain why Netflix is paying Comcast, not the other way around, or, even better, with noone paying anyone. This clearly shows that Comcast had an upper hand in the negotiations, probably because of its monopolistic position.
All internet services pay for their bandwidth, they don't get paid to be online. Usually that means paying a hosting company, which pays a data center, which pays some transit provider like Cogent or Level 3 to get onto the internet. Netflix was paying Cogent to link it to the internet. Now Netflix's servers are directly linked to Comcast, so Comcast is connecting it to the internet [ed: to Comcast's customers], and that's who they're paying instead.
Why would they want to pay Comcast instead of Cogent? Because Comcast's fight with Cogent was resulting in poor performance for Netflix customers. They had the option of waiting for it to get bad enough that one side or the other gave in on the negotiations and increased capacity, meanwhile losing customers frustrated with buffering at peak hours, or choosing an alternate route to get their bits to those customers.
It also nicely sets up barriers for entry to new video streaming services, which is something maturing companies tend to do. Netflix has grown big enough to buy transit directly from ISPs instead of through middle-men like Cogent, while startups don't have the capital for that.
Netflix was paying Cogent to link it to the internet. Now Netflix's servers are directly linked to Comcast, so Comcast is connecting it to the internet, and that's who they're paying instead.
No, Netflix is paying Comcast to connect them to only Comcast's customers, not the Internet. Suggesting that they're paying Comcast for transit is disingenuous.
All internet services pay transit providers but they don't typically pay last mile networks for access to their subscribers.
Comcast is able to fight with Cogent while their own customer's connectivity suffers precisely because the customers often don't have an alternative ISP. Netflix argues that if the TWC merger proceeds the situation will get even worse.
No, what is shows is that it was going to be more expensive for Netflix to buy market-rate transit through another provider than to peer directly with Comcast. (Or Netflix is playing the long game here and taking an unnecessary financial hit by peering with Comcast to put them in a favorable PR position, but I think that's less likely.)
Cogent's thing is selling massively cut-rate transit. They've been involved in constant peering disputes over their method of doing so since at least 2002. This isn't new for them - Netflix is just big enough that people really took notice this time.
But if it is a zero-sum game, 'crime by omission' may be as bad as 'commission' of throttling. So the question is: is it a zero-sum game? If Paying Company A pays for more bandwidth, are they doing so at the cost of revenues for competing nonpaying company, say Nonpaying Company B? In a sense, yes, because then, users are likely switch from the Nonpaying Company B to Paying Company A (whose product/service is superior by being on the 'paid fast lane'). So though it is not directly a zero-sum game, in terms of customers and revenue, it has the potential to be an indirect zero-sum game.
I think the real question is... why haven't they already done this? Are they worried about pushback at the political level? Certainly their users would agree.
In a lot of cases it will do nothing but frustrate users. Why? Because where I live, the only really decent internet is Comcast. If I wanted to change providers, I would go from 16 MB/s internet down to approximately 1 MB/s (advertised speed...so like 1/10th of that in reality).
If users get frustrated enough, the public policy on net equality might change. ISPs get away with lobbying, the revolving door, and exclusivity deals with city governments because only techies care.
Yeah but nobody will be willing to be the ones who suffer horrible or no service in the short term, certainly not long enough to enact the required changes to bring better competition to the areas that Comcast monopolizes.
That's assuming that "the people" have a measurable influence on politics. Plenty of data show that no such influence exists in certain aspects of policy.
They have horrible customer service and service for one reason only: because they get away with it. In many markets you have no real choice, and even if you do is the Horrible Comcast or just a as horrible Verizon, AT&T and so on.
Comcast can do what it does because Democrats portray the company as a problem with capitalism, while Republicans portray it as a problem with regulation.
In truth, it's both. It's a problem where a company has co-opted well-meaning regulation to snuff out competition, which is a thing capitalist companies do. Meanwhile, citizens point the finger at each other instead of removing Comcast's ability to monopolize.
Either the regulations that make competing as an ISP difficult need to be stripped out (my personal favorite), or the last mile needs to be public property. Either way, Comcast getting to be the only one sitting on that last-mile connection is what has given rise to its stranglehold on the internet.
We need to support fixing this on a local/state level. Support smaller ISPs in their attempts to disrupt the incumbents, and support legislation that actually encourages innovation in the marketplace (even if that means striking policies from the law books).
This is the brilliance of what Google is doing. They are making cities compete for Google Fiber. You compete by making it easier, faster, and cheaper to lay fiber.
It seems to be working. Seattle's new mayor recently talked about how the city needs to make it easier to build out internet infrastructure.
> Another possible solution includes granting internet companies access to utility poles at little or no charge, so that building more infrastructure is not cost prohibitive.
> We are considering [changing policies] which make it nearly impossible for internet providers to expand existing services without an unusually high super majority of support from neighbors.
GFiber is the antidote to these parasitic bastards. They have the cash to push last-mile FTTH to every metro area in the country. I think they own the dark fiber backbone to do it too.
Here's hoping that G thoroughly topples every last one of them.
Five years ago I might have agreed with you. There's no reason to trust that google (another giant private company that disabled my ability to comment on youtube or rate android apps because I dared not to want to join g+) won't pull some crap once they're in a dominant position in the ISP market. Like with many other things google saw the writing on the wall regarding how they at the behest of the ISP and looked for ways to be at least minimally prepared for the worst.
At the risk of a trite response, it's the evil you know versus the evil you don't, and frankly getting rid of the evil I know (pretty consistent 5-10% packet loss across the board, spiking during bad weather, repeatedly tracked to terrible work in the outdoor cable boxes, but that I get charged 40$ a pop to temporarily have them fix) is worth a gamble in my books.
It's not the Fiber that's the fee earner. By having more people connected with faster, more stable connections they can continue pushing the envelope with things like higher definition on YouTube and suchlike but this is the razzle-dazzle that people will see without noting the true purpose; you're accessing the net through their service so they can now pin all of your surfing habits to a physical person driving more benefit to their primary Ad business.
Just imagine what end-users can do with gigabit upload speeds. If you think streaming is big now, just wait until Google Fiber kicks the gigabit market into high gear. This is one of the best things Google is doing right now, and I can't wait to see how it changes the Internet.
Is it possible that's more to do with people testing over WiFi than with the speed of Google Fiber itself? It's hard to tell a lot about the data, since they only provide averages.
I've heard that Google Fiber has inspired other ISPs in its service areas to initiate high-speed service. I wanted to see this contention though so I clicked around in speedtest.net a bit. Can you point out some contention, because I didn't actually find it [http://www.speedtest.net/local/kansas-city-mo].
This is the best thing about what Google is doing. 90% of the blame for this situation belongs on municipalities. They're the ones that created these franchises in the first place. They're the ones that imposed huild-out requirements and regulated rates. They're the ones that have made it impossible to build new infrastructure even after Congress made exclusive franchises illegal in 1992.
As someone who's trying to bootstrap a small, local, last-mile ISP right now, my only wish is that there were many more people that held your opinion on this matter, and were willing to back that opinion with their dollars.
I can't offer you cash (I'm in Chicago), but if you ever want free help, even with trenching, pulling fiber, long-haul wireless throws, system/network admin, email me (email in profile).
- Figure out how to get access to right of ways [Completed]
- Figure out how to get internet access to sell [Completed]
- Figure out how much it will cost to start a CO [Mostly Completed]
- Acquire a place to put a CO [Completed]
- Acquire high quality, redundant backhaul [Under Construction Now]
- Figure out who's willing to pay to switch away from Comcast [I have some names, but this is definitely not completed]
- Beg them for money [Some have subscribed, but definitely not enough yet]
There's a couple of big things that work against me:
City + State taxes for right of ways are pricy. We're talking multiple thousands of dollars per mile per year, on every single mile, just in taxes. (And I'm lucky, my city still owns a lot of their own right of ways. A lot of cities just wholesale gave them away to AT&T/Comcast).
Backhaul is expensive. Most ISP's are monopolies, so they can safely buy in scale. I have zero scale, so I pay full retail for everything. This is obviously very pricy. (I'm guessing I pay at least 3 times more, per Mbps, than Charter pays for backhaul, and probably 4 - 6+ times more than Comcast pays)
Fiber lines are dirt cheap. Install is expensive. Easily 95% of the cost of the network is that initial installation, and it can run into past the million dollar mark even in the smallest of neighborhoods.
If you want more info, let me know. I'd happily talk more about this, and provide solid numbers.
This sounds fascinating, I'd love to hear more. Incidentally, that makes me wonder if the backhaul cost problem could be something Google could see their way to sell to local ISPs at closer to bulk rates. It aligns with the general goals of Google Fiber, while allowing Google to pressure big telecom companies with less capital outlay.
Is backhaul more expensive than getting a big Cogent connection for like $1/Mbps? Or do places like Charter/etc pay even less than that for their paid peering connections?
Yes. I've never seen a quote at $1/Mbps regardless of scale.
It wouldn't surprise me if speed/pricing like that existed around major network hubs (like Chicago).
I'm paying $8/Mbps. If I were to afford five times more backhaul than I have now, that price eventually drops to about $3.8/Mbps.
Places like Charter and Comcast pay less for their backhaul, in part because they are their backhaul, in part because they are large enough to negotiate their own peering agreements.
Recently, that's swung even more in their favour. Netflix, for instance, pays Comcast money for the "privilege" of paying the costs for all of the backhaul Netflix uses over Comcasts lines.
People sometimes call this "double dipping". But it's actually "triple dipping". Comcast charges subscribers money, Comcast makes Netflix buy the backhaul to Comcast, and Comcast charges Netflix again, to allow them to connect the backhaul they already bought, to Comcast's network.
Yeah I'm not sure the cost for the infrastructure but I've seen quotes of $1-$2/Mbps for Cogent in various datacenters (assuming a 1-2Gbps commit I think).
Makes sense though, I'm sure if you had to get the fiber to your location that would easily push the price up a ton.
This makes me wonder how hard it would be to create a cooperative ISP, where the users own the network. I wonder if you could crowdfund something like that easily in, say, San Francisco.
Are you targeting businesses first? I know that some local ISPs have been successful by first poaching the fraternities/sororities around a school, then expanding from there.
I'm game - I worked for a major national DSL provider doing field work, a major ISP doing support, and a major telecom carrier now doing field support.
There were fights at the municipal level too, but by that point, only a few medium-sized places like Hull in the UK, and Rochester, New York had phone companies that weren't owned by NTC or AT&T.
All of this seems strikingly familiar today, except that there seem to be a lot fewer politicians willing to propose any of the real profound changes like the ones that came a century ago.
When you think about it, Comcast NBC Universal is doing the same stuff with Aereo.
Broadcasters get a government monopoly on a finite resource. Broadcasters don't provide a good signal to all customers. Some customers want to use Aereo (that's not the only reason, but a real reason) instead of paying for cable.
But no, the broadcasters want to be paid for not-providing a signal to those customers. Instead of investing to improve their infrastructure, they want to be paid for not improving it. With a straight face.
Of course it's especially ridiculous in the case where it's Comcast NBC, and they are also an ISP. If Aereo wins at SCOTUS, and grows big enough, Comcast NBC can get their pound of flesh as an ISP like they do with Netflix.
> But no, the broadcasters want to be paid for not-providing a signal to those customers. Instead of investing to improve their infrastructure, they want to be paid for not improving it.
This is the best one paragraph summary of the Aereo case I've heard. Certainly makes it absolutely plain why it needs to be allowed to continue.
This image http://www.willchatham.com/wp-content/uploads/2014/01/net.pn... has been making the rounds on reddit, surprised it hasn't appeared here yet. This is the most visceral/succinct explanation of the consequences of allowing the ISPs to have a "fast lane"...
It's going to take a lot to fight the likes of Comcast in this one, which means Netflix will also need support from the community - so Netflix, please stop it with the boneheaded DRM push on the web. Guess what, if you are going to use P2P, you're going to need a plugin or native app anyway. So just stop trying to corrupt W3C already in a pointless move that can't and won't stop piracy anyway (which I'm sure you already know).
Is it Netflix fighting for DRM, or is it the content creators demanding DRM and Netflix simply wanting to move to HTML5 while still being able to continue licensing shows from the creators.
"Though Comcast doesn't disclose it specifically, analysts have estimated that Comcast's profit margins on broadband service are 80 percent or higher. In 2008, Sanford C. Bernstein & Co. analyst Craig
Moffett estimated Comcast's data margins at 80 percent, and Credit Suisse reported in fall 2010 that Comcast's gross margins on high-speed data were 93 percent."
It would interfere with customers' other uploads (like backups), it would probably lead to poor service (how do you use 2Mbps peers to deliver 5Mbps streams?), and it would create another loophole for Comcast to degrade service that we'd spend months arguing about.
You only need two or three major website who blackout their services to the Comcast user and redirect them to a page explaining that situation. Imagine if their subscriber can't access to Google and Facebook, and on that redirection page they target ads for alternative provider in your area. Would be nice to see how long Comcast would take to change their mind.
No it is not. ISP throttling their paying customers because of their internal nonsense is user-hostile. Informing consumers that they're being undermined by their ISP is quite possibly the most pro-consumer act you could do. If it gets us one step closer to treating the bandwidth like electricity or water, it's a good thing.
"Facebook is a little slow because my ISP might be throttling, but I'm not sure, it could also be my computer"
"I cannot access Facebook at all because Facebook blocked my ISP"
Getting users pissed off at web services is not a good way to effect change. We need users on our side, so we can't have them think of us as crybabies.
>> "In this way, Comcast is double dipping by getting both its subscribers and Internet content providers to pay for access to each other."
..and this:
>> "Put simply, there is one and only one way to reach Comcast’s subscribers at the last mile: Comcast."
...should be illegal. This really torques me. This smacks of how AT&T acted in the 80's when I was in my 20's and starting out on my own and how Microsoft acted in the 90's before they got busted, which is what lead me to Linux.
Except Facebook's customers use Facebook to access Facebook's data. Comcast's customers use Comcast to access the internet's data, not Comcast's data. At least until the definition of Internet Service Provider is changed.
Same problem exists in France with the "Free" (actual business name) ISP.
It's a large ISP here, and it throttles access to youtube (and probably other content providers). It has been so for years. For some reason, there is no public awareness about that, people just say "my internet is not working well".
It's an act of aggression against their customers, to me, since all ISPs are marketing their offers around connection speed. And it's violating net neutrality, too. I don't get how it does not become a major PR problem for ISPs doing that.
Not just Iliad's Free - I know first hand that others admit internally that they are doing it too... But they choose to let Free be publicly vocal about it while they are just as violent behind closed doors.
Yes, I've heard some vague reports from others ISP users too, but not anything I have measured of even just experienced first hand.
That's the reason why I've switched to the new ISP services from ovh, though, as they made net neutrality their first marketing point (but for the same price, you don't have tv, if that matters).
Has there been discussion of passing this fee on to customers who access from Comcast in the form of a surcharge? If Netflix hiked prices for customers who were connecting from Comcast networks, it would encourage news of this business practice to spread.
Netflix presumably was paying Cogent a lot of money for all those 10Gbit uplinks. Now a few cross-connects and BGP tweaks later, a traceroute shows Netflix packets leaving a Netflix server and going more directly onto Comcast's network. I can only speculate that they turned on some dark fiber to build cross-connects between their cages and Comcast cages. Maybe they were already in the same building...
How much did Netflix's Cogent bill go down? How much of the "work" that Cogent was doing is Netflix now doing themselves? How many hops of the traceroute are owned by Netflix, and how many hops by Comcast?
When I rent a rack in One Wilshire in LA, CA, it comes with power, cooling, and a network link. When I pay let's say $1000/mo for that 1Gbps link, it's not just the link from my rack to their router that I'm paying for. And when I send 1Gbps of packets up through that link, it's not just CoreSite at One Wilshire carrying the cost of delivering those packets.
At some point, yes, I do expect Comcast to get their "fair share" of that $1k/mo. If Comcast is PAID zero dollars for delivering that 1Gbps, then I want this free bandwidth gravy train to propagate all the way back to me. Comcast, please sign me up for 10 of those free 10Gbe links you're giving out, ok? Just tell me where to rent space where I can get those free 10Gbe cross connects to your network.
But then, you can also flip the perspective and ask the question, why doesn't Comcast have to pay for all their bandwidth? Shouldn't they be paying $1,000/mo for their Gbe's?
So why does bandwidth cost money, or more specifically when should bandwidth cost money? Price, as always, is just a means of efficiently allocating scarce resources. Price can only possibly be zero if there is no scarcity. Is it possible there's no scarcity of bandwidth on Comcast's network? Now that would be comical!
I think the answer lies partially in another development. Ask Netflix to chart the cost in terms of Watts and Rack Units to deliver 1Gbps of video streams. My theory would be the cost in terms of space and electricity to generate 10Gbe of video streams has dropped orders of magnitude. This means bandwidth will be an increasing portion of your bill no matter what. It also puts more pressure on scarcity of bandwidth when the ease of which you can generate 10Gbps of useful traffic is down to just a few servers.
Comcast gets paid $50/month per subscriber for that bandwidth. It costs Comcast less money to peer with Netflix than to get it from cogent. Comcast wants Netflix to pay to solve a problem that Comcast's customers have.
> Is it possible there's no scarcity of bandwidth on Comcast's network?
Absolutely! In fact, that's part of what the chart shows; when Netflix peered with Comcast, the video quality (which is a proxy for bitrate to the consumer) went up significantly, which indicates that there was no bottleneck on the Comcast network.
> Price ... is just a means of efficiently allocating scarce resources.
Not in the case of a monopoly. Comcast has a monopoly on a large fraction of the broadband market. They also compete directly with Netflix, so they will leverage that monopoly to make it as expensive as possible for Netflix to continue to compete with them. This is Comcast creating an artificial scarcity of paths from Netflix to Comcast's customers.
On another note, I have referred to Comcast subscribers as "customers" above; if agreements like the above continue, then Comcast subscribers are no longer customers, but rather the product that Comcast charges for efficient access to.
Lets assume Comcast honestly thinks that Netflix owes them money for the bandwidth. I know it's highly generous, but it's a fair starting point.
How does it costs Comcast less money to peer with Netflix than to get it from Cogent? Does Comcast pay Cogent too?
Of course the whole confrontation was manufactured, I don't think Netflix was ever arguing that Comcast needed to boost its entire buildout just to support their video streams, and by the way they shouldn't pay a cent. Comcast doesn't have to wait till its pipe are full before controlling traffic flows.
If Comcast had any competition they would be the ones paying for the better connection to Netflix. It is supposed to be in Comcast's interest to provide their customers with the access they want or the customers would switch to another provider that does offer it. Comcast is selling access to the Internet are they not?
>Comcast is selling access to the Internet are they not?
There's the rub. As a cable company, Comcast are not just selling internet access, they're also selling access to TV and Movies, infact historically this is all they sold and they're still stuck in this mindset. Netflix threaten to disrupt this lucrative business.
Everyone knows cable make money from bundling a whole bunch of rubbish channels together with a handful of high quality channels and charging a lot for it. As soon as someone (Netflix) offers to let you just watch the good stuff, for less dollar, Comcast's profit line is going to take a hit. It's no coincidence that it's Netflix who has to pay extra whilst the non-threatening Google/YouTube/Facebook/Porn get past without hassle.
They should use classic Cable TV vs. Local Broadcasters tactics:
Block all IPs from Comcast and force them to pay access to your content...which happens to be the exact amount they want for the Tolls.
If we consumers have to pay to access to content why shouldn't ISPs?
That might work if Netflix had compelling content. Maybe they do, I don't know. But it is not helping Netflix that Comcast is ranked near the top for ISP market share and also owns a ton of its own content (NBC Universal).
This makes me so mad. I kind of wish all of the Big Internet providers like Level3, and Sprint would team up and block all traffic in and out of Comcast. Maybe redirect all http traffic to a page listing alternative ISPs.
Reading this actually makes me hope that Facebook gets their wifi-beaming balloons up and running quickly. I'd like to support Comcast as little as humanly possible.
I think it is actually Google looking to deploy Wi-Fi beaming balloons and Facebook is looking at solar powered drones. I vote for Google over drones...
The correct way to look at this is to think of the drones as very low, maneuverable satellites, not airplanes. They're up in unregulated airspace above airliners and above most weather.
The lower you are, the lower the latency. Think of a laser-based optical network that connects the drones, with laser links going down to earth as well.
When you think of the control required to ensure a quality of service and full coverage, the balloons seems like a terrible solution.
Why is what? Are they any different? Are you balking at the word "drone"? How about "unmanned aircraft", which describes both Google and Facebook's approach? It's not like Facebook will be using the same "drones" the Air Force uses to bomb ground targets.
I know Facebook's drones are not the same as the military drones but... The idea of using "drones" leaves a bad taste in my mouth since they are currently used as immoral killing machines. I do agree with robszumski that drones may be a better bet then balloons.
Hope Google provides the service as I prefer them as a company. Yes, both are data mining beasts but at least Google does some cool things and they have services I actually use. Not one Facebook service comes to mind that I use...
That's what I was expecting you'd say, but I had really hoped I was wrong. Again, a drone is an unmanned aerial vehicle, so Google is using drones as well. In the comparison with military drones, you might as well say you don't want to see Google's balloons because the Hindenburg was such a disaster. Completely different beast.
I would wager that there are more drones out there being used for entertainment than there are being used for killing... we call them quadricopters and you can buy them at Wal Mart.
Your repeating yourself and I am being down-voted? No sh*t there are different types of drones. I simply meant they leave a bad taste in my mouth because how they are mostly being used now but did acknowledge drones may be a better bet then balloons.
Oh, the Hindenburg is a bad analogy as it's purpose wasn't to target people from the sky to kill, it was an accident. Now, if Google is going to be using Hydrogen instead of Helium in their balloons like the Hindenburg then you would have a point.
I see all these people saying Comcast was feuding with cogent... that is just not the case. For anyone in the hosting business we all know cogent is the cheapest option, and a pretty spotty service as a whole. What do we do when our customers complain and complain? Well, we seek out more reliable options and spend the money needed better serve our customers.
So why does Netflix get to complain? They have already spent the money for the ridiculously one sided peering(Netflix sending tons of traffic and Comcast sending close to none). You would think Netflix would be happy to spend this money to better their service! Right? Or is it so detrimental to their cash flow that they would rather complain? They should have just told their customers "hey, we heard you were having some problems with our service, look what we did for you!"
In the end this is just a smear campaign to distract Netflix customers from the price hike that is coming. It is obvious Netflix was going to raise its subscription prices eventually. So why not point the blame to a company who is already disliked. The majority of HN seems to be pretty smart people, it would be a shame if you fell for this rather underhanded ploy.
Who sends traffic is just easier to measure, the person receiving it may want it more and in the Netflix case they are requesting the traffic.
In any case all Comcast's traffic is for their customers, when peering needs to be balanced to be free is when to parties are offering transit to other networks beyond themselves.
I really hope Netflix has what it takes to do what is necessary if it comes to that point. And what it is going to take is for Netflix to simply blackout Comcast.
But actions like that can be easily spun out in the media (media of which has very close ties to Comcast) to make Netflix look bad.
What would make it better is for all of the major players (Google, Facebook, Twitter, Apple) to collectively blackout Comcast, whilst educating the public on how they are being used and screwed at the same time.
If Netflix are right an Comcast are providing insufficient connectivity through to all of the major transit networks (rather than just Cogent) it feels to me as if Comcast are not providing the internet connection speeds that they are selling. It would be interesting to see what would happen if Netflix bought from all the many transit providers and spread the load across them, would all of Comcasts general service grind to a halt.
The problem is sufficiently complex and boring to favor Comcast. What I would like to know is this: It is already a regional monopoly in high speed internet because no on can touch their speeds, why would anyone consider extending their monopoly nation-wide?
I think the real problem is unlimited internet connections.
If ISP customers paid per-GB and/or per-Mbps, then the ISP shouldn't care if it's Netflix or Hulu or YouTube being consumed.
The issue is that today, it's difficult to get a good feel for how much bandwidth is being used at any given moment. Unlike a water faucet, our devices don't really have a knob we can adjust the speed with.
None of this has been a problem here in Sweden, where the infrastructure is solid and easily supports end users with 100 Mbit home connections without me hearing any complaints.
The quality of infrastructure is the result of political and cultural factors, which are therefore the more underlying/real problem.
Do you have a choice of multiple ISPs? If yes, then that's the critical difference vs the U.S., not the infrastructure. If no, then why has your monopoly ISP built such good infrastructure?
France's situation: national public telecom company (France Telecom, now Orange) built the network, private competitors appeared, Orange got privatized, competitors didn't have the network to be everywhere and allow proper competition so the law forced Orange to allow other ISPs to use Orange's network for a fee (so I could be a client of, say, Bouygue Telecom, but using Orange network to join Bouygue's).
A huge and easy justification for that is that Orange's network had mostly been built using public funds since it was a public company at the time.
Since then, several competitors have built their own network, but most still use Orange's sharing for remote place, small villages etc where it's not worth it for them to deploy their own.
So your question is hard to answer. Why has the monopoly ISP built the infrastructure ? Because it was a public company, and if you wanted your house to be connected to the network they had to do it. Why have the others built their own ? Because it was much cheaper for them long term, especially since they also use it for their cellphone network.
And from what I see, it seems to be the same in a lot of Europe, the national companies made the original network, and competitors piggy backed on it until they had their own.
(I've oversimplified of course, but that's the gist of it)
the mistake is to think about data as water and wires as pipes, this is a classic but understandable fallacy for those that do not grasp the basic tenets of this topic
In his book Tubes: A Journey to the Center of the Internet, Andrew Blum presents a partial defense of the late senator's analogy:
"I have confirmed with my own eyes that the Internet is many things, in many places. But one thing it most certainly is, nearly everywhere, is, in fact, a series of tubes."
Yes, Tata Group from India owns an Internet backbone (formerly VSNL & Teleglobe). Netflix traffic used to go through Tata to get to Comcast until Comcast deliberately congested it.
This is a very public battle between two companies about who should shoulder the cost of delivering video to your home.
Comcast's argument is that they have built a network at great expense for carrying content that others make a profit from, and they want to capture at least some of that value.
Netflix's argument is that they have built a CDN to deliver that content to Comcast's doorstep, and that Comcast won't accept it.
In my mind, Netflix charges money for their service and is not entitled to a free ride. The fact remains that running an ISP is not free, and you can't scale out a network faster than the cash flows (of which, yes, profits are a part) allow. But think about this: Netflix has a 36% gross profit margin, and Comcast's cable/Internet business has an 11% gross profit margin. Netflix's cost structure is lower in large part because Comcast does a lot of the expensive work for them (i.e. last-mile network maintenance and distribution.)
Netflix drives significantly increased Internet traffic usage, and Comcast's investors should not have to accept lower profit margins so that Netflix's investors can enjoy larger margins because of investments that Comcast's investors have made.
I would have a lot more sympathy for Netflix if their margins weren't so high. It just comes off as greedy when they have a 36% gross margin.
You are forgetting one central point, customers pay Comcast to receive netflix. It is their customers requesting this data, if Comcast truely has issues with the costs of this it is customers who should be paying more, not Netflix.
The truth of the matter is this has absolutely zero to do with costs, and everything to do with extracting rent via market position.
In this case, Comcast has two choices: raise prices on all their customers, or raise prices on Netflix's customers by charging Netflix for their increased usage. Prior to paying Comcast for transit, if Netflix decided to push 4k streams, that decision carried no repercussions for Netflix. Now they have to do a business case to see if the increased cost is worth it.
I know it's not a popular opinion on HN, but I don't think it's unreasonable for Comcast to ask Netflix to shoulder some of the cost. It also has the added benefit of incentivizing Comcast to provide faster service. If they can charge Netflix more for pushing more data, then it directly leads to more profit if their users consume more data.
> if Netflix decided to push 4k streams, that decision carried no repercussions for Netflix
This is not true. Netflix has to pay for transit of the bits leaving their servers just like everyone else. They just pay their transit provider for this, not Comcast (well, except when it is Comcast, but that's beside the point).
The right ways for Comcast to keep their costs under control re: video streaming would be to either meter pricing or drop rate limits. The problem is that metering is unpalatable to their customers, and rate limit drops would expose the lie that the entire cable modem marketing engine is based on (that last mile downstream throughput maximum is the only number that matters).
> This is not true. Netflix has to pay for transit of the bits leaving their servers just like everyone else. They just pay their transit provider for this, not Comcast (well, except when it is Comcast, but that's beside the point).
Netflix doesn't have a transit provider any more. They are piping directly into Comcast and wanted to do so for free. I want my website to be delivered to all Comcast customers for free, do you support that?
The alternative is for Comcast to buy enough transit to receive those bits from Netflix. I'm sure Netflix would be happy enough to deliver those bits to Comcast via transit, if enough capacity was available.
Shouldn't Comcast be required to purchase enough capacity to provide high-quality service to their residential customers? SFI between a content provider and an eyeball network should be a mutually beneficial cost reduction, but Comcast has enough market power (captive broadband subs) that they can get away with intentionally not buying enough capacity to serve their customers.
Comcast does transit with many peers, but Netflix doesn't do business with any of them because they want to lower their own costs by cutting out the middlemen.
A peering agreement is free when both sides are accepting traffic for the others, but that's not what is happening in this case. Netflix -> Comcast is one way.
Still waiting for an answer to my previous question, should Comcast delivery my personal website to their customers for free?
Netflix never attempted to directly send data one-way to Comcast for free. It paid tens of millions of dollars a year to a transit provider, Cogent, that has a peering agreement with Comcast. It only stopped paying Cogent and started paying Comcast (an estimated $25-50M/year) after Comcast refused to provide adequate connectivity between its customers and Cogent for months, despite their links being maxed out at peak hours. Netflix was becoming a casualty of a long-term contract dispute between Comcast and Cogent over peering fees. Comcast was being greedy, trying to get paid twice for the service its customers already pay it to provide.
Cogent is the bottom of the barrel transit provider. You use Cogent when you have a lot of bits to push and don't particularly care if they're going to get there (seriously, Cogent has like 25% packet loss at peak times.) Most companies use Cogent for overflow; i.e. they purchase enough transit through other providers for 80-90% of peak load, then fill out the rest with Cogent. It's not that Cogent is a terrible company or anything; they just provide an inferior product at an appropriate price point. Cogent wasn't interested in paying Comcast for better service because quality of service is not the product that Cogent provides. Their business model is to take advantage of their free peering status to sell unreliable transit at rock-bottom prices. Again; it's not evil or anything, but when you use Cogent that has to be what you expect.
But Netflix was using them for almost 100% of their traffic delivery. That's insane. There are other transit providers they could have paid more to and gotten better service, but that would have meant higher costs.
The whole irony of this situation is that I've heard rumors that Netflix is paying less per gigabit to Comcast than they paid Cogent while getting better service than they would get from a top-tier provider. If that's true, other companies will be lining up to get a similar deal because they'd just be cutting out the middlemen.
Netflix is attempted to send one-way data right now. That's what their whole campaign is about, shifting their costs onto everyone else. They don't want to go back to paying Cogent, that was much more expensive.
Where are you getting this info that Netflix didn't want to pay Cogent and Level3?
It seems like Comcast entered a shitty peering arrangement with Cogent and decided not to upgrade the interconnect links in retaliation. Netflix gets caught in the crossfire and ends up having to pay Comcast directly to avoid a few million angry customers. Is there any transit provider that Netflix could have gone with that has enough interconnects with the Comcast network? If not, Netflix is forced to pay whatever price Comcast names for their customers to receive adequate video streams.
Yes they do. They buy transit from (e.g.) Cogent. The reason Netflix has peering agreements with Comcast is that Comcast's peering links with those providers don't have the capacity to carry the volume of traffic its users are requesting. I suspect Netflix would be equally happy if those peering links just got fixed.
> I want my website to be delivered to all Comcast customers for free, do you support that?
Sure, if your website has a peering arrangement with Comcast. Comcast isn't required to offer one to you, though.
Anyone that's willing to deliver content at all interchange points into the Comcast network should be able to do it for free. That's the only policy that makes sense. Once the traffic is inside Comcast's network it's their responsibility to upgrade the network to be able to deliver the service their clients are paying for.
Once again, it is the user who requested that data from Netflix. The user pays the ISP. The ISP is already immensely profitable. Because of these things, the ISP has no need or ethical right to demand more from the user or from Netflix for providing a service the ISP already promised and the user already purchased.
The ISP doesn't promise to make every website / service on the internet as fast as possible. It's up to every service to make themselves fast by using CDNs. In the case of Netflix, they have become big enough where they now want to be their own CDN.
The ISP sells "Internet" access, so it's reasonable to expect them to meet customer demand at common Internet peering points, right up to the edge of their net profits plus a reasonable return for investors.
It also has the added benefit of incentivizing Comcast to provide faster service.
Actually it gives them an incentive to provide slower Internet service. This encourages content providers to pay to use the Comcast private network instead of the Internet.
The decision to push 4k streams certainly would carry repercussions for Netflix both from a storage and transit standpoint, even absent paying interconnect fees to Comcast.
As a cable subscriber, I expect that when paying for N Mbps of bandwidth, I'm entitled to N Mbps of bandwidth of the content of my choosing. If Comcast's pricing model needs to change to a cost-per-gigabyte model in order to cope with the increased quantity of data customers consume, so be it. But sneaking the costs onto Netflix's tab effectively shifts Comcast's costs to all Netflix customers, allowing Comcast to artificially lower their prices relative to smaller ISPs without the market share necessary to effectively extract rent from Netflix.
> As a cable subscriber, I expect that when paying for N Mbps of bandwidth, I'm entitled to N Mbps of bandwidth of the content of my choosing
100% of the time ? not going to happen
So let's say I'm an ISP and I bring a fiber to your home, and gives you a gigabit ethernet port. You cannot expect all current and future customers to be able to use 1 Gbit/s at the same time. You would need a big non blocking switch with as many ports as subscribers, the technology for this does not exist once you reach a large customer base.
Now do you want me to shape your link to 0.1Mbit/s, because that's the only thing I can guarantee if all customers uses their link at the same time ? Or you'd rather have the 1Gbit/s possible bandwidth ?
Which one is better:
1) guaranteed 0.1Mbit/s for 10$, 5Mbit/s for 100$, 1Gbit/s for 4000$
2) possible 1Gbit/s for 20$ ?
If you take the globalized approach, you cannot have business like Netflix, they destabilize the equation.
I'll take the non-false dichotomy, please. Having enough capacity to meet the streaming demand of your current customers during peak times is a solvable problem that doesn't require a full gigabit for everybody, all the time. The pricing model required to support this while still looking attractive to customers I'll leave to Comcast's well-funded marketing department.
In order to make that happen, of course, we'd have to live in a fanciful world where shifting last-mile delivery cost to content providers wasn't an option so the painful process of exposing this cost to customers couldn't be hidden in a rat's nest of perverse incentives that benefit the most entrenched corporations. (Ironically, and despite its protestations, Netflix's ability to pay this rent is a barrier to entry for its own future competitors.)
> Having enough capacity to meet the streaming demand of your current customers during peak times is a solvable problem that doesn't require a full gigabit for everybody, all the time.
With adaptive streaming technology, there is not such thing as "today streaming demand"
If whenever you raise the pipe size, it is automatically filled by an higher resolution/quality/bitrate stream, it is a never ending game.
What I mean is that, whatever the state/quality of ISP networks is or could have been, we are doomed to face that "Netflix problem". If networks were better, then Netflix would already offer 4k or dual/triple HD streams and so there would be saturation anyway.
> (Ironically, and despite its protestations, Netflix's ability to pay this rent is a barrier to entry for its own future competitors.)
Actually, moving all Netflix traffic to private pipes will free the shared ones, so a new freeloader can use them and cut Netflix prices ;)
When they sell me the service, they don't sell it as possibly 1 Gbit/s. They sell me 1 Gbit/s. If they can't deliver, it is time to adjust that promise when I purchase the service.
How does this incentivize them to provide faster service?
From my point of view, if they're allowed to let their service degrade so that they can turn around and charge Netflix more money to make it go faster again, how is that an incentive to provide a faster service? To me, this is an incentive to provide slower service so that they can charge companies like Google and Netflix more money.
Edit: Funny this sentence can be turned around on you by clarifying who requested those streams.
Prior to paying Comcast for transit, if Comcast Customers asked Netflix to push 4k streams to them, that decision carried no repercussions for Comcast Customers.
Yes, but Netflix has the option to not provide those streams to Comcast customers if it's not economically viable for them. As it stands, most Comcast customers only request 4k streams because Netflix defaults to the highest bitrate. Basically, by changing a default setting, Netflix pushes bandwidth and creates a tragedy of the commons situation.
It incentivizes Comcast to provide faster service for those who pay. It incentivizes them to provide slower service for those who don't.
We can comfortably say that 99.9% of the services on the internet are not going to pay Comcast. The big players might, but there's an extremely long tail.
Thus, for most of the internet, Comcast encounters a conflict of interest. Its actual paying customers want higher speeds. But it has every incentive to reduce speeds for services that aren't paying for more.
I pay my ISP for access to the entire internet. If they screw with my access to a service simply because they didn't pay, that makes no sense.
If I'm paying $75/month for high-speed internet access, that means the entire internet, not just the parts they get kickbacks from.
I pay my ISP for my access, Netflix pays their ISP for their access, and we get to talk to each other. That's how this stuff works. Comcast charging Netflix is double-dipping, and is directly opposed to their supposed purpose of serving me, their customer.
(Hypothetically. I'm no longer a Comcast customer, thank goodness.)
Edit: I usually hate real-world analogies, but I think it could be instructive here. Let's take the USPS as an example of something in the real world that looks a lot like an ISP.
I can pay USPS $5.60 for a flat rate Priority Mail envelope, and they will deliver the contents anywhere in the country within 2-3 days. Doesn't matter who's getting it, and doesn't matter what's in the envelope as long as it fits.
Now, let's imagine that some company ends up receiving a lot of mail. Providing mail service is expensive, right? It takes a lot of trucks and postal workers to deliver all this mail to this one company. Surely they should be charged for what they're using, right?
Of course, this company says no. These people who are sending us mail already paid USPS. It's their job to deliver mail to its destination once they get paid to do so. Why should they get paid twice to deliver the same piece of mail?
In retaliation, to "convince" this company to pay, USPS starts slowing their mail. Those trucks and mail carriers are expensive, we need some of them elsewhere. You'll just have to wait.
And as a result, because USPS is demanding that this company pay to receive mail that I already paid to send them, my "Priority Mail" starts taking 4-5 days to arrive instead of 2-3.
I hope it's obvious that USPS is completely unreasonable in this hypothetical scenario. I see no difference between this and Comcast charging Netflix for service I'm already paying Comcast for.
> I pay my ISP for my access, Netflix pays their ISP for their access, and we get to talk to each other. That's how this stuff works. Comcast charging Netflix is double-dipping, and is directly opposed to their supposed purpose of serving me, their customer.
Netflix doesn't pay an ISP. That's the whole point of the arrangement, it goes straight from Netflix to Comcast. There is no middleman. So their peering arrangement is the equivalent of "paying their ISP".
To say that Netflix should not have to pay for internet traffic is ludicrous. I want a piece of that. Comcast should hook up their network to my servers so that my personal website is fast for Comcast users. For free of course.
Your assertion that Netflix "doesn't pay an ISP" is at odds with their description of having previously paid Cogent to carry traffic over long distances. Now, "Netflix is itself shouldering the costs and performing the transport function for which it used to pay transit providers."
In other words, you're describing a situation in which Comcast blackmailed Netflix for access to Comcast's customers as one in which Netflix claims they "should not have to pay for internet traffic". This is detached from reality.
How was Netflix "blackmailed"? Cogent failed to provide the service they needed so they are doing it themselves. Doing it themselves is saving them a lot of money.
Now they're attempting to shift costs onto everyone else. And they're using the public's distrust of ISPs to get the public to pay for Netflix's business expenses.
Previously, they paid transit providers for everything, but Comcast wouldn't allow them access to their network at sufficient speeds, so they were forced into paying Comcast directly.
In other words, we're already in the double-dipping situation I'm decrying. The "we each pay our ISPs" scenario that I described and that you quoted is now, sadly, a hypothetical.
Comcast and Cogent disagreed on who should burden the costs. Because of this Netflix went around Cogent and are doing it themselves, and saving a lot of money by doing so.
Now they are using the biased tech media to create a campaign whereby they will lower their costs even further and shift them onto others (including you and me).
Netflix went around Cogent because Cogent couldn't get Comcast to bump up their peering capacity. Comcast wouldn't bump up their peering capacity because they're a monopoly in many places and have no incentive to improve.
In a reasonable free market, none of this discussion would matter. The agreements would all shake themselves out in the marketplace. But give a company a monopoly and suddenly they're not subject to the same forces. Between Netflix, Cogent, and Comcast, only one of them is a monopoly.
You pay for guaranteed bandwidth within Comcast network. Not the whole internet.
Which gets back to the whole issue at hand. If Netflix wants to improve their service they need to pay for better transit. So they cut out the middleman and go directly to the source of the majority of their viewers.
This is no different than Twitch, Google, or Microsoft direct peering with Comcast. They see the value in allowing people to access their services as fast as possible, so they spend the money to do so.
This is internet 101. Netflix just wants everything for free.
If "not the whole internet" then why does Comcast provide any bandwidth outside of their own network?
I accept that they can't guarantee bandwidth once traffic gets out of their network, but as long as both sides and everything in between has adequate bandwidth, shouldn't I be able to use it?
That's nice and all but the fact of the matter is Comcast doesn't give a crap about its own network or its customers. In the last 6-8months their PNW, SoCal, Colorado, and NY/NJ routing centers have sat, 24/7, at significantly higher than 50% util (Think closer to 70% at "lows" and 90+% at peaks). As far as I can tell, they have no real plans to fix that.
Comcast's CEO is a smug bastard who doesn't think his customers want faster speeds, or cheaper internet, or higher level of service than Comcast currently provides. The CEO doesn't care that his customer satisfaction and service ranks dead last -- he things that's just dandy. Comcast's CEO thinks they were ranked the "Worst Company In America" not because they might be the least liked in general but because a few, vocal, not-representative people on the internet were mean.
I guarantee you they do give a crap. But there's only so much money available from subscription fees to provide network upgrades, and Comcast spends ~$6-8 billion a year doing so.
I've seen nothing of this low level of service or high cost; I can get a 150mbit connection for $89/mo. It's not Google Fiber, but it's really not bad at all (especially compared to FiOS, which I have and hate with a passion). I mean yeah, their customer service is shit, but the Internet service itself is good.
I pay that price for that service, but comcast doesn't deliver that service to me, because they decided to throw away the existing play book of Internet peering.
Can we class action sue comcast?
Comcast's customers are paying Comcast to deliver Netflix's bits to them. Why does Comcast need to be paid for those bits twice?
If we're going to throw out some numbers, Netflix's net income was just $112M on $4.3B in revenue last year (2.6%). Comcast's net income was $6.82B on $64.66B in revenue (10.5%). Netflix's spend on content acquisition and production is increasing, which means that 2.6% is shrinking; they're already cash flow negative IIRC. Not much room to start doubling up on bandwidth fees they already pay.
> Comcast's customers are paying Comcast to deliver Netflix's bits to them. Why does Comcast need to be paid for those bits twice?
Because the customers have to pay for the bits being transmitted to them over Comcast's network, but Netflix has to pay for the privilege of competing with Comcast's own video services. It isn't coincidental that this became an issue after the FCC's Open Internet Order -- which specifically forbade discrimination against competitors to a broadband provider's own video services -- was struck down. This was pretty much the prime example of what it was intended to prevent.
Because Comcast does not incur cost per-bit, they incur cost based on the maximum required capacity of their network. Netflix significantly increases those costs on a per-customer basis.
No, customers who want to watch Netflix increase those costs. Which really means, customers who want HD video streaming bandwidth increase those costs. To which the right answer is, build more bandwidth and charge the customers for it. Customers who want to watch Netflix, or any other HD video streaming service, can pay Comcast for "HD streaming" bandwidth; customers who don't can pay for the bandwidth they need. None of this should have anything to do with Netflix.
I understand that Comcast doesn't like this because it makes them a simple seller of bandwidth instead of a "provider of premium services", or whatever marketing mantra they're using this week. But if customers do not want their "premium services", but just want bandwidth, the solution to that is to build better premium services, not to try to get paid twice for one stream of bits.
No, Comcast customers who use Netflix significantly increase those costs. It's not Netflix's doing.
If the usage pattern of Comcast's users is changing such that Comcast's internet offerings are no longer bringing in enough revenue to support usage, that's a problem with the offerings. They need to charge more, or charge differently.
However they change it (if indeed they need to change it, and it's not just a money grab, which seems more likely) then they need to charge based on how their users use the service, not what specifically they're using it for. If peak usage is a problem then charge more for peak usage, but there's no reason that A should pay more for the same peak usage as B just because A's peak usage involves Netflix.
Every single Comcast customer is already paying for a speed that is, at minimum, 5 times higher than required by a Netflix super HD stream. Netflix isn't trying to buy any special handling for its traffic.
Comcast sells access to the internet to households.
To provide the service of access to the internet, Comcast must connect with other networks that the stuff on the internet is connected to. This is the business they're in. That's specifically what customers are paying for when they buy internet access: Comcast gets money from customers, that money goes to connecting Comcast to the internet, then connecting Comcast to the customers' homes. Thus internet service is delivered.
One of these networks they must connect to in order to sell internet access is Cogent's. Behind Cogent were hundreds of thousands of sites and services, including Netflix. These customers all pay for their internet connectivity. Cogent's being paid to connect these customers to the internet (Netflix's bandwidth bill was at least in tens of millions per year), Comcast is being paid to connect its customers to the internet, they both need to connect with an adequately sized pipe to meet their obligations to their respective customers. They're both paid handsomely for the service.
Comcast was being derelict in its duty to its customers by refusing to maintain a wide enough connection between its network and Cogent's network to reliably deliver access to that part of the internet. Customers are paying for that service (unlimited access to the internet at 15+mbps), but if the part of the internet they wanted to reach was Cogent's, they couldn't get anywhere near the advertised speed, and videos sometimes buffered or downgraded bitrates at prime time.
Netflix should not have had to pay to correct this, Comcast should've. The connection between Netflix and Comcast customers wasn't poor because Netflix refused to send the bits fast enough, but only because Comcast refused to accept them fast enough. That's Comcast's responsibility to address. Providing that network on their end is the business they're in. Netflix is not trying to pay for any special "fast lane" speed that other sites don't get, and its' already paid for its internet access, it's just trying to avoid going out of business as a side effect of Comcast screwing its own customers by failing to provide the service it's been paid for.
> Netflix should not have had to pay to correct this, Comcast should've. The connection between Netflix and Comcast customers wasn't poor because Netflix refused to send the bits fast enough, but only because Comcast refused to accept them fast enough.
One thing to add to that is that Cogent was happy to deliver the bits fast enough.
You've failed to explain how Comcast was in the wrong in their dispute with Cogent. Until you do that, it's just 2 businesses that could come to an agreement, a thing that happens thousands of times every day.
In my mind, Netflix charges money for their service and is not entitled to a free ride. The fact remains that running an ISP is not free, and you can't scale out a network faster than the cash flows (of which, yes, profits are a part) allow. But think about this: Netflix has a 36% gross profit margin, and Comcast's cable/Internet business has an 11% gross profit margin. Netflix's cost structure is lower in large part because Comcast does a lot of the expensive work for them (i.e. last-mile network maintenance and distribution.)
When you start looking at the question like this, you're abandoning any technical argument and instead move the conversation into "well, look at all this money moving near my proximity, I should be getting my beak wet." This is a pretty good warning sign someone is/has started framing the conversation on something other than the issues at hand.
Fairness in profit margins is a red herring at best. I like the idea of network neutrality, but I have a much much much stronger fundamental belief: nobody is entitled to a business model. That kind of thing shouldn't even be given the slightest bit of consideration when setting policy.
Unless the explicit purpose of a policy is to create a business model, as is the case in Intellectual Property. Creation, improvement, destroying, or diminishing of business models is a lever that is rushed to way too quickly in some nonsense idea of "fairness" for people that are too used to cooperate welfare and don't believe in anything but entrenchment of middle of the road "compromises" that just serve as regulatory entrenchment of the status quo.
Comcast charges me, the customer, for Internet access. This means access to any network that's present in a meet-me room. That's how the Internet works - each participant pays their share for getting to the exchange. Comcast knew this when they got into the ISP business.
Then, suddenly, they are complaining that the customers that already bought their service, want to use the service they paid for. The only reason they can get away with this is that they have an effective monopoly.
Rent seeking monopoly middle men will squeeze both ends of the connection until the monopoly is busted.
Their gross margins are the result of their positions in the value chain which, in turn, determine their cost structure. What Comcast is trying to do is to extract more rent out of its position.
As others have commented, consumers already pay for both services.
If it's infrastructure cost that you're concerned about I've read somewhere (Bloomberg?) that Netflix offered to co-locate its equipment in Comcast's data centers but the latter refused.
That makes no sense. Charging Netflix for congestion their traffic creates on Comcast's upstream pipes has nothing to do with congestion or routing difficulties inside Comcast's network. Comcast still has to route "each bit to customer houses" in the current scenario of Netflix paying for access.
The fact that Comcast is eschewing the Netflix offer of free Open Connect CDN appliances (https://www.netflix.com/openconnect) tells me that this isn't an issue relating to upstream congestion, but rather with Netflix competing with Comcast's own VOD service.
It may not has to do with it today, but it will tomorrow.
As I said, generating traffic is easy. If there is no drawbacks in raising streaming bandwidth on Netflix side, they will continue to do so, 4k streaming, multiple streams per account, YouKnowWhat... all the way until all ISP subscribers will use 100% of their available bandwidth at all time.
I may pay for a XXX Mbit/s access, but I know that no ISP can sustain that XXX limit for all customers at the same time. And I certainly don't want my ISP to shape my fiber access to 10kbit/s because that's the guarantied bandwidth they can offer me.
Note that I'm in Europe, I don't have an resentful/hateful relationship with my ISP, but I can understand the US case is different because cable companies (used to) have bad business practices.
I think you mean "still have bad business practices." Comcast regularly gets rated as the worst company in the country by consumers (in some cases the worst entity, public or private, beating out the IRS for that coveted position).
nearly 1/3 of the US have only a single option for broadband internet, and that company is typically a relatively unregulated company with low business costs that charges high prices. That's just a recipe for bad behavior.
> Routing each bit to customer houses is the difficult part.
But the customer is paying for that to happen. The last mile is payed for by the customer. The deal between Comcast and Netflix is designed to handle the macro level bandwidth, which is only painful when you need to cover distances.
By putting themselves wherever Comcast asks, Netflix removed all but the last mile costs. Those costs are paid for by end users.
> Routing each bit to customer houses is the difficult part.
That's what CDNs are for. Which, as the article shows, is how Netflix delivers video to subscribers of every other major ISP. Why is it they can somehow route all that traffic to customer houses but Comcast can't?
> If Netflix was able to dispatch its traffic up to each of these squares
That's not how a CDN works.
Look at the diagram in the article, between the fourth and fifth paragraphs. Every other ISP uses transit networks to distribute Netflix traffic (like all other traffic) to its subscribers. Netflix puts its content on CDNs that are conveniently located to make the transit networks' job easier, by making "netflix.com" endpoints closer to large populations of subscribers. Somehow all the other ISPs manage to "route traffic to customer houses" using that arrangement. Why does Comcast have to be different?
I think maxhou is referring to the segment of the network from Comcast's (or any other ISP's) local data center to its box (I think they're called ATM switches) and eventually your house
Yes; so what? The traffic always has to travel over that segment, regardless of whether the ISP is working with transit networks or not. So whatever the difference supposedly is between Comcast and all the other ISPs, it isn't that. So what is the difference?
The difference is how Comcast interprets its competitive position compared to its peers. The barriers to entry in the last mile is high regardless of the ISP but it is willing to leverage/extract rents/push the boundaries more compared to others that may be building goodwill/not push the regulatory boundary/sitting and waiting.
In other words it is acting like a monopolist or a shrewd business.
In other words, there is no technical reason why it's any harder for Comcast to "route traffic to the customer's house" using the same arrangement as other ISPs. The reason it doesn't is that it's trying to get paid twice for the same traffic, which is a "business" reason (at least for some value of "business"), not a technical reason.
"Your video is buffering because Comcast is slowing your connection down. Click here to contact Comcast customer support. Click here to find Netflix Certified providers in your area."
Netflix and Google and Amazon and Apple all have great brands with high customer satisfaction. They should utilize all those eyeballs to funnel even more hate toward Comcast.