Broadband in America – We’re doing it wrong

For the last six months or so, the focus has been on Netflix and net neutrality. Should ISPs be allowed to rig their high speed networks, through various means like traffic shaping, peering and interconnection decisions, etc., to purposefully disadvantage or advantage an internet-based service or experience. The majority of the users on the internet seem to think that no, they shouldn’t. But corporations and corporate apologists seem to think that without giving high-speed internet monopolies or duopolies the freedom to do whatever they want will mean an end to broadband investment, and a loss of competitiveness as a whole.

First its important to note why regulation is so heavy-handed in this market. Its all about competition, or lack there of. The barriers to entry for building out a wireline or wireless high speed data infrastructure are immense. And those high barriers to entry mean there is a lack of competition. And that lack of competition means that the competitors who are established in the existing market environment need to be regulated to ensure that the consumer is protected, especially with a product especially as important as internet access.

The current broadband situation in America isn’t great, its not even good. We have more than one problem, even if the Netflix/Comcast issue is the most salient issue right now. Our issues are…

  • Speed: our speeds are slower than most other developed countries around the world, and even some developing countries have faster, cheaper internet in their cities than we do in the US
  • Usage Caps: most large ISPs that offer fast speeds have or will soon have data caps to prevent users from downloading a lot of data (read: protect their expensive TV packages from over-the-top competition)
  • Open access: companies like Comcast and Verizon have decided not to increase their network interconnect capacity with certain companies to keep Netflix traffic slow. Level3, a tier 1 ISP, even illustrated the habit of ISPs creating congestion to incent data providers to enter paid-peering arrangements.
  • Strategy Taxes/Internal conflicts-of-interest: companies that provide internet access have their own vested interests (television, phone) so they have internal conflicts of interest to providing an open, unfettered, fast internet experience
  • Lack of competition: it costs a lot of money to build a broadband network from scratch, so there is not a lot of incentive to enter a market to create an additional competitor, especially if a regional or nationwide competitor can create a price war locally until you run out of cash and go bankrupt.

How can we solve all five of these problems? New rules for Net Neutrality, or regulating ISPs as Common Carriers under Title II don’t solve all the problems. We’re still left with the monopoly/duopoly with companies we have now. They’ll begin work immediately to lobby for loopholes in the rules or legislation, while working to undermine the enforcement through lawsuits, and finally by getting people friendly to them in the regulator’s chair to keep the rules from being enforced.

It turns out our approach to internet access its entirely wrong-headed. Infrastructure should only be built once and from there, upgraded over time to meet demand. Otherwise we’re just wasting money. Let the market compete on top of the infrastructure, not by building separate infrastructure networks and selling proprietary access to that network.

But we can’t just throw it all out and start over.

Infrastructure

The internet is most often referred to in analogy as a highway network. Billions of miles of fiber optic cable criss-cross the globe carrying your request to watch that funny cat video for the 100th time.

There is robust, healthy competition at the “Tier 1” level – that is, the worldwide backbones that carry traffic all over the world. But when it comes to your metropolitan area, that last few miles of connectivity are often dominated by one or two competitive ISPs ((I moved recently, and actually lost my phone company – when I moved in, Centurylink wasn’t able to provide DSL at my house, so my only option is Cox)), in the same way your local roads are maintained by your local municipal agency (whether its a city or county).

This infrastructure is expensive. Its expensive to build – which is why we have so few options in the first place. Its expensive to maintain – everyone keeps using more and more data, and it costs money to add capacity to the network. So why are we building redundant infrastructure in many places?

Does it make sense to build two power grids to provide some fig leaf of “choice” for consumers, while the fixed costs of building and maintaining two separate power grids are factored into everyone’s monthly bill? Does it make sense for your community to build two sets of roads for the same purpose ((By this, I mean having two driveways, two street addresses, two non-interconnecting local streets that both connect to the highway network)).

But with our current state of broadband, its not as straightforward as that since the cable and phone networks were originally built for other purposes, and were re-purposed for internet access. Phone lines are inherently more handicapped than their coax sibling – DSL speeds are pretty much always slower than cable. But phone lines are more ubiquitous, usually required by regulation. Cable companies can pick and choose who they serve – if you’re not sufficiently urban or suburban, I hope you like DSL and Dish/DirecTV!

Wireless-only options are a very distant third place in the broadband game, encumbered by very low monthly data transfer caps, high cost, and questionable signal strength ((Masayoshi Son’s fevered dream of using Sprint’s 2.5GHz spectrum to compete with wireline home broadband is laughable – its a simple matter of physics, even 120MHz of shared wireless communication at QAM64 isn’t able to compete with 200MHz of coax bandwidth operating up to QAM4096 (the DOCSIS 3.1 spec) to offer 10Gbit speeds to your home)).

So we are left with different kinds of monopolies in the two different situations – in exurban and rural areas, DSL is your only option because you are unserved by cable; elsewhere cable ISPs can offer much faster speeds than DSL can manage and its really a not much of a competition ((My Cable ISP’s two main tiers are now 50Mbps and 100Mbps, while the fastest the DSL company can go in most locations its 10Mbps)).

Even in the best case (fast cable and fiber-to-the-home from the phone company), we’re left with duopolies that have similar products (triple play bundle of TV, phone and Internet). This means that, in the case of Verizon and Comcast, they have no incentive to help Netflix or future over-the-top video providers eat away at their revenue for entertainment services. Which means we still lack a competitive environment, as long as the ISPs have something else to sell you that could easily be provided over the internet in general? ((Cable and phone companies have started to get into the home automation and alarm markets lately, does that mean that AT&T should be able to handicap their internet traffic from ADT and alarm.com to harm competition?))

Loop Unbundling

One option for solving the problems would be to force loop unbundling on the cable and phone companies, requiring them to lease their own lines at competitive rates ((Which probably would not be competitive, but what are you going to do? Sue them and their lawyer army in court? How profitable is that? Its just cheaper to put up with their uncompetitive rates.)). This would allow other companies to sell internet service over the hosting company’s coax or fiber-optic network. These companies would be responsible for their own transport out of the network to whatever Tier 1 ISPs they purchase transport from, and all of the technical support, billing and usage information, etc.

Loop unbundling works well in Europe, but their business environments are drastically different than ours in the US (PDF), and I fear that between the relentless lobbying budgets of the cable and phone industry, the bad-faith dealings and legal shenanigans that would occur against those who want to come in and resell transport, it would not be a profitable business. In the same way we see cable and phone companies working hard to prevent municipal broadband, we would see an even more rigorous offensive campaign against loop unbundling before, during and after its implementation.

Heavy Regulation

Another option would be heavy regulation of internet service provided by wireline companies. This regulation has been happening at the national level with the FCC, but maybe that’s the wrong place for it. The best option may be with the local and state agencies like Corporation Commissions or Public Utilities Commissions. In the same way electricity rates or phone companies are regulated by a PUC or Corp Comm., so would the internet access. The Department of Energy in DC doesn’t regulate your electricity rates, so why does the FCC want to regulate broadband with the goal of making it more affordable and ubiquitous? The local solution may be much more effective because local officials are much more responsive to citizen complaints. If everyone is complaining that Netflix is running slow on Comcast, or that Verizon is intentionally letting its copper-based infrastructure degrade, the PUC may be in a better position to force Verizon or Comcast to deal with it than a gridlocked FCC or Congress.

This would put some burden on the telecommunications companies because its more government they have to deal with. In my case with my ISP (Cox), they seem very receptive and integrated with the local community ((Even though they raise my cable and internet bills by 10% a year, 3-5x the rate of inflation)), so I don’t believe that for companies like Cox it would put an undue burden on them, but for large companies stuck in that unaccountable monopoly mindset like AT&T, Verizon and Comcast, it would be more difficult.

Municipal Broadband Networks

One option that has had mixed results so far is municipal broadband networks, where the local government agency owns and runs the ISP. Unfortunately, approximately 20 states have legislation that prevents or has halted the growth of these types of networks. The FCC is looking into creating regulation that would overrule states’ abilities to pass laws against them.

The existing companies have one legitimate issue with municipal broadband networks as competitors – these private companies have invested billions and billions of dollars in infrastructure and they don’t want the government to offer a subsidized competitor with the community’s tax dollars making up for the financial losses. However, that is not an excuse for the banning or restriction of municipal broadband networks. The community’s side is that broadband is (in their opinion) more expensive than it should be – see Chattanooga’s successful $70/mo Gigabit internet service – and nationwide ISPs are unresponsive or just don’t see the return on investment needed to build out or improve service. Ultimately, the benefits of municipal broadband networks is that, as taxpayer-owned agencies, they are ultimately responsible to the people they serve through the feedback look of their board of directors being locally elected officials. Short-circuiting the dysfunction in the marketplace and in Washington DC through local elections.

Municipal broadband networks aren’t always successful though, either through internal mismanagement or being undercut by a national competitor (who can sustain losses in one region) for a long enough period of time to lead to financial ruin for the publicly funded network.

Building a True 21st Century Broadband Network

The goal would be to build a network that combines the advantages of the methods described above, while trying to do this as cheaply as possible and without building a redundant network.

Eminent domain

Eminent domain is essentially creating the municipal broadband network while removing the threat of a private broadband company attempting to undercut and destroy the effort ((This isn’t socialism or communism — as long as the assets are purchased in the public interest (to provide faster internet at cheaper prices) for fair market value)). Customers can continue to use their existing ISP under a temporary management agreement until the necessary infrastructure upgrades can be made to support Loop Unbundling.

Network Migration & Loop Unbundling

Eventually, all devices on the network (Internet, TV, Phone) will need to be IP or have a set-top-box for incompatible devices. This would be the most difficult aspect – you would need either affordable devices that can authenticate themselves to the network and then broadcast the subscribed set of channels over the customer’s home coax, or you would need even cheaper boxes for people to hook up to individual TVs and output the broadcast signal over RCA, S-Video or HDMI to the TV.

Management

The new network would be managed by a regional “Broadband District” for a metropolitan area. That BD would issue bonds to purchase the physical infrastructure from the cable company or phone company (whoever has the best infrastructure) by force at market value under eminent domain. Once the network migration is complete, it would be opened up to anyone who wanted to run their own ISP.

Eventually the goal would be to use the local fiber loop purchased by the government agency to provide everyone fiber to the home, and then allow everyone to have a competitive list of broadband providers to choose from.

Successful Business Models

The unbundled nature of the network would provide for real competition, ranging from little value-add (here is your internet service, have a nice day) to high value-add (on-location tech support, home networking, security, phone, TV, whatever else they can think of) at various prices.

Companies like Google or Facebook could offer subsidized service that, in exchange for your browsing habits, phone records, TV watching habits, dignity, and whatever shreds of privacy you have left, give you a discount on service.

For economically disadvantaged areas, instead of providing fiber to the household or multi-tenant building, offer WiFi services and prepaid broadband cards for a fixed amount of data transfer (1GB, 5GB, 10GB) sharable amongst a number of devices. This way, even those with only a basic smartphone could still use the internet for essential tasks like applying for jobs or requesting benefits.

Conclusion

First, we need to remove redundant investment. We don’t have competing road networks, power grids, or water systems ((Even in the great free market of Texas, there are multiple energy companies, but only one energy transmission company – why? Because infrastructure is expensive!)). Why do we have these expensive, redundant last-mile communication networks? Because of some historical legacy? Time to ditch them and have one broadband infrastructure for all Americans.

Next, we need to figure out where competition works – clearly its not working in its current state as prices climb and the US lags in broadband penetration. Letting companies compete over a shared infrastructure will work – it works every day as companies compete with each other while driving over our shared roads, utilizing the power grid and water and sewer networks.

Finally, costs need to be managed as we convert the proprietary networks of today to the open networks of tomorrow. We can’t have gold-plated equipment blowing up the budget. Any agency willing to do this would need to be responsible and disciplined in its conversion to eventually opening the network up to other companies to compete on.

Building an all-in-one home server?

I’ve been thinking about building an all in one server lately. The goal would be to consolidate the two servers I currently have (Win2k3 outside server, Windows Home Server inside) into one physical box using virtualization.

The difficulty is trying to make up for Windows Home Server 2011′s missing features (that were in the original WHS). So I’ve been reading and have been thinking about building a computer built on ESXi 4.1 with OpenIndiana and a SATA controller card through VT-d hosting up to 8 HDDs which would be the host storage for the WHS 2011 drive. The HDDs would be setup using ZFS RAID Z-2 (2 parity drives), at with 3TB per drive, just 4 drives would yield 6TB usable storage. It would be exposed to WHS 2011 through iSCSI in ESXi.

From there the WHS 2011 VM would use that storage to store all my media for sharing. Because WHS 2011 doesn’t have any sort of data duplication (Drive Extender was removed in 2011), just putting the data on 1 HDD wont protect it from failure. And WHS 2011 doesn’t support drives larger than 3TB (the VHD file format used to backup one drive to another for redundancy doesn’t support images larger than 2TB).

Finally, I’d also have a VM of Windows 2k3 server as the outside server, hooked to a VT-d NIC (either the 2nd one on the motherboard or a separate add-in card compatible with VT-d). This would be the web server, server I remote to from work to browse the internet, etc.

The difficulty is that all this requires a fair amount of computer hardware. Just about $2,000 for a nice configuration (from the ground up, no recycled hardware). This includes case, PSU, motherboard, CPU, RAM, SAS/SATA controller card, HDDs, and a 160GB SSD for all the operating systems (maybe that I could replace with a cheapo 500GB drive, or two in RAID-1). Considering I just upgraded my desktop this year, I’m not in a hurry to run out and spend more money on computer parts.

Initial Thoughts on the HP Touchpad

got a great deal on the new HP Touchpad, and I had wanted to get it just to get it and try it out. I think that Palm’s (now HP) WebOS operating system was probably the best phone/tablet operating system other than Apple’s iOS.

And after an hour there are already some things I like better about WebOS than iOS, particularly how multitasking works. And there are a few things I definitely don’t like – media sync and the HP Play Beta software for the Mac.

UI

The user interface is good, especially when it comes to multitasking. The WebOS card system is the best way to handle multitasking on a mobile device. Period. Even after a few hours I like it way better than iOS’s implementation. Swiping up from the screen to kill apps (or app tabs/fragments), swiping up from the bottom to bring up the task switcher, they just seem more natural than the double-tap of the home button on iOS devices. And the card interface is (sadly) the smoothest experience of the entire OS.

But even after the update design to improve performance, the touch response is still lacking. I don’t know how Apple does it, but everything is always more responsive on an iOS vs Android and HP phones and tablets. All the Android phones I’ve ever played with and the TouchPad have that same, slightly lagged response. If iOS never existed we might never notice, we might not have this issue. But coming from iOS devices I certainly can tell, and sometimes I’ll get ahead of the screen, resulting in unwanted taps. If they cant fix it (because of the way the device or OS is designed) then we’ll just have to live with it, and thats disappointing.

Apps

The dearth of Touchpad-native apps is one of the bigger issues, mostly due to it being a relatively new platform. The nice part is that since I’ve got great web development skills, I should be able to make my own applications fairly easily (unlike the 10 times I tried to make an iOS app and gave up because I couldn’t wrap my brain around Objective-C — too much C# and web dev have rotted my core I guess).

That said, even some of the apps the Touchpad has look incredibly awful. The USA Today app (also available on the iPad, and an app that I use somewhat frequently) has this weird side-scrolling news article list for each “section” of the newspaper, and then an article pop-over on the right side. I’m really not sure what design decisions were made, but the app you get on the iPad is infinitely more readable and usable. Similarly with the Facebook app – you just wonder what the designers were thinking. There are at least two or three better layouts they could have used.

There is a free Angry Birds HD app. So, there is that.

Media

For me, the media experience didn’t go so well for me. I have a Mac and currently manage everything through iTunes. My goal was to load about 10GB of music from my library and some movies (540p H.264 medium profile aka iPad 2 compatible). Loading music is awkward because you have to put the device in USB disk mode, which keeps you from doing anything else while the transfer is going. The HP Play Beta software did a half decent job of importing my iTunes library, but it didn’t do very well with Smart Playlists or the Folder/Playlist hierarchy I have. That, and I tried to add “Recently Added” songs to the Touchpad, and it tried to import my entire music library (rather than songs I had recently added to the iTunes library it imported from). Video didn’t turn out so well either, I copied Iron Man 2 to the device, but wasn’t able to play it. Googling for help didn’t work since all the links end up being for pay-for software to convert it down for the device, not much about specifications or what Handbrake settings I can use. As someone who has a large and carefully manicured digital movie and TV show library, its something of a letdown to not have it work right out of the box.

Edit: I was able to get some movies from my media library working, but many didn’t work. Considering I used the same settings on all the rips, I really don’t know what the problem is.

Follow Up – August 20

HP announced they’re discontinuing the TouchPad. Which is incredibly heartbreaking, knowing they could have done something really good with it. I took mine back and got my $400 back – luckily it was the last day of the 14 day return period. Then, as the fire sales happened around the internet this weekend, I spent $99 on a new 16GB TouchPad. So I’ll still end up with one, but for a lot less.