Why Internet Outages Occur

We wrote this article to help customers better understand why AireBeam or any other ISP can be affected by service interruptions occurring beyond our network.  The particular cases in point, the recent Thanksgiving outage and the outage on 11/28/12 at 6:50pm.  From your living room, while working on your laptop, you can instantaneously send and receive data to servers located half way around the globe.  To send that data from your laptop, you purchase time and bandwidth on your ISP’s (AireBeam) network.  AireBeam’s network, in turn connects to Regional Networks that connect Phoenix to LA, San Jose, Seattle, Denver, Chicacgo and Dallas. The Regional Networks are themselves interconnected at certain critical locations in the USA called InterNaps, which are simply six or eight hub locations around the USA where all traffic converges and is switched to other networks.

Now, suppose that a 10gigabit fiber circuit goes down in Phoenix.  That’s what happened on Thanksgiving.  All of the networks fed by that 10gig circuit went down. Those networks included Century Link DSL, AireBeam and about 6 other ISPs.  The reason why these outages occur are basically three:

1) Electronic failure in network equipment
2) Software failure in network equipment
3) Fiber cuts

The outage on 11/28/12 was caused when a regional router crashed in Denver and the routes to and from AireBeam’s network for traffic going through Denver were disrupted. We don’t yet know why that router crashed but, working with our colleagues at upstream providers in Denver and Wyoming, we were able to bypass the failing regional router, thus restoring normal traffic flow, even while that Regional router was “belly up”. These outages occur infrequently on a per carrier basis because we all spend a ton of money and time replacing equipment before it fails, but since the Internet is a network that depends on thousands of carriers and their interconnections, failures anywhere along the line are felt by you, the consumer. Normally, these network wide outages don’t occur frequently.  The last time we had one was back in 2006 when a backhoe cut fiber feeding Pinal County.  Everything was down from cell phones to 911 service.  It is odd that we had two a week apart.  The second outage which occurred on 11/28/12, was attributable to a regional router in Denver that rebooted several times, making the Internet as we feel it seem to go up and down and slow then fast.  Network engineers eventually bypassed the failing router until technicians can repair or replace it. The last reality to contemplate is that the Internet has no “time off” so all maintenance, planned or otherwise, has to take place when consumers are using the network. Even at 3 in the morning, the Internet users who work swing or graveyard shifts are using the Internet with a vengeance!  So, there is no “good time” when AireBeam or Internet carriers can make changes without affecting users. Let’s face it! Internet is now a basic part of our lives.  When it goes down, we feel cut off, unproductive, isolated, frustrated and angry that we “cannot get on the net!”  Trust me, we understand and we feel your pain.  Our team works very hard to proactively replace and upgrade our network components BEFORE they fail…But, the Internet does not start and end in Casa Grande…It reaches around the globe and failures anywhere around the world will, alas, interrupt your traffic flow from time to time.

Questions / Comments: Send an email to the Boss.  Click HERE!