This morning many people in Silicon Valley woke up without 911 service, Internet, cellular phones, and in some cases TV. Web sites were impacted and Internet traffic between a few major datacenters stopped flowing.  Several of our employees were cut off from the Internet and phone service.

AT&T put out a press release stating that there was a fiber cut, but to make this happen, there had to be several cuts. According to several employees that work at AT&T, it may have been done by the very people that repair this stuff, the Communication Workers of America Union (CWA).  

This of course is speculation on behalf of these individuals.  The cuts could have also been framed to make it look like it was done by a competent group, someone that knows where the major fibers are sitting inside specific manholes. However, the CWA contract to do work for AT&T expired last Saturday night. According to various press releases from CWA, “five of CWA’s six agreements with various AT&T companies expire at midnight, April 4, 2009″. The cuts were clean, done apparently by a hacksaw. The first major cut went down around 2 AM in the South East Bay which isolated the city of Santa Cruz. Another cut around 4 AM took out the major Metromedia Fiber Network (MFN) in the San Francisco bay area as well.

Regardless of who did it - someone did it.

These areas are within driving distance of each other.

What’s more terrifying, the cuts were clean and easy to fix, but what would happen if they were mangled and more calculated? What happened if rather than going down into the manhole, the perpetrator poured many gallons of gasoline down the hole, and tossed a match on it? It could have melted all the fiber/glass/plastics together, causing complete mess making the problem much worse.

Fiber maps such as the one below show details on exactly where fiber runs are and how to locate perfect targets, found directly off of almost every network provider’s web site:

Now, what would happen if someone were to coordinate with a group of people and demolish key areas where fiber concentrations are very thick in major cities around the world.  What if it were done all at the same time? 

If that were to have happened, more than just Silicon Valley would have been impacted: Globally people would be waking up to find out their land lines don’t work, 911 calls fail, their email is down, and yes no Twitter.

Sound improbable?  Well, three undersea cables: FLAG, SMW-3, and SMW-4 where all cut at the same time on January 5th 2009, which crippled communications to several countries.  Traffic had to be re-routed and it really was a mess, but nobody was left in the dark.  

Realistically it would be a few days of an outage, but it would not be great given society is depending more and more on the Internet for day-to-day things such as the DMV, voting, work, and daily life.  Our society is becoming dependent on something very fragile and somewhat unprotected.

Greetings.  My name is Ed Crump, and I’m the chief scientist behind the BitGravity LiveBroadcast system.  I’d like to take a minute to share some thoughts, inside information, and tips about the system we’ve developed for you.  First, some background …

Real-time, Live, to the Desktop!

Over 10 years ago at Wayfarer, I was developing a real-time messaging system (ala Push Technology) with the dream of eventually bringing real-time, high quality, Live Internet broadcasts to millions of desktops around the world.  Flash had just come on the scene (and was quickly snatched up by Macromedia to replace Shockwave).  A/V capture cards were a curiosity.  Dial-up users were the norm.  Pentium 100MHz systems were hawt.  Yahoo had a whole building of servers pushing a whopping 100Mbps.  Sadly, none of the pieces I needed were really ready.  The dream would have to wait.

I was able to realize the dream at DEMO 2008 when we announced affordable Standard Definition Live Broadcast for the masses.  And today, we’ve taken it to the next level at DEMO 2009 with our announcement of affordable HD Live Broadcast for the masses!

Here’s some insight to what happens behinds the scenes to deliver BGLive …

Pushing HD 1080p Live broadcasts to the masses

BitGravity LiveBroadcast is an end-to-end, global Live broadcasting solution that brings affordable HD 1080p broadcasting to the masses.  BGLive is built on top of BitGravity’s global next generation backbone which provides BGLive its scalability, efficiency, and cost-effective delivery.

Broadcasting HD Live is a challenge …

• capturing an HD feed in real time requires special hardware
• compressing (transcoding) an HD feed, in real time, so it can be broadcast over the Internet takes lots of number crunching
• delivering HD feeds to hundreds of thousands of users requires a global, next generation network
• watching heavily compressed HD broadcasts requires high end desktop machines or hardware decoding

NOTE:  Because of the volume of data that makes up an HD video stream capturing and transcoding are usually combined into a single step.

The main challenge with capturing HD video is matching the inputs:  connectors (Component, HDMI, HD-SDI, etc.), video frame rates, colorspaces, etc.  The video broadcast and production industries are fond of standards (which is why there are so many).  Fortunately affordable HD capture cards are becoming mainstream (e.g. the BlackMagic HD DeckLink Extreme), and they can insulate you from a lot of this pain.

Compressing (transcoding) an HD feed in real time is a much thornier problem.  The issue is you are trying to push ~12Gbps (1920 x 1080 x 24 x 30 * 8) bits into a 2Mbps stream that the end user can actually download and watch in real time.  There are hardware solutions for transcoding HD in real-time, but these all have issues, the predominant ones being that they are very expensive.  This can be a major roadblock.  We recognized this early on and decided to invest in building an affordable software encoding solution that runs on off-the-shelf hardware (Mac).

Delivering HD feeds to large audiences of Internet viewers is the real showstopper for most.  There is virtually no way to scale this by yourself without inventing your own Internet backbone first (i.e. to make a small fortune, you must first spend a large one).  Basically, you need to use a CDN or a compute cloud like Amazon EC2.  Either way, this is a distraction from your core business/service.

BGLive pulls all of this together and gives you an affordable, end-to-end solution for HD broadcasts.

1080p just doesn’t look good in anything less than 1.5Mbps

Most CDN’s that offer Live broadcast services have problems reliably delivering streams over 1.5Mbps.  The most notable reason is that their infrastructure was designed for the Internet that used to be, not the Internet of today.  The higher the bitrate, the fewer the concurrent streams that can be handled by individual servers and routers.

It’s very popular at the moment to use Amazon EC2 machines with Flash Media servers to try to scale Live broadcasting to the masses.  Unfortunately, Amazon throttles bandwidth to their EC2 machines down to 160Mbps (to prevent harm to their network) which in turn forces you to rent more machines to serve more streams.

BitGravity on the other hand developed a next generation network to support its CDN, and BGLive got to take advantage of it.  BitGravity’s network allows my Live servers to sport 10Gbps interfaces and deliver 6000 HD 1080p streams @ 1.5Mbps each whereas a 160Mbps connected EC2 machine maximum of 100.

Bottom line, our costs for delivering HD 1080p throughout the world are lower which directly translates to lower costs for our customers.

Don’t compromise quality

Live Internet broadcasting services traditionally trade off video quality (low bitrates and video frames) to reduce costs and reach a wider audience.  It makes for a poor viewing experience though.  This used to be an important feature when everyone was on dial-up modems, but the Internet has changed.

BitGravity provides for lower bitrate streams that are encoded at the same time as the high bitrate streams, and dynamically switches between them in our Flash player.  The net difference to the end user is the lower bitrate streams from BitGravity are much higher quality.  This is because BitGravity streams have the full frame rate and were generated using the original feed (not the transcoded copy).

Let’s face it, you don’t want to be dropping frames or pixelating your HD broadcast.  1080p deserves better!

Real-time isn’t exactly

We use the phrase real-time encoding, but this is misleading.  Real-time encoding means that the video stream can be encoded in real time:  it takes 1 second to transcode 1 second of video.  The piece everyone leaves out is how long the system takes to get to this state (like a CPU pipeline).  Some systems take minutes to achieve this state.  This means you are watching a live broadcast from minutes ago.  This is referred to as the delay from live.

Obviously the viewer’s sensitivity to the delay from live is a function of the stream content.  In particular, when you combine real-time chat with Live video where the folks in the video can react to the chat, it makes a big difference.

BitGravity delivers HD video to viewers around the globe within 4-6 seconds after it hits the camera, and our goal is to get that down to 2 seconds.  Obviously we are good net citizens and will obey all laws related to the speed of light!

(obscure reference:  “More Live Than Live!  That’s our motto!”)

As the lead developer of BitGravity’s Flash player I am very proud to introduce what we have been working on for the past eight months!

We have launched our newest Flash Player which has a lot of great features and a very easy wizard that helps anyone configure and embed video. We decided to build this player from the ground up, instead of modifying our Player 5. We did this so that we could create a really solid, modular, extensible foundation. This is going to open the door for a lot of customer suggestions to be implemented in a really quick way. So we’re going to be adding playlists in the near future, and we’re geared up to support third party advertising networks.

Some exciting new features that we’ve already added for the 6.0 release are pre-rolls, post-rolls, theming, RTMP streaming, some new multi-bitrate options and improved support for H.264 advanced seeking. We’ve also done a complete overhaul on the configuration wizard, so you can access all these new features very easily.

Best of all we’re giving it out for free! There are a lot of great players out there, and we often get asked by our customers for Flash player assistance, so we figured it might be nice to provide a general player, that we will know inside out, for anyone to use.

Here is a quick general overview of a few of the features to be found in BitGravity Player 6:

Player Configuration Wizard
The new and improved BitGravity Player Configuration Wizard has been extended to include the player’s new features, and provide quick and intuitive setup and skinning controls. A “Simple Mode” has been included to get your video playing in seconds, and a preview of your partial configuration is visible throughout the setup.

Theming
There are four elements available to customize the look of the new BitGravity Player to fit right in to your website.

• Custom logo
• Logo position
• Color scheme (made up of four colors, and each is adjustable)
• Video background

Multi-bitrate
While high definition video is very desirable, reaching a wider audience often requires a lower bitrate video. With Multi-bitrate, you no longer need to have just one or the other. With the BitGravity Player, offering your video in 2 to 5 different bitrates will allow your viewers to switch bitrates to match their connection speed from within the player. For video hosted on BitGravity CDN, users can switch bitrates mid-stream! There is even the option to have the quality selection automated based on the user’s connection speed.

Improved support for H.264
On the server side we’ve been putting a lot of resources into supporting as many types of video encoding as we can with our AdvancedProgressive seeking - so you can seek to a point in the video ahead of the loading bar. A much wider range of H.264 and FLV encoded videos are now supported for full BitGravity AdvancedProgressive seeking and mid-stream bitrate adjustments on BitGravity CDN.

Advertising
Monetize your videos by playing pre-roll advertisements before your video or less-intrusive post-rolls.

Speed and Stability
A modularization of video playback modes allows for fast player initialization and is kind to your users’ CPUs, getting your video started even faster.

Get Set Up with BitGravity Player 6!

When Perry Wu and I started BitGravity we had high hopes and dreams to build technology that may someday change the Internet.  The thought of someday creating technology that pushes the limits of the Internet gave me the drive to build our first server farms in my garage, design a new traffic distribution system, and eventually take our baby and scale it across the entire planet.

As the company grew from my garage and a Starbucks “office”, our views began to culminate.  I began to view software as art and not merely a means to an end. I also began to view our team of technologists as wizards that could create and accomplish anything. To this day, if I were to ask one of our principle engineers Edward Crump if something were possible he would undoubtedly respond with his token mantra:

The view that Internet service engineering is an art with endless possibilities has become a philosophical pillar here at BitGravity.  We view the designs on the network, servers, drivers, kernels, and everything up the stack should be done as an art, not strictly a science that results in a bottom line.

It is our view that new technology is only possible because passionate people dream of new concepts and harness their passion to drive it to completion.  It’s those same people who are not afraid to answer a question that appears to be impossible to solve.

As we continue to develop our network something unexpected is happening:  Our customers’ creativity is exploding and we are in awe of their accomplishments.  Every day we see our customers use our technology to do something new, unique, and innovative, which further motivates us to work harder and stretch the limits of the Internet.

As the Chief Technologist here at BitGravity, I am pleased to introduce our Sandbox.  We will be releasing free technology, views and opinions of our developers, and generally using this area as a playful outlet for technology and philanthropy that should give back to the general community that has helped us become successful.