Crossbow on Big F#@!ing Webtone Switch

July 29, 2013 at 10:30 am 2 comments

Back in the days of SUN Microsystem, Scott McNealy asked us to build a big F#@!ing Webtone Switch. At that time, the underlying pieces weren’t there but over last few years the possibilities have opened up. We now have the switch chips from Broadcom and Intel that switch at 1.2Tbps in H/W. From a OS view, 1.2 Tbps of switching at 300ns latency is great but the more amazing thing is PCIe as a control plane which allows 20-40Gbps of control plane B/W where you can change switch registers, L2/L3-tables, TCAMs, etc at nano-second rates.

So after more than three years of work and million lines of C code, the Pluribus Network’s engineering team has the switch chip under Crossbow control. For people who are not sure what I am talking about, in 2005 project Crossbow invented virtual switching inside a server hypervisor and introduced hardware based Virtual NICs and dynamic polling to get 40Gbps of bandwidth through a server OS. The details were published in “Crossbow: From Hardware Virtualized NICs to Virtualized Networks” in ACM Sigcomm VISA 09.

In the goal to benefit from merchent silicon ecosystem and orchestrate the entire infrastructure using Open source OS on switches, the industry has been going on suboptimal paths. The most notable efforts around a centralized controller can barely deal with the scale of single switch and typically requires sending a packet to a controller running on a separate server. The latency of these transactions (typically in milliseconds to seconds) defeats the required reaction time in microseconds in virtualized environments where Network resources are shared. The other approach of just throwing the Intel or Broadcom SDK on a whitebox switch with Linux and Quagga doesn’t really solve the control plane problem. The Broadcom and Intel SDK are crafted for their specific switch chips and meant for configuration ease and not for high speed control plane software.

By bringing the Crossbow Architecture on the switch chip where it is part of the Network OS directly controlling the switch chip via the PCIe allowsus to get following benefit:

  • Integrated Switch Hardware with fully programmable Control Plane allowing the performance and scale necessary to deal with 10Gbps switches (the distributed control plane is part of the Network OS running on the switch itself).
  • Enable applications like DDoS, IDS, Firewall, Load Balancer, routing, messaging, etc that need to be in network to run on the switch itself and benefit from the H/W offload, high speed snooping, and flow capability that switch chip offers via C, Java, Perl, Python, etc programming interfaces in UNIX/Linux environment. Development, Deployment and Resource provisioning of these applications on Crossbow enabled switches is same as current server mechanisms and uses the existing tool chain (gcc/gdb, kvm, etc).
  • Bring the benefit of merchent silicon ecosystem on switches under Openstack control enabling faster pace of innovation and cost advantages.

As we get ready to roll Netvisor (and its open source version – openNetvisor) out, I will discuss more details on this blog in near future.

Entry filed under: Uncategorized. Tags: , , , , , , , .

Netvisor and iTOR Unvieled The Battle for the Top of the Rack

2 Comments Add your own

  • 1. Sriram Narayanan  |  July 29, 2013 at 10:41 am

    Nice to know ! I’m eagerly looking forward to announcements !

  • 2. Ravi  |  August 13, 2013 at 11:39 pm

    In general the approach is to merge the control plane and data plane in a single box?
    Agreed there is a significant benefit saving the additional hop before the control policies can be applied, but how does a general purpose OS fare against the RTOS which most switches run on?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

Top Rated

Recent Posts

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 178 other subscribers

%d bloggers like this: