Mellanox switches

These switches are members of a family that use Mellanox's SwitchX silicon that was introduced in a press release on April 25, 2011. This silicon permits mixing Ethernet, Infiniband and Fiber Channel ports in the same device.

Mellanox had a substantial presence at the Ethernet Summit in May 2014. This is not yet another Broadcom top of rack switch. Mellanox has expertise both as a silicon ASIC manufacturer and as a switch manufacturer. They point out that the speed of a single switch is only part of the speed of a system. To the extent that increased port count lets you reduce chassis count, you can get a big boost in system performance by traversing, on average, fewer switches. And in a pure Mellanox data center, inter-switch links can be kicked up to 56 Gb/s -- not a standard, but a substantial increment over 40 Gb/s.

Latency is important for the data center and would also be important for HPC clusters. It is not important in the WAN. Mellanox ASICs were developed for Infiniband and enhanced so a single switch chip part could be programmatically set up for either Infinband or Ethernet. Perhaps because of its Infiniband roots, the chip is not a shared memory design. Each port has its own buffers and there is no common pool or sharing. Long RTT networks need large packet memory that is available to buffer bursts.

Mellanox hired Tolly to do performance report #212113 on their Ethernet products about two years ago. Tolly's fair testing charter not withstanding, he tests only the parameters commisioned by his patron. While Tolly reports for other similar products have highlighted the ability to deal with microbursts, buffer capabilities are not even whispered for Mellanox. And Mellanox data sheets do not contain information about packet memory or buffer depth. We are greatful to Amit Katz for proving the buffer sizes in the table via email: 128 KB for 40/56 Gb/s ports and 64 KB for SFP+ ports.

Mellanox switches are ripping fast. They are undoubtedly the right choice for many applications. If your application needs WAN-sized buffers, they will not, however, be your choice.