These switches are members of a family that use Mellanox's SwitchX silicon that was introduced in a press release on April 25, 2011. This silicon permits mixing Ethernet, Infiniband and Fiber Channel ports in the same device. SwitchX-2 was introduced in October 2012. Amit Katz gave an introductory web presentation in 2012. In Q&A following the presentation, Katz was asked what, exactly, is the difference between X and X-2. What Mellanox was willing to say is that X-2 has enhanced microcode that makes better use of internal tables to permit OpenFlow 1.3 to work better.
Mellanox had a substantial presence at the Ethernet Summit in May 2014. This is not yet another Broadcom top of rack switch. Mellanox has expertise both as a silicon ASIC manufacturer and as a switch manufacturer. They point out that the speed of a single switch is only part of the speed of a system. To the extent that increased port count lets you reduce chassis count, you can get a big boost in system performance by traversing, on average, fewer switches. And in a pure Mellanox data center, inter-switch links can be kicked up to 56 Gb/s -- not a standard, but a substantial increment over 40 Gb/s.
Latency is important for the data center and would also be important for HPC clusters. It is not important in the WAN. Mellanox ASICs were developed for Infiniband and enhanced so a single switch chip part could be programmatically set up for either Infinband or Ethernet. Perhaps because of its Infiniband roots, the chip is not a shared memory design. Each port has its own buffers and there is no common pool or sharing. Long RTT networks need large packet memory that is available to buffer bursts.
Mellanox hired Tolly to do performance report #212113 on their Ethernet products in May 2012. Tolly's fair testing charter not withstanding, he tests only the parameters commisioned by his patron. While Tolly reports for other similar products have highlighted the ability to deal with microbursts, buffer capabilities are not even whispered for Mellanox. And Mellanox SX1024 and SX1036 data sheets do not contain information about packet memory or buffer depth. We are greatful to Amit Katz for proving the buffer sizes in the table via email: 128 KB for 40/56 Gb/s ports and 64 KB for SFP+ ports.
In Februrary 2015 Mellanox commissioned a another Tolly performance report 215111 to emphasize their bone fides relative to the Broadcom Trident2 switch ASICs. Mellanox claims the evidence shows that Broadcom cut-thru mode is only available when all ports are running at a common speed. When the ASIC is called upon to do a speed conversion, Broadcom shifts out of cut-thru mode. While the first Tolly report came out ahead of the SwitchX-2 announcement, the later report says it represents performance with the newer ASIC. Indeed, while the model number remains SX1036, part numbers changed between X and X-2. While the new data sheet carries the new part number, no additional information about buffer details has been provided. Mellanox claims switchX-2 is able to maintain cut-thru mode in the face of speed matching.
Mellanox switches are ripping fast. They are undoubtedly the right choice for many applications. If your application needs WAN-sized buffers, SX switches will not, however, be your choice.