Probably the best place to start is with the problem: microbursts.

If packet buffers can be provided at point where packets get delayed, it may be possible to queue packets and deliver them when a burst has subsided. Without a buffer, those packets would be dropped.

The easiest to understand cause of short term packet bunching is where a switch is being used to slow a packet stream. For example data might be arriving from the WAN on a 40 Gb/s connection while being sent along to an end host at 10 Gb/s. This is short term because the burst can be no longer than the TCP window size. If packets are coming in faster than they can be fed out, some buffer can help.

Bursts can also be caused by cross traffic. If some packets in a stream are slowed down, the packets behind may catch up. Packets from a flow that started out evenly spaced will arrive in clumps because of the action of cross traffic. The more cross traffic, the more clumping.

Too much buffer can, it turns out, also cause a problem.

If two 10G connected researchers send streams off your campus at the same time, and your off-campus connection is 10G, you don't have a microburst problem. You have oversubscription. Buffers won't help.

Alan Weckel from Del'Oro Group made the point that 10 Gig-E is driven by the data center. The needs of the network are just a footnote. There is are lots of switches out there that are designed to solve someone else's problem.

  • Switches designed for the data center interconnects will have ultra-low latency. RTTs are short so buffer requirements are minimal. Flow control or priorty flow control would be useful.
  • Switches designed to cope with incast in the data center will have more generous buffers. These switches are for data center secenarios with high East-West traffic. You can easily kick the price of a switch up by 10X. But it will be a nice switch.
  • Switches designed for the LAN access layer may have multiservice features like QoS. Manufacturers will emphasize the number of different queues to sort out video, voice and data.

Writing a purchase order for large buffers makes sense where there is a down speed step. It makes sense in the core of the network where cross traffic can cause packets to bunch up. But in the campus, big expensive buffers may not have a payback. Ability to buffer 6 Mbytes is sufficient for a 10 Gb/s sender and a 1 Gb/s receiver.

There is an analogy to adding RAM to a computer. A computer that does not have sufficient RAM to build a working set of memory pages will thrash its swap files. But once you have added enough RAM to get a working set, adding more will not improve performance.