Cannery Row to UCSC microwave

Video originated at the edge of Monterey Bay on Cannery Row is captured by VBrick encoders and is sent as MPEG-2 across the Bay to UCSC. There it joins an Internet-2 connection to travel to Mystic Connecticut. The graphs below cover from approximately 3 p.m. Saturday, June 14, 2003 through 4:30 p.m. Monday.
The VBrick video decoders at Mystic register a sync loss event whenever a group of one or more packets is lost in transit. One sync loss event in a five minute sample window counts as 12 events per hour and that is the minimum sized tick above the event-free baseline.

We also measure the data flow in bits per second arriving at UCSC's main router. The video cameras are on all the time so to first approximation we would expect to see steady transmission from four cameras at 6 Mb/s each or 24 Mb/s aggregate. Notches below the expected rate are likely microwave dropouts. We draw special attention to the large spike in the sync-loss graph at 14:05 on Monday and another smaller peak at 15:05. These correspond to the dips in traffic arriving at Santa Cruz from Cannery row in the graph below. In contrast, the burst of sync losses that occurred over three hours starting at noon on Sunday appears not to correlate with microwave dropouts.

Increases above the 24 Mb/s line are caused by a motion j-peg transmission from the Lightwave computer that is demand switched.

The network glue that attaches the DS-3 rate microwave radios to the LANs in Santa Cruz and Monterey is the DNE4500 made by NetToNet. It is not directly accessible to the network management tools used to make the graphs above. It does, however, withhold network carrier when it has lost the receive signal from it's peer box at the far end of the Bay. We get an indication of radio distress when the attached port on a monitored switch shows on/off behavior in its logs.
no port events on 6/16/03 prior to those listed here 

I 06/16/03 14:06:43 ports: port 2 is now off-line
I 06/16/03 14:06:49 ports: port 2 is now on-line
I 06/16/03 14:06:53 ports: port 2 is now off-line
I 06/16/03 14:07:00 ports: port 2 is now on-line
I 06/16/03 14:07:07 ports: port 2 is now off-line
I 06/16/03 14:07:14 ports: port 2 is now on-line
I 06/16/03 14:07:17 ports: port 2 is now off-line
I 06/16/03 14:07:23 ports: port 2 is now on-line
I 06/16/03 14:08:14 ports: port 2 is now off-line
I 06/16/03 14:08:20 ports: port 2 is now on-line
I 06/16/03 15:06:06 ports: port 2 is now off-line
I 06/16/03 15:06:12 ports: port 2 is now on-line

no additional events to the end of the period of the graphs
It appears that the biggest impairments in the delivered video are due to radio dropouts. Problems occuring elsewhere along the path produce effects that are less severe. UC Santa Cruz does not have the ability to find where in the network the point losses are occurring. We doubt given the problems in the radio link that it would be worth chasing the smaller gremlins.

July 4, 2003 -- longer dropout examples

The empirical evidence of radio dropouts versus Internet2 problems is even more compelling.

One theory: If the afternoon sun causes differential expansion of the antenna mounts, it might change the aim of the antennas enought to affect the signal. While the radio link can be good for weeks at a time, it can also be pretty poor for extended periods.

Better theory: From some radio engineers with a lot of experience. They believe temperature inversions over the Bay are pushed down near the water during the summer. When our radio path penetrates through an inversion, the beam is scattered or reflected destructively into itself.

While it is likely that the small green ticks at 22:00 and 02:00 near the left red day marker were not caused by radio dropouts, all sync losses of significance correlate with the radio link.

The sync-loss graphics are snapshots from a live graph. The flow graphics are also available live.