Computing in Transit to Identify Rare Events in Streaming Scientific Data

Programmable networks, aside from carrying out their core network functions, can look deep into the data stream and perform application layer processing. But, expect for a few demonstrations, this capability remains largely under explored and under utilized. Currently, scientific computing leverages networks only for communication and not for computation. We propose Computing in Transit to unleash the potential of network computing for scientific workflows. Specifically, we investigate computing in transit in the context of light source experiments. Researchers using light sources are interested in rare events and we intend to leverage computing in transit to solve this problem. As the compute and memory resources available within the network are scarce, we must use these resources prudently without sacrificing on performance metrics. Computing within the network can support significantly higher throughput at low latency but it may be less accurate as there are limitations to how deep a network can inspect the payload. We propose a neutralized checksum that takes in TCP checksum as an input to avoid processing the entire payload. We evaluate this approach to identify rare events by introducing random perturbations to reference frames. We measure the effectiveness of neutralized checksum to identify changes. We see that neutralized checksum identifies all changes and is a very promising approach to rare event detection.