At the HPC User Forum in Seattle this week, PayPal along with HP and Texas Instruments talked about making order out of Chaos using the HP Moonshot systems ARM Cortex-A15 based TI cartridge.
PayPal Engineers talked about how they need to make order out of chaotic data streams that include 3 million events per second, 25TeraBits of data ingestion per hour and 20MB/second of machine data from thousands of servers. They need to process all this disparate data real time and now with creative processing they can correlate events in seconds vs hours.
PayPal talked about watching the HPC space because their use case falls between the grey area of Enterprise servers and HPC and they were not able to scale and meet their processing needs using existing methods and approaches. It was exciting to hear PayPal engineer Ryan Quick talk about his Aha moment when he saw a presentation from HP about the Moonshot cartridge from TI and the realization that with the powerful combination of Cortex A-15 processors, 8 TI DSPs, internal fabric and networking capabilities, TI had effectively built an HPC cluster on an SoC. The proof point was when he measured the power running his application and found that running his application he was getting 11GigaFlops/Watt processing throughput.
You can see his presentation at the HPC forum here - HPC at Paypal: Leveraging DSPs for Systems Intelligence - insideHPC. He talks in detail about the SoC capabilities he uses on the TI Cartridge in this blog from our partner TI here - Creating order out of chaos – in real time - Multicore Mix - Blogs - TI E2E Community.
As I was watching the video the statement about 11GigaFlops/Watt peaked my interest and got me thinking about how would this compare to the world's Top 500 super computers. So I checked out the data on the Green500 website (The Green500 List - June 2014 | The Green500) from June of this year. To be fair your mileage will vary based on the HPC workload that is being executed and some configurations are more suited to a broader set of HPC workloads than the example discussed in this blog. But… I looked for a supercomputer that was in the Top10 on both the Green list and the Top500 list and the best number that I got was 3.2GigaFlops/Watt. To me that is certainly something that engineers will sit up and take notice of.
The PayPal presentation is a great inside look at how Ryan Quick developed an innovative solution for real-time data processing using HP Moonshot. Thanks for this summary and the research on the supercomputer GigaFlops/Watt data points. It is good to know that HP Moonshot is pushing new boundaries in Green Data Center technology!