ErlangInside

7

okt

Phoebus: A Distributed System for Processing Very Large Graphs

On Phoebus, an Erlang version of Google’s Pregel

Phoebus is a system written in Erlang for distributed processing of very large graphs that span billions of vertices and edges. Phoebus is an implementation of the Pregel paper published by Google Research. Phoebus supports a distributed model of computation similar to MapReduce, but more tuned to Graph processing.

From the Google Research Blog post by Grzegorz Malewicz:

Take, for example, geographic locations. A relatively simple analysis of a standard map (a graph!) can provide the shortest route between two cities. But progressively more sophisticated analysis could be applied to richer information such as speed limits, expected traffic jams, roadworks and even weather conditions. In addition to the shortest route, measured as sheer distance, you could learn about the most scenic route, or the most fuel-efficient one, or the one which has the most rest areas. All these options, and more, can all be extracted from the graph and made useful  provided you have the right tools and inputs.

Essentially, Phoebus makes calculating data for each vertice and edge in parallel possible on a cluster of nodes. Makes me wish I had a massively large graph to test it with.

Leave a comment