loading subjects...
Purpose:
With increasing size of Resource Description Framework (RDF) graphs, the resulting graph structures can become too large to be managed on a single compute node, lacking the necessary resources to execute a partitioning of the graph – in particular, when the partitioning method relies on global graph information for which the entire graph has to be loaded into the main memory. This paper introduces a window-based streaming partitioning technique to obtain distributed RDF graphs, overcoming the memory limitations of traditional partitioning methods.
Methodology:
We evaluated our approach, UniPart, by comparing it with established graph partitioning algorithms such as METIS, LDG, and WStream. The comparison focused on key metrics, including the proportion of edge cuts.
Findings:
Through practical assessments using the LUBM dataset, our algorithm demonstrated strong performance in load balance, execution time, and memory usage. Notably, under the DFS streaming order, UniPart achieved a 20% reduction in edge-cut ratio compared to LDG.
Value:
UniPart operates without the need for global graph information, making it exceptionally suited for dynamic environments with unbounded streams and unpredictable data sizes.
