Search
 
loader loading subjects...
cover
UniPart: Optimizing Streaming Graph Partitioning Towards Universal Adaption in RDF Triple Stores
Wenhui Ya, Lena Wiese, Ahmed Al-Ghezi
311 - 326
10.3233/SSW240024
Research Article
Studies on the Semantic Web
Volume 60: Knowledge Graphs in the Age of Language Models and Neuro-Symbolic AI
Abstract

Purpose:

With increasing size of Resource Description Framework (RDF) graphs, the resulting graph structures can become too large to be managed on a single compute node, lacking the necessary resources to execute a partitioning of the graph – in particular, when the partitioning method relies on global graph information for which the entire graph has to be loaded into the main memory. This paper introduces a window-based streaming partitioning technique to obtain distributed RDF graphs, overcoming the memory limitations of traditional partitioning methods.

Methodology:

We evaluated our approach, UniPart, by comparing it with established graph partitioning algorithms such as METIS, LDG, and WStream. The comparison focused on key metrics, including the proportion of edge cuts.

Findings:

Through practical assessments using the LUBM dataset, our algorithm demonstrated strong performance in load balance, execution time, and memory usage. Notably, under the DFS streaming order, UniPart achieved a 20% reduction in edge-cut ratio compared to LDG.

Value:

UniPart operates without the need for global graph information, making it exceptionally suited for dynamic environments with unbounded streams and unpredictable data sizes.

Download
Creative Commons License