Scalable reduction of large datasets to interesting subsets

Date
2010-11
Authors
Williams, G. T.
Weaver, J.
Atre, M.
Hendler, James
Journal Title
Journal ISSN
ISSN
1570-8268
Volume Title
Publisher
Elsevier
Peer reviewed
Yes
Abstract
With a huge amount of RDF data available on the web, the ability to find and access relevant information is crucial. Traditional approaches to storing, querying, and reasoning fall short when faced with web-scale data. We present a system that combines the computational power of large clusters for enabling large-scale reasoning and data access with an efficient data structure for storing and querying the accessed data on a traditional personal computer or other resource-constrained device. We present results of using this system to load the 2009 Billion Triples Challenge dataset, materialize RDFS inferences, extract an “interesting” subset of the data using a large cluster, and further analyze the extracted data using a personal computer, all in the order of tens of minutes.
Description
Keywords
Billion Triples Challenge, scalability, Parallel, inferencing, query, Triplestore
Citation
Williams, G.T., Weaver, J., Atre, M. and Hendler, J.A. (2010) Scalable reduction of large datasets to interesting subsets. Web Semantics: Science, Services and Agents on the World Wide Web, 8 (4), pp. 365-373
Research Institute