Improving data processing time with access sequence prediction
Conference proceedings article
Authors/Editors
Strategic Research Themes
No matching items found.
Publication Details
Author list: Boonserm P., Wang B., See S., Achalakul T.
Publication year: 2012
Start page: 770
End page: 775
Number of pages: 6
ISBN: 9780769549033
ISSN: 1521-9097
eISSN: 1521-9097
Languages: English-Great Britain (EN-GB)
View in Web of Science | View on publisher site | View citing articles in Web of Science
Abstract
Genomic research nowadays often faces the problem of big data. The data size from genome sequencing process can grow very quickly and continuously creating the problem with storage and processing. BGI, one of the renowned genomic research institutes in China also faces the similar problem. The research at BGI depends on several sequencing machines. One machine pipeline may generate temporary data of around 1.4 terabytes. In addition, multiple read and write operations occur continuously during processing time. The I/O bottleneck thus degrades research throughput tremendously. Using a high performance computing system alone is not sufficiently effective in experimental results processing. In order to hide the I/O latency, an effective big data management framework is needed at BGI. In this paper, we proposed the hybrid prediction model for data access pattern. The goal is to predict the next pieces of data needed in the processor and preload them into the memory in order to improve the overall processing time. From the results obtained from the initial experiments, the proposed model can deliver high prediction accuracy in linear-time. Moreover, the error rate is low at 1.85%, which is better than the common methods used, such as Prediction Graph, ANN and ARMA. We believe that with some further fine-tuning, the model can be used as a part of the big data management framework deployed at BGI in the near future. ฉ 2012 IEEE.
Keywords
Big Data, Hybrid ARMA model, I/O bottleneck, Paired t-Test
Contact Information