大数据关键技术

[摘要] 结合大数据系统的一般结构，介绍和对比了当前大数据领域在文件存储、数据处理和数据库领域的关键技术。通过各种技术的对比，得到了一些分析结果。分析结果表明大数据系统的解决方案必将落地于现有的云计算平台；云计算平台的分布式文件系统、分布式运算模式和分布式数据库管理技术是解决大数据问题的基础；一些大的依靠数据盈利的大公司必然会是大数据应用的主体。

[关键词] 大数据；分布式文件系统；分布式数据库；MapReduce技术

[Abstract] In this paper, we discuss the general structure of a big-data system as well as key technologies in big-data storage, processing, and database. We compare these technologies in order find problems in the big-data system and propose solutions that will be used in the cloud computing platform. We propose distributed file system, computing model, and database management to solve problems associated with big data. Big companies that profit from big data will be the main users of big-data applications.

[Keywords] big data; distributed file system; distributed database; MapReduce