基于分布式文件系統(tǒng)電力大數(shù)據(jù)存儲(chǔ)實(shí)現(xiàn)

打開(kāi)文本圖片集

摘要：基于Hadoop存儲(chǔ)架構(gòu)設(shè)計(jì)了電網(wǎng)非結(jié)構(gòu)化數(shù)據(jù)管理平臺(tái)，主要包括存儲(chǔ)分析與搜索讀取2大模塊，整合HDFS、Hbase等存儲(chǔ)設(shè)備，利用HDFS實(shí)現(xiàn)海量數(shù)據(jù)的快速讀寫(xiě)，采用基于ZooKeeper及Solr搭建的開(kāi)源分布式搜索引擎SolrCloud實(shí)現(xiàn)數(shù)據(jù)檢索，提供了高效便捷的智能化管理手段。

關(guān)鍵詞：電網(wǎng)管理;Hadoop存儲(chǔ);分布式;數(shù)據(jù)檢索

中圖分類(lèi)號(hào)：TP311.13

文獻(xiàn)標(biāo)識(shí)碼：B文章編號(hào)：1001-5922（2022）06-0172-04

Realization of power big data storage based on distributed file system

CHEN Xingbin WANG Zhou ZHENG Piaopiao LIN Dewei LIU Qing

（1. State Grid Fujian Electric Power Co.， Ltd.， Fuzhou 350000， China; 2. Information and Telecommunication Branch， State Grid Fujian Electric Power Co.， Ltd.， Fuzhou 350000， China; 3. State grid Xintong Yili Technology Co.， Ltd.， Fuzhou 350000， China

）

Abstract：This paper designs an unstructured data management platform for power grids based on the Hadoop storage architecture. It mainly includes two modules： storage analysis and search and reading. It integrates storage devices such as HDFS and Hbase. It uses HDFS to achieve rapid reading and writing of massive data and adopts an open-source distributed search engine SolrCloud built by ZooKeeper and Solr to implement data retrieval， providing an effective and convenient method for smart management.

Key words：power grid management; Hadoop storage; distributed; data retrieval

電力行業(yè)的非結(jié)構(gòu)化數(shù)據(jù)存儲(chǔ)的內(nèi)容包括圖像、視頻、報(bào)表、網(wǎng)頁(yè)等不同格式，其中70%以上源自人與人之間的協(xié)作，可以說(shuō)是以人為中心產(chǎn)生的數(shù)據(jù)。（剩余4218字）