摘要
针对智能电网大数据流的实时性、易失性、无序性等特点,提出智能电网大数据的实时流处理框架,实现数据收集、数据缓冲与流式计算,满足状态监测异常检测与用电数据分析等快速处理需要。通过采集系统节点监听数据源变化并实时收集数据,利用消息订阅模式对数据进行缓冲,解决数据采集与流式计算速度不一致的问题。提出一种基于Storm的状态监测数据流滑动窗口处理方法,在规定时间内分批处理状态监测数据流,保证数据的连续计算,通过阈值判断进行异常检测。实验结果表明,在集群规模一定的条件下,适当地改变工作进程数以及执行器线程的并发数设置,可以增大滑动窗口的元件吞吐量,提高状态监测异常检测的实时处理效率。
In view of the characteristics of smart grid,such as instantaneity,volatility,disorder,etc.,a real-time stream processing framework of big data in the smart grid is proposed.The framework can realize data collection,data buffer and flow calculation,so as to meet the fast processing need,such as condition monitoring anomaly detection and analysis of electric behaviors.In order to solve the problem of inconsistency in the speed of data acquisition and flow calculation,the data source change is monitored,while the data are collected in real-time and buffered by using the message subscription mode.A Stormbased sliding window processing approach to monitoring the data stream is proposed to process the condition monitoring data stream in batches within the specified time,while ensuring continuous data computation and detecting abnormality by threshold determination.Experimental results show that,under the condition of a definite cluster size,appropriately changing the number of working processes and the number of concurrent execution threads can increase the tuple throughput of sliding window and improve the real-time processing efficiency of condition monitoring anomaly detection.
出处
《电力系统自动化》
EI
CSCD
北大核心
2016年第14期122-128,共7页
Automation of Electric Power Systems
基金
国家自然科学基金资助项目(61074078)~~
关键词
智能电网
大数据
实时流
状态监测
异常检测
滑动窗口
smart grid
big data
real-time stream
condition monitoring
anomaly detection
sliding window
