To improve data distribution efficiency a load-balancing data distribution LBDD method is proposed in publish/subscribe mode.In the LBDD method subscribers are involved in distribution tasks and data transfers while r...To improve data distribution efficiency a load-balancing data distribution LBDD method is proposed in publish/subscribe mode.In the LBDD method subscribers are involved in distribution tasks and data transfers while receiving data themselves.A dissemination tree is constructed among the subscribers based on MD5 where the publisher acts as the root. The proposed method provides bucket construction target selection and path updates furthermore the property of one-way dissemination is proven.That the average out-going degree of a node is 2 is guaranteed with the proposed LBDD.The experiments on data distribution delay data distribution rate and load distribution are conducted. Experimental results show that the LBDD method aids in shaping the task load between the publisher and subscribers and outperforms the point-to-point approach.展开更多
This paper analyzes the Parallel Packet Switch(PPS) architecture and studies how to guarantee its performance. Firstly a model of Stable PPS (SPPS) is proposed. The constraints of traffic scheduling algorithms, the nu...This paper analyzes the Parallel Packet Switch(PPS) architecture and studies how to guarantee its performance. Firstly a model of Stable PPS (SPPS) is proposed. The constraints of traffic scheduling algorithms, the number of switching layers and internal speedup, for both bufferless and buffered SPPS architecture, are theoretically analyzed. Based on these results, an example of designing a scalable SPPS with 1.28T capacity is presented, and practical considerations on implementing the scheduling algorithm are discussed. Simulations are carried out to investigate the validity and delay performance of the SPPS architecture.展开更多
Fingerprint has been widely used in a variety of biometric identification systems in the past several years due to its uniqueness and immutability. With the rapid development of fingerprint identification techniques, ...Fingerprint has been widely used in a variety of biometric identification systems in the past several years due to its uniqueness and immutability. With the rapid development of fingerprint identification techniques, many fingerprint identification systems are in urgent need to deal with large-scale fingerprint storage and high concurrent recognition queries, which bring huge challenges to the system. In this circumstance, we design and implement a distributed and load-balancing fingerprint identification system named Pegasus, which includes a distributed feature extraction subsystem and a distributed feature storage subsystem. The feature extraction procedure combines the Hadoop Image Processing Interface（HIPI） library to enhance its overall processing speed; the feature storage subsystem optimizes MongoD B＇s default load balance strategy to improve the efficiency and robustness of Pegasus.Experiments and simulations are carried out, and results show that Pegasus can reduce the time cost by 70% during the feature extraction procedure. Pegasus also balances the difference of access load among front-end mongos nodes to less than 5%. Additionally, Pegasus reduces over 40% of data migration among back-end data shards to obtain a more reasonable data distribution based on the operation load（insertion, deletion, update, and query） of each shard.展开更多
基金The National Key Basic Research Program of China(973 Program)
文摘To improve data distribution efficiency a load-balancing data distribution LBDD method is proposed in publish/subscribe mode.In the LBDD method subscribers are involved in distribution tasks and data transfers while receiving data themselves.A dissemination tree is constructed among the subscribers based on MD5 where the publisher acts as the root. The proposed method provides bucket construction target selection and path updates furthermore the property of one-way dissemination is proven.That the average out-going degree of a node is 2 is guaranteed with the proposed LBDD.The experiments on data distribution delay data distribution rate and load distribution are conducted. Experimental results show that the LBDD method aids in shaping the task load between the publisher and subscribers and outperforms the point-to-point approach.
文摘This paper analyzes the Parallel Packet Switch(PPS) architecture and studies how to guarantee its performance. Firstly a model of Stable PPS (SPPS) is proposed. The constraints of traffic scheduling algorithms, the number of switching layers and internal speedup, for both bufferless and buffered SPPS architecture, are theoretically analyzed. Based on these results, an example of designing a scalable SPPS with 1.28T capacity is presented, and practical considerations on implementing the scheduling algorithm are discussed. Simulations are carried out to investigate the validity and delay performance of the SPPS architecture.
基金Project supported by the National Basic Research Program（973）of China（No.2014CB340303） the National Natural Science Foundation of China（Nos.61222205 and 61402490）+1 种基金 the Program for New Century Excellent Talents in University,China（No.141066） the Fok Ying-Tong Education Foundation
文摘Fingerprint has been widely used in a variety of biometric identification systems in the past several years due to its uniqueness and immutability. With the rapid development of fingerprint identification techniques, many fingerprint identification systems are in urgent need to deal with large-scale fingerprint storage and high concurrent recognition queries, which bring huge challenges to the system. In this circumstance, we design and implement a distributed and load-balancing fingerprint identification system named Pegasus, which includes a distributed feature extraction subsystem and a distributed feature storage subsystem. The feature extraction procedure combines the Hadoop Image Processing Interface（HIPI） library to enhance its overall processing speed; the feature storage subsystem optimizes MongoD B＇s default load balance strategy to improve the efficiency and robustness of Pegasus.Experiments and simulations are carried out, and results show that Pegasus can reduce the time cost by 70% during the feature extraction procedure. Pegasus also balances the difference of access load among front-end mongos nodes to less than 5%. Additionally, Pegasus reduces over 40% of data migration among back-end data shards to obtain a more reasonable data distribution based on the operation load（insertion, deletion, update, and query） of each shard.
