Deep reinforcement learning(DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management.However, due to the model's inherent uncertainty...Deep reinforcement learning(DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management.However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata,which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications.First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units(PDMUs), and a reverse breadth-first search(BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.展开更多
The concept of sharing of personal health data over cloud storage in a healthcare-cyber physical system has become popular in recent times as it improves access quality.The privacy of health data can only be preserved...The concept of sharing of personal health data over cloud storage in a healthcare-cyber physical system has become popular in recent times as it improves access quality.The privacy of health data can only be preserved by keeping it in an encrypted form,but it affects usability and flexibility in terms of effective search.Attribute-based searchable encryption(ABSE)has proven its worth by providing fine-grained searching capabilities in the shared cloud storage.However,it is not practical to apply this scheme to the devices with limited resources and storage capacity because a typical ABSE involves serious computations.In a healthcare cloud-based cyber-physical system(CCPS),the data is often collected by resource-constraint devices;therefore,here also,we cannot directly apply ABSE schemes.In the proposed work,the inherent computational cost of the ABSE scheme is managed by executing the computationally intensive tasks of a typical ABSE scheme on the blockchain network.Thus,it makes the proposed scheme suitable for online storage and retrieval of personal health data in a typical CCPS.With the assistance of blockchain technology,the proposed scheme offers two main benefits.First,it is free from a trusted authority,which makes it genuinely decentralized and free from a single point of failure.Second,it is computationally efficient because the computational load is now distributed among the consensus nodes in the blockchain network.Specifically,the task of initializing the system,which is considered the most computationally intensive,and the task of partial search token generation,which is considered as the most frequent operation,is now the responsibility of the consensus nodes.This eliminates the need of the trusted authority and reduces the burden of data users,respectively.Further,in comparison to existing decentralized fine-grained searchable encryption schemes,the proposed scheme has achieved a significant reduction in storage and computational cost for the secret key associated with users.It has been verified both theoretically and practically in the performance analysis section.展开更多
基金supported by the Shanghai Science and Technology Committee (22511105500)the National Nature Science Foundation of China (62172299, 62032019)+2 种基金the Space Optoelectronic Measurement and Perception LaboratoryBeijing Institute of Control Engineering(LabSOMP-2023-03)the Central Universities of China (2023-4-YB-05)。
文摘Deep reinforcement learning(DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management.However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata,which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications.First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units(PDMUs), and a reverse breadth-first search(BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.
文摘The concept of sharing of personal health data over cloud storage in a healthcare-cyber physical system has become popular in recent times as it improves access quality.The privacy of health data can only be preserved by keeping it in an encrypted form,but it affects usability and flexibility in terms of effective search.Attribute-based searchable encryption(ABSE)has proven its worth by providing fine-grained searching capabilities in the shared cloud storage.However,it is not practical to apply this scheme to the devices with limited resources and storage capacity because a typical ABSE involves serious computations.In a healthcare cloud-based cyber-physical system(CCPS),the data is often collected by resource-constraint devices;therefore,here also,we cannot directly apply ABSE schemes.In the proposed work,the inherent computational cost of the ABSE scheme is managed by executing the computationally intensive tasks of a typical ABSE scheme on the blockchain network.Thus,it makes the proposed scheme suitable for online storage and retrieval of personal health data in a typical CCPS.With the assistance of blockchain technology,the proposed scheme offers two main benefits.First,it is free from a trusted authority,which makes it genuinely decentralized and free from a single point of failure.Second,it is computationally efficient because the computational load is now distributed among the consensus nodes in the blockchain network.Specifically,the task of initializing the system,which is considered the most computationally intensive,and the task of partial search token generation,which is considered as the most frequent operation,is now the responsibility of the consensus nodes.This eliminates the need of the trusted authority and reduces the burden of data users,respectively.Further,in comparison to existing decentralized fine-grained searchable encryption schemes,the proposed scheme has achieved a significant reduction in storage and computational cost for the secret key associated with users.It has been verified both theoretically and practically in the performance analysis section.
