Activity recognition plays a key role in health management and security.Traditional approaches are based on vision or wearables,which only work under the line of sight(LOS)or require the targets to carry dedicated dev...Activity recognition plays a key role in health management and security.Traditional approaches are based on vision or wearables,which only work under the line of sight(LOS)or require the targets to carry dedicated devices.As human bodies and their movements have influences on WiFi propagation,this paper proposes the recognition of human activities by analyzing the channel state information(CSI)from the WiFi physical layer.The method requires only the commodity:WiFi transmitters and receivers that can operate through a wall,under LOS and non-line of sight(NLOS),while the targets are not required to carry dedicated devices.After collecting CSI,the discrete wavelet transform is applied to reduce the noise,followed by outlier detection based on the local outlier factor to extract the activity segment.Activity recognition is fulfilled by using the bi-directional long short-term memory that takes the sequential features into consideration.Experiments in through-the-wall environments achieve recognition accuracy>95%for six common activities,such as standing up,squatting down,walking,running,jumping,and falling,outperforming existing work in this field.展开更多
A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a force...A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.展开更多
The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning mode...The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.展开更多
To enhance the refinement of load decomposition in power systems and fully leverage seasonal change information to further improve prediction performance,this paper proposes a seasonal short-termload combination predi...To enhance the refinement of load decomposition in power systems and fully leverage seasonal change information to further improve prediction performance,this paper proposes a seasonal short-termload combination prediction model based on modal decomposition and a feature-fusion multi-algorithm hybrid neural network model.Specifically,the characteristics of load components are analyzed for different seasons,and the corresponding models are established.First,the improved complete ensemble empirical modal decomposition with adaptive noise(ICEEMDAN)method is employed to decompose the system load for all four seasons,and the new sequence is obtained through reconstruction based on the refined composite multiscale fuzzy entropy of each decomposition component.Second,the correlation between different decomposition components and different features is measured through the max-relevance and min-redundancy method to filter out the subset of features with strong correlation and low redundancy.Finally,different components of the load in different seasons are predicted separately using a bidirectional long-short-term memory network model based on a Bayesian optimization algorithm,with a prediction resolution of 15 min,and the predicted values are accumulated to obtain the final results.According to the experimental findings,the proposed method can successfully balance prediction accuracy and prediction time while offering a higher level of prediction accuracy than the current prediction methods.The results demonstrate that the proposedmethod can effectively address the load power variation induced by seasonal differences in different regions.展开更多
Architecture framework has become an effective method recently to describe the system of systems(SoS)architecture,such as the United States(US)Department of Defense Architecture Framework Version 2.0(DoDAF2.0).As a vi...Architecture framework has become an effective method recently to describe the system of systems(SoS)architecture,such as the United States(US)Department of Defense Architecture Framework Version 2.0(DoDAF2.0).As a viewpoint in DoDAF2.0,the operational viewpoint(OV)describes operational activities,nodes,and resource flows.The OV models are important for SoS architecture development.However,as the SoS complexity increases,constructing OV models with traditional methods exposes shortcomings,such as inefficient data collection and low modeling standards.Therefore,we propose an intelligent modeling method for five OV models,including operational resource flow OV-2,organizational relationships OV-4,operational activity hierarchy OV-5a,operational activities model OV-5b,and operational activity sequences OV-6c.The main idea of the method is to extract OV architecture data from text and generate interoperable OV models.First,we construct the OV meta model based on the DoDAF2.0 meta model(DM2).Second,OV architecture named entities is recognized from text based on the bidirectional long short-term memory and conditional random field(BiLSTM-CRF)model.And OV architecture relationships are collected with relationship extraction rules.Finally,we define the generation rules for OV models and develop an OV modeling tool.We use unmanned surface vehicles(USV)swarm target defense SoS architecture as a case to verify the feasibility and effectiveness of the intelligent modeling method.展开更多
Accurately predicting motion responses is a crucial component of the design process for floating offshore structures.This study introduces a hybrid model that integrates a convolutional neural network(CNN),a bidirecti...Accurately predicting motion responses is a crucial component of the design process for floating offshore structures.This study introduces a hybrid model that integrates a convolutional neural network(CNN),a bidirectional long short-term memory(BiLSTM)neural network,and an attention mechanism for forecasting the short-term motion responses of a semisubmersible.First,the motions are processed through the CNN for feature extraction.The extracted features are subsequently utilized by the BiLSTM network to forecast future motions.To enhance the predictive capability of the neural networks,an attention mechanism is integrated.In addition to the hybrid model,the BiLSTM is independently employed to forecast the motion responses of the semi-submersible,serving as benchmark results for comparison.Furthermore,both the 1D and 2D convolutions are conducted to check the influence of the convolutional dimensionality on the predicted results.The results demonstrate that the hybrid 1D CNN-BiLSTM network with an attention mechanism outperforms all other models in accurately predicting motion responses.展开更多
针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from...针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from transformers)预训练模型得到输入序列语义的词向量;然后将训练后的词向量输入双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)模型进一步获取上下文特征;最后根据条件随机场(conditional random fields,CRF)的标注规则和序列解码能力输出最大概率序列标注结果,构建油气领域命名实体识别模型框架。将BERT-BiLSTM-CRF模型与其他2种命名实体识别模型(BiLSTM-CRF、BiLSTM-Attention-CRF)在包括3万多条文本语料数据、4类实体的自建数据集上进行了对比实验。实验结果表明,BERT-BiLSTM-CRF模型的准确率(P)、召回率(R)和F_(1)值分别达到91.3%、94.5%和92.9%,实体识别效果优于其他2种模型。展开更多
Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the ...Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.展开更多
The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended inter...The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended intervals and time delays in time series data.Additionally,the sequence-to-sequence(Seq2Seq)model,known for handling temporal relationships,adapting to variable-length sequences,effectively capturing historical information,and accommodating various influencing factors,emerges as a robust and flexible tool in discharge forecasting.In this study,we introduce the application of LSTM-based Seq2Seq models for the first time in forecasting the discharge of a tidal reach of the Changjiang River(Yangtze River)Estuary.This study focuses on discharge forecasting using three key input characteristics:flow velocity,water level,and discharge,which means the structure of multiple input and single output is adopted.The experiment used the discharge data of the whole year of 2020,of which the first 80%is used as the training set,and the last 20%is used as the test set.This means that the data covers different tidal cycles,which helps to test the forecasting effect of different models in different tidal cycles and different runoff.The experimental results indicate that the proposed models demonstrate advantages in long-term,mid-term,and short-term discharge forecasting.The Seq2Seq models improved by 6%-60%and 5%-20%of the relative standard deviation compared to the harmonic analysis models and improved back propagation neural network models in discharge prediction,respectively.In addition,the relative accuracy of the Seq2Seq model is 1%to 3%higher than that of the LSTM model.Analytical assessment of the prediction errors shows that the Seq2Seq models are insensitive to the forecast lead time and they can capture characteristic values such as maximum flood tide flow and maximum ebb tide flow in the tidal cycle well.This indicates the significance of the Seq2Seq models.展开更多
Determining which variables affect price realized volatility has always been challenging.This paper proposes to explain how financial assets influence realized volatility by developing an optimal day-to-day forecast.T...Determining which variables affect price realized volatility has always been challenging.This paper proposes to explain how financial assets influence realized volatility by developing an optimal day-to-day forecast.The methodological proposal is based on using the best econometric and machine learning models to forecast realized volatility.In particular,the best forecasting from heterogeneous autoregressive and long short-term memory models are used to determine the influence of the Standard and Poor’s 500 index,euro-US dollar exchange rate,price of gold,and price of Brent crude oil on the realized volatility of natural gas.These financial assets influenced the realized volatility of natural gas in 87.4% of the days analyzed;the euro-US dollar exchange rate was the primary financial asset and explained 40.1% of the influence.The results of the proposed daily analysis differed from those of the methodology used to study the entire period.The traditional model,which studies the entire period,cannot determine temporal effects,whereas the proposed methodology can.The proposed methodology allows us to distinguish the effects for each day,week,or month rather than averages for entire periods,with the flexibility to analyze different frequencies and periods.This methodological capability is key to analyzing influences and making decisions about realized volatility.展开更多
Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural ...Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.展开更多
The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the ...The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the inception of financial markets.By examining historical transaction data,latent opportunities for profit can be uncovered,providing valuable insights for both institutional and individual investors to make more informed decisions.This study focuses on analyzing historical transaction data from four banks to predict closing price trends.Various models,including decision trees,random forests,and Long Short-Term Memory(LSTM)networks,are employed to forecast stock price movements.Historical stock transaction data serves as the input for training these models,which are then used to predict upward or downward stock price trends.The study’s empirical results indicate that these methods are effective to a degree in predicting stock price movements.The LSTM-based deep neural network model,in particular,demonstrates a commendable level of predictive accuracy.This conclusion is reached following a thorough evaluation of model performance,highlighting the potential of LSTM models in stock market forecasting.The findings offer significant implications for advancing financial forecasting approaches,thereby improving the decision-making capabilities of investors and financial institutions.展开更多
Very short-term prediction of ship motion is critically important in many scenarios such as carrier aircraft landings and marine engineering operations.This paper introduces the newly developed functional deep learnin...Very short-term prediction of ship motion is critically important in many scenarios such as carrier aircraft landings and marine engineering operations.This paper introduces the newly developed functional deep learning model,named as Deep Operator networks neural network(DeepOnet)to predict very short-term ship motion in waves.It takes wave height as input and predicts ship motion as output,employing a cause-to-effect prediction approach.The modeling data for this study is derived from publicly available experimental data at the Iowa Institute of Hydraulic Research.Initially,the tuning of the hyperparameters within the neural network system was conducted to identify the optimal parameter combination.Subsequently,the DeepOnet model for wave height and multi-degree-of-freedom motion was established,and the impact of increasing time steps on prediction accuracy was analyzed.Lastly,a comparative analysis was performed between the DeepOnet model and the classical time series model,long short-term memory(LSTM).It was observed that the DeepOnet model exhibited a tenfold improvement in accuracy for roll and heave motions.Furthermore,as the forecast duration increased,the advantage of the DeepOnet model showed a trend of strengthening.As a functional prediction model,DeepOnet offers a novel and promising tool for very short-term ship motion prediction.展开更多
为解决文本类地铁应急处置流程存在的流程顺序关系不明确、流程执行人员模糊等问题,提出了基于BiLSTM-CRF(Bidirectional Long Short-Term Memory-Conditional Random Field)的地铁应急处置知识抽取与推理方法。首先,利用BiLSTM-CRF方...为解决文本类地铁应急处置流程存在的流程顺序关系不明确、流程执行人员模糊等问题,提出了基于BiLSTM-CRF(Bidirectional Long Short-Term Memory-Conditional Random Field)的地铁应急处置知识抽取与推理方法。首先,利用BiLSTM-CRF方法对地铁应急处置流程的文本资料进行命名实体识别,完成文本资料的知识抽取;其次,选用TransD模型对识别后实体数据进行知识推理,从而完成以实体和属性对为节点、关系对为边的知识图谱构建;最后,利用Neo4j图数据库对构建的地铁应急处置流程知识图谱进行了可视化展示和案例分析。研究结果表明,基于BiLSTM-CRF的知识抽取模型的精确率、召回率和F1值均达到了90%以上,且基于BiLSTM-CRF的TransD模型的推理结果准确率提升了22.92%,保证了知识图谱构建的准确性,可为地铁应急管理提供决策支持。展开更多
基金the Key Research and Development Projects of Sichuan Science and Technology Department under Grant No.2018GZ0464the UESTC-ZHIXIAOJING Joint Research Center of Smart Home under Grant No.H04W210180.
文摘Activity recognition plays a key role in health management and security.Traditional approaches are based on vision or wearables,which only work under the line of sight(LOS)or require the targets to carry dedicated devices.As human bodies and their movements have influences on WiFi propagation,this paper proposes the recognition of human activities by analyzing the channel state information(CSI)from the WiFi physical layer.The method requires only the commodity:WiFi transmitters and receivers that can operate through a wall,under LOS and non-line of sight(NLOS),while the targets are not required to carry dedicated devices.After collecting CSI,the discrete wavelet transform is applied to reduce the noise,followed by outlier detection based on the local outlier factor to extract the activity segment.Activity recognition is fulfilled by using the bi-directional long short-term memory that takes the sequential features into consideration.Experiments in through-the-wall environments achieve recognition accuracy>95%for six common activities,such as standing up,squatting down,walking,running,jumping,and falling,outperforming existing work in this field.
基金supported by the Ministry of Trade,Industry & Energy(MOTIE,Korea) under Industrial Technology Innovation Program (No.10063424,'development of distant speech recognition and multi-task dialog processing technologies for in-door conversational robots')
文摘A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.
基金funded by the National Natural Science Foundation of China (41807285)。
文摘The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.
文摘To enhance the refinement of load decomposition in power systems and fully leverage seasonal change information to further improve prediction performance,this paper proposes a seasonal short-termload combination prediction model based on modal decomposition and a feature-fusion multi-algorithm hybrid neural network model.Specifically,the characteristics of load components are analyzed for different seasons,and the corresponding models are established.First,the improved complete ensemble empirical modal decomposition with adaptive noise(ICEEMDAN)method is employed to decompose the system load for all four seasons,and the new sequence is obtained through reconstruction based on the refined composite multiscale fuzzy entropy of each decomposition component.Second,the correlation between different decomposition components and different features is measured through the max-relevance and min-redundancy method to filter out the subset of features with strong correlation and low redundancy.Finally,different components of the load in different seasons are predicted separately using a bidirectional long-short-term memory network model based on a Bayesian optimization algorithm,with a prediction resolution of 15 min,and the predicted values are accumulated to obtain the final results.According to the experimental findings,the proposed method can successfully balance prediction accuracy and prediction time while offering a higher level of prediction accuracy than the current prediction methods.The results demonstrate that the proposedmethod can effectively address the load power variation induced by seasonal differences in different regions.
基金National Natural Science Foundation of China(71690233,71971213,71901214)。
文摘Architecture framework has become an effective method recently to describe the system of systems(SoS)architecture,such as the United States(US)Department of Defense Architecture Framework Version 2.0(DoDAF2.0).As a viewpoint in DoDAF2.0,the operational viewpoint(OV)describes operational activities,nodes,and resource flows.The OV models are important for SoS architecture development.However,as the SoS complexity increases,constructing OV models with traditional methods exposes shortcomings,such as inefficient data collection and low modeling standards.Therefore,we propose an intelligent modeling method for five OV models,including operational resource flow OV-2,organizational relationships OV-4,operational activity hierarchy OV-5a,operational activities model OV-5b,and operational activity sequences OV-6c.The main idea of the method is to extract OV architecture data from text and generate interoperable OV models.First,we construct the OV meta model based on the DoDAF2.0 meta model(DM2).Second,OV architecture named entities is recognized from text based on the bidirectional long short-term memory and conditional random field(BiLSTM-CRF)model.And OV architecture relationships are collected with relationship extraction rules.Finally,we define the generation rules for OV models and develop an OV modeling tool.We use unmanned surface vehicles(USV)swarm target defense SoS architecture as a case to verify the feasibility and effectiveness of the intelligent modeling method.
基金the National Natural Science Foundation of China (Grant No. 52301322)the Jiangsu Provincial Natural Science Foundation (Grant No. BK20220653)+1 种基金the National Science Fund for Distinguished Young Scholars (Grant No. 52025112)the Key Projects of the National Natural Science Foundation of China (Grant No. 52331011)
文摘Accurately predicting motion responses is a crucial component of the design process for floating offshore structures.This study introduces a hybrid model that integrates a convolutional neural network(CNN),a bidirectional long short-term memory(BiLSTM)neural network,and an attention mechanism for forecasting the short-term motion responses of a semisubmersible.First,the motions are processed through the CNN for feature extraction.The extracted features are subsequently utilized by the BiLSTM network to forecast future motions.To enhance the predictive capability of the neural networks,an attention mechanism is integrated.In addition to the hybrid model,the BiLSTM is independently employed to forecast the motion responses of the semi-submersible,serving as benchmark results for comparison.Furthermore,both the 1D and 2D convolutions are conducted to check the influence of the convolutional dimensionality on the predicted results.The results demonstrate that the hybrid 1D CNN-BiLSTM network with an attention mechanism outperforms all other models in accurately predicting motion responses.
文摘针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from transformers)预训练模型得到输入序列语义的词向量;然后将训练后的词向量输入双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)模型进一步获取上下文特征;最后根据条件随机场(conditional random fields,CRF)的标注规则和序列解码能力输出最大概率序列标注结果,构建油气领域命名实体识别模型框架。将BERT-BiLSTM-CRF模型与其他2种命名实体识别模型(BiLSTM-CRF、BiLSTM-Attention-CRF)在包括3万多条文本语料数据、4类实体的自建数据集上进行了对比实验。实验结果表明,BERT-BiLSTM-CRF模型的准确率(P)、召回率(R)和F_(1)值分别达到91.3%、94.5%和92.9%,实体识别效果优于其他2种模型。
基金supported by Sichuan Science and Technology Program(2023YFSY0026,2023YFH0004).
文摘Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.
基金The National Natural Science Foundation of China under contract Nos 42266006 and 41806114the Jiangxi Provincial Natural Science Foundation under contract Nos 20232BAB204089 and 20202ACBL214019.
文摘The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended intervals and time delays in time series data.Additionally,the sequence-to-sequence(Seq2Seq)model,known for handling temporal relationships,adapting to variable-length sequences,effectively capturing historical information,and accommodating various influencing factors,emerges as a robust and flexible tool in discharge forecasting.In this study,we introduce the application of LSTM-based Seq2Seq models for the first time in forecasting the discharge of a tidal reach of the Changjiang River(Yangtze River)Estuary.This study focuses on discharge forecasting using three key input characteristics:flow velocity,water level,and discharge,which means the structure of multiple input and single output is adopted.The experiment used the discharge data of the whole year of 2020,of which the first 80%is used as the training set,and the last 20%is used as the test set.This means that the data covers different tidal cycles,which helps to test the forecasting effect of different models in different tidal cycles and different runoff.The experimental results indicate that the proposed models demonstrate advantages in long-term,mid-term,and short-term discharge forecasting.The Seq2Seq models improved by 6%-60%and 5%-20%of the relative standard deviation compared to the harmonic analysis models and improved back propagation neural network models in discharge prediction,respectively.In addition,the relative accuracy of the Seq2Seq model is 1%to 3%higher than that of the LSTM model.Analytical assessment of the prediction errors shows that the Seq2Seq models are insensitive to the forecast lead time and they can capture characteristic values such as maximum flood tide flow and maximum ebb tide flow in the tidal cycle well.This indicates the significance of the Seq2Seq models.
文摘Determining which variables affect price realized volatility has always been challenging.This paper proposes to explain how financial assets influence realized volatility by developing an optimal day-to-day forecast.The methodological proposal is based on using the best econometric and machine learning models to forecast realized volatility.In particular,the best forecasting from heterogeneous autoregressive and long short-term memory models are used to determine the influence of the Standard and Poor’s 500 index,euro-US dollar exchange rate,price of gold,and price of Brent crude oil on the realized volatility of natural gas.These financial assets influenced the realized volatility of natural gas in 87.4% of the days analyzed;the euro-US dollar exchange rate was the primary financial asset and explained 40.1% of the influence.The results of the proposed daily analysis differed from those of the methodology used to study the entire period.The traditional model,which studies the entire period,cannot determine temporal effects,whereas the proposed methodology can.The proposed methodology allows us to distinguish the effects for each day,week,or month rather than averages for entire periods,with the flexibility to analyze different frequencies and periods.This methodological capability is key to analyzing influences and making decisions about realized volatility.
文摘Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.
文摘The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the inception of financial markets.By examining historical transaction data,latent opportunities for profit can be uncovered,providing valuable insights for both institutional and individual investors to make more informed decisions.This study focuses on analyzing historical transaction data from four banks to predict closing price trends.Various models,including decision trees,random forests,and Long Short-Term Memory(LSTM)networks,are employed to forecast stock price movements.Historical stock transaction data serves as the input for training these models,which are then used to predict upward or downward stock price trends.The study’s empirical results indicate that these methods are effective to a degree in predicting stock price movements.The LSTM-based deep neural network model,in particular,demonstrates a commendable level of predictive accuracy.This conclusion is reached following a thorough evaluation of model performance,highlighting the potential of LSTM models in stock market forecasting.The findings offer significant implications for advancing financial forecasting approaches,thereby improving the decision-making capabilities of investors and financial institutions.
基金Project supported by the National Natural Science Foundation of China(Grant No.51679021)supported by the Southern Marine Science and Engineering Guangdong Laboratory(Guangzhou)(Grant Nos.GML20240001,GML2024009).
文摘Very short-term prediction of ship motion is critically important in many scenarios such as carrier aircraft landings and marine engineering operations.This paper introduces the newly developed functional deep learning model,named as Deep Operator networks neural network(DeepOnet)to predict very short-term ship motion in waves.It takes wave height as input and predicts ship motion as output,employing a cause-to-effect prediction approach.The modeling data for this study is derived from publicly available experimental data at the Iowa Institute of Hydraulic Research.Initially,the tuning of the hyperparameters within the neural network system was conducted to identify the optimal parameter combination.Subsequently,the DeepOnet model for wave height and multi-degree-of-freedom motion was established,and the impact of increasing time steps on prediction accuracy was analyzed.Lastly,a comparative analysis was performed between the DeepOnet model and the classical time series model,long short-term memory(LSTM).It was observed that the DeepOnet model exhibited a tenfold improvement in accuracy for roll and heave motions.Furthermore,as the forecast duration increased,the advantage of the DeepOnet model showed a trend of strengthening.As a functional prediction model,DeepOnet offers a novel and promising tool for very short-term ship motion prediction.
文摘为解决文本类地铁应急处置流程存在的流程顺序关系不明确、流程执行人员模糊等问题,提出了基于BiLSTM-CRF(Bidirectional Long Short-Term Memory-Conditional Random Field)的地铁应急处置知识抽取与推理方法。首先,利用BiLSTM-CRF方法对地铁应急处置流程的文本资料进行命名实体识别,完成文本资料的知识抽取;其次,选用TransD模型对识别后实体数据进行知识推理,从而完成以实体和属性对为节点、关系对为边的知识图谱构建;最后,利用Neo4j图数据库对构建的地铁应急处置流程知识图谱进行了可视化展示和案例分析。研究结果表明,基于BiLSTM-CRF的知识抽取模型的精确率、召回率和F1值均达到了90%以上,且基于BiLSTM-CRF的TransD模型的推理结果准确率提升了22.92%,保证了知识图谱构建的准确性,可为地铁应急管理提供决策支持。
