Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across vari...Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across various domains.However,the deployment of such models in resource-constrained environments presents a unique set of challenges that require innovative solutions.Resource-constrained environments encompass scenarios where computing resources,memory,and energy availability are restricted.To empower sentiment analysis in resource-constrained environments,we address the crucial need by leveraging lightweight pre-trained models.These models,derived from popular architectures such as DistilBERT,MobileBERT,ALBERT,TinyBERT,ELECTRA,and SqueezeBERT,offer a promising solution to the resource limitations imposed by these environments.By distilling the knowledge from larger models into smaller ones and employing various optimization techniques,these lightweight models aim to strike a balance between performance and resource efficiency.This paper endeavors to explore the performance of multiple lightweight pre-trained models in sentiment analysis tasks specific to such environments and provide insights into their viability for practical deployment.展开更多
Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Com...Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.展开更多
With current success of large-scale pre-trained models(PTMs),how efficiently adapting PTMs to downstream tasks has attracted tremendous attention,especially for PTMs with billions of parameters.Previous work focuses o...With current success of large-scale pre-trained models(PTMs),how efficiently adapting PTMs to downstream tasks has attracted tremendous attention,especially for PTMs with billions of parameters.Previous work focuses on designing parameter-efficient tuning paradigms but needs to save and compute the gradient of the whole computational graph.In this paper,we propose y-Tuning,an efficient yet effective paradigm to adapt frozen large-scale PTMs to specific downstream tasks.y-Tuning learns dense representations for labels y defined in a given task and aligns them to fixed feature representation.Without computing the gradients of text encoder at training phrase,y-Tuning is not only parameterefficient but also training-efficient.Experimental results show that for DeBERTaxxL with 1.6 billion parameters,y-Tuning achieves performance more than 96%of full fine-tuning on GLUE Benchmark with only 2%tunable parameters and much fewer training costs.展开更多
We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract informa...We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable parameters.Extensive experiments conducted on various datasets demonstrate the effectiveness of our approach.展开更多
In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple e...In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.展开更多
Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained La...Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained Language Models(PLMs)offers new possibilities.PLMs excel at contextual learning,potentially simplifying many natural language processing tasks.However,their application to NER remains underexplored.This paper investigates leveraging the GPT-3 PLM for NER without fine-tuning.We propose a novel scheme that utilizes carefully crafted templates and context examples selected based on semantic similarity.Our experimental results demonstrate the feasibility of this approach,suggesting a promising direction for harnessing PLMs in NER.展开更多
The Coronavirus Disease 2019(COVID-19)is wreaking havoc around the world,bring out that the enormous pressure on national health and medical staff systems.One of the most effective and critical steps in the fight agai...The Coronavirus Disease 2019(COVID-19)is wreaking havoc around the world,bring out that the enormous pressure on national health and medical staff systems.One of the most effective and critical steps in the fight against COVID-19,is to examine the patient’s lungs based on the Chest X-ray and CT generated by radiation imaging.In this paper,five keras-related deep learning models:ResNet50,InceptionResNetV2,Xception,transfer learning and pre-trained VGGNet16 is applied to formulate an classification-detection approaches of COVID-19.Two benchmark methods SVM(Support Vector Machine),CNN(Conventional Neural Networks)are provided to compare with the classification-detection approaches based on the performance indicators,i.e.,precision,recall,F1 scores,confusion matrix,classification accuracy and three types of AUC(Area Under Curve).The highest classification accuracy derived by classification-detection based on 5857 Chest X-rays and 767 Chest CTs are respectively 84%and 75%,which shows that the keras-related deep learning approaches facilitate accurate and effective COVID-19-assisted detection.展开更多
Sporadic E(Es)layers in the ionosphere are characterized by intense plasma irregularities in the E region at altitudes of 90-130 km.Because they can significantly influence radio communications and navigation systems,...Sporadic E(Es)layers in the ionosphere are characterized by intense plasma irregularities in the E region at altitudes of 90-130 km.Because they can significantly influence radio communications and navigation systems,accurate forecasting of Es layers is crucial for ensuring the precision and dependability of navigation satellite systems.In this study,we present Es predictions made by an empirical model and by a deep learning model,and analyze their differences comprehensively by comparing the model predictions to satellite RO measurements and ground-based ionosonde observations.The deep learning model exhibited significantly better performance,as indicated by its high coefficient of correlation(r=0.87)with RO observations and predictions,than did the empirical model(r=0.53).This study highlights the importance of integrating artificial intelligence technology into ionosphere modelling generally,and into predicting Es layer occurrences and characteristics,in particular.展开更多
Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism rem...Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism remains unknown.Therefore,experimental models of neuromyelitis optica spectrum disorders are essential for exploring its pathogenesis and in screening for therapeutic targets.Since most patients with neuromyelitis optica spectrum disorders are seropositive for IgG autoantibodies against aquaporin-4,which is highly expressed on the membrane of astrocyte endfeet,most current experimental models are based on aquaporin-4-IgG that initially targets astrocytes.These experimental models have successfully simulated many pathological features of neuromyelitis optica spectrum disorders,such as aquaporin-4 loss,astrocytopathy,granulocyte and macrophage infiltration,complement activation,demyelination,and neuronal loss;however,they do not fully capture the pathological process of human neuromyelitis optica spectrum disorders.In this review,we summarize the currently known pathogenic mechanisms and the development of associated experimental models in vitro,ex vivo,and in vivo for neuromyelitis optica spectrum disorders,suggest potential pathogenic mechanisms for further investigation,and provide guidance on experimental model choices.In addition,this review summarizes the latest information on pathologies and therapies for neuromyelitis optica spectrum disorders based on experimental models of aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders,offering further therapeutic targets and a theoretical basis for clinical trials.展开更多
This paper presents a comparative study of ARIMA and Neural Network AutoRegressive (NNAR) models for time series forecasting. The study focuses on simulated data generated using ARIMA(1, 1, 0) and applies both models ...This paper presents a comparative study of ARIMA and Neural Network AutoRegressive (NNAR) models for time series forecasting. The study focuses on simulated data generated using ARIMA(1, 1, 0) and applies both models for training and forecasting. Model performance is evaluated using MSE, AIC, and BIC. The models are further applied to neonatal mortality data from Saudi Arabia to assess their predictive capabilities. The results indicate that the NNAR model outperforms ARIMA in both training and forecasting.展开更多
This study was aimed to prepare landslide susceptibility maps for the Pithoragarh district in Uttarakhand,India,using advanced ensemble models that combined Radial Basis Function Networks(RBFN)with three ensemble lear...This study was aimed to prepare landslide susceptibility maps for the Pithoragarh district in Uttarakhand,India,using advanced ensemble models that combined Radial Basis Function Networks(RBFN)with three ensemble learning techniques:DAGGING(DG),MULTIBOOST(MB),and ADABOOST(AB).This combination resulted in three distinct ensemble models:DG-RBFN,MB-RBFN,and AB-RBFN.Additionally,a traditional weighted method,Information Value(IV),and a benchmark machine learning(ML)model,Multilayer Perceptron Neural Network(MLP),were employed for comparison and validation.The models were developed using ten landslide conditioning factors,which included slope,aspect,elevation,curvature,land cover,geomorphology,overburden depth,lithology,distance to rivers and distance to roads.These factors were instrumental in predicting the output variable,which was the probability of landslide occurrence.Statistical analysis of the models’performance indicated that the DG-RBFN model,with an Area Under ROC Curve(AUC)of 0.931,outperformed the other models.The AB-RBFN model achieved an AUC of 0.929,the MB-RBFN model had an AUC of 0.913,and the MLP model recorded an AUC of 0.926.These results suggest that the advanced ensemble ML model DG-RBFN was more accurate than traditional statistical model,single MLP model,and other ensemble models in preparing trustworthy landslide susceptibility maps,thereby enhancing land use planning and decision-making.展开更多
Modeling HIV/AIDS progression is critical for understanding disease dynamics and improving patient care. This study compares the Exponential and Weibull survival models, focusing on their ability to capture state-spec...Modeling HIV/AIDS progression is critical for understanding disease dynamics and improving patient care. This study compares the Exponential and Weibull survival models, focusing on their ability to capture state-specific failure rates in HIV/AIDS progression. While the Exponential model offers simplicity with a constant hazard rate, it often fails to accommodate the complexities of dynamic disease progression. In contrast, the Weibull model provides flexibility by allowing hazard rates to vary over time. Both models are evaluated within the frameworks of the Cox Proportional Hazards (Cox PH) and Accelerated Failure Time (AFT) models, incorporating critical covariates such as age, gender, CD4 count, and ART status. Statistical evaluation metrics, including Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), log-likelihood, and Pseudo-R2, were employed to assess model performance across diverse patient subgroups. Results indicate that the Weibull model consistently outperforms the Exponential model in dynamic scenarios, such as younger patients and those with co-infections, while maintaining robustness in stable contexts. This study highlights the trade-off between flexibility and simplicity in survival modeling, advocating for tailored model selection to balance interpretability and predictive accuracy. These findings provide valuable insights for optimizing HIV/AIDS management strategies and advancing survival analysis methodologies.展开更多
Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in speci...Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.展开更多
The UK’s economic growth has witnessed instability over these years. While some sectors recorded positive performances, some recorded negative performances, and these unstable economic performances led to technical r...The UK’s economic growth has witnessed instability over these years. While some sectors recorded positive performances, some recorded negative performances, and these unstable economic performances led to technical recession for the third and fourth quarters of the year 2023. This study assessed the efficacy of the Generalised Additive Model for Location, Scale and Shape (GAMLSS) as a flexible distributional regression with smoothing additive terms in forecasting the UK economic growth in-sample and out-of-sample over the conventional Autoregressive Distributed Lag (ARDL) and Error Correction Model (ECM). The aim was to investigate the effectiveness and efficiency of GAMLSS models using a machine learning framework over the conventional time series econometric models by a rolling window. It is quantitative research which adopts a dataset obtained from the Office for National Statistics, covering 105 monthly observations of major economic indicators in the UK from January 2015 to September 2023. It consists of eleven variables, which include economic growth (Econ), consumer price index (CPI), inflation (Infl), manufacturing (Manuf), electricity and gas (ElGas), construction (Const), industries (Ind), wholesale and retail (WRet), real estate (REst), education (Edu) and health (Health). All computations and graphics in this study are obtained using R software version 4.4.1. The study revealed that GAMLSS models demonstrate superior outperformance in forecast accuracy over the ARDL and ECM models. Unlike other models used in the literature, the GAMLSS models were able to forecast both the future economic growth and the future distribution of the growth, thereby contributing to the empirical literature. The study identified manufacturing, electricity and gas, construction, industries, wholesale and retail, real estate, education, and health as key drivers of UK economic growth.展开更多
Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein functio...Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein function or structure,understanding their genetic basis is crucial for accurate diagnosis and targeted therapies.To investigate the underlying pathogenesis of these conditions,researchers often use non-mammalian model organisms,such as Drosophila(fruit flies),which is valued for their genetic manipulability,cost-efficiency,and preservation of genes and biological functions across evolutionary time.Genetic tools available in Drosophila,including CRISPR-Cas9,offer a means to manipulate gene expression,allowing for a deep exploration of the genetic underpinnings of rare neurological diseases.Drosophila boasts a versatile genetic toolkit,rapid generation turnover,and ease of large-scale experimentation,making it an invaluable resource for identifying potential drug candidates.Researchers can expose flies carrying disease-associated mutations to various compounds,rapidly pinpointing promising therapeutic agents for further investigation in mammalian models and,ultimately,clinical trials.In this comprehensive review,we explore rare neurological diseases where fly research has significantly contributed to our understanding of their genetic basis,pathophysiology,and potential therapeutic implications.We discuss rare diseases associated with both neuron-expressed and glial-expressed genes.Specific cases include mutations in CDK19 resulting in epilepsy and developmental delay,mutations in TIAM1 leading to a neurodevelopmental disorder with seizures and language delay,and mutations in IRF2BPL causing seizures,a neurodevelopmental disorder with regression,loss of speech,and abnormal movements.And we explore mutations in EMC1 related to cerebellar atrophy,visual impairment,psychomotor retardation,and gain-of-function mutations in ACOX1 causing Mitchell syndrome.Loss-of-function mutations in ACOX1 result in ACOX1 deficiency,characterized by very-long-chain fatty acid accumulation and glial degeneration.Notably,this review highlights how modeling these diseases in Drosophila has provided valuable insights into their pathophysiology,offering a platform for the rapid identification of potential therapeutic interventions.Rare neurological diseases involve a wide range of expression systems,and sometimes common phenotypes can be found among different genes that cause abnormalities in neurons or glia.Furthermore,mutations within the same gene may result in varying functional outcomes,such as complete loss of function,partial loss of function,or gain-of-function mutations.The phenotypes observed in patients can differ significantly,underscoring the complexity of these conditions.In conclusion,Drosophila represents an indispensable and cost-effective tool for investigating rare neurological diseases.By facilitating the modeling of these conditions,Drosophila contributes to a deeper understanding of their genetic basis,pathophysiology,and potential therapies.This approach accelerates the discovery of promising drug candidates,ultimately benefiting patients affected by these complex and understudied diseases.展开更多
Precise and accurate rainfall simulation is essential for Tanzania, where complex topography and diverse climatic influences result in variable precipitation patterns. In this study, the 31st October 2023 to 02nd Nove...Precise and accurate rainfall simulation is essential for Tanzania, where complex topography and diverse climatic influences result in variable precipitation patterns. In this study, the 31st October 2023 to 02nd November 2023 daily observation rainfall was used to assess the performance of 5 land surface models (LSMs) and 7 microphysics schemes (MPs) using the Weather Research and Forecasting (WRF) model. The 35 different simulations were then evaluated using the observation data from the ground stations (OBS) and the gridded satellite (CHIRPS) dataset. It was found that the WSM6 scheme performed better than other MPs even though the performance of the LSMs was dependent on the observation data used. The CLM4 performed better than others when the simulations were compared with OBS whereas the 5 Layer Slab produced the lowest mean absolute error (MAE) and root mean square error (RMSE) values while the Noah-MP and RUC schemes produced the lowest average values of RMSE and MAE respectively when the CHIRPS dataset was used. The difference in performance of land surface models when compared to different sets of observation data was attributed to the fact that each observation dataset had a different number of points over the same area, influencing their performances. Furthermore, it was revealed that the CLM4-WSM6 combination performed better than others in the simulation of this event when it was compared against OBS while the 5 Layer Slab-WSM6 combination performed well when the CHIRPS dataset was used for comparison. This research highlights the critical role of the selection of land surface models and microphysics schemes in forecasting extreme rainfall events and underscores the importance of integrating different observational data for model validation. These findings contribute to improving predictive capabilities for extreme rainfall events in similar climatic regions.展开更多
Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language rep...Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next,we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.展开更多
Time series forecasting is essential for generating predictive insights across various domains, including healthcare, finance, and energy. This study focuses on forecasting patient health data by comparing the perform...Time series forecasting is essential for generating predictive insights across various domains, including healthcare, finance, and energy. This study focuses on forecasting patient health data by comparing the performance of traditional linear time series models, namely Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA, and Moving Average (MA) against neural network architectures. The primary goal is to evaluate the effectiveness of these models in predicting healthcare outcomes using patient records, specifically the Cancerpatient.xlsx dataset, which tracks variables such as patient age, symptoms, genetic risk factors, and environmental exposures over time. The proposed strategy involves training each model on historical patient data to predict age progression and other related health indicators, with performance evaluated using Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) metrics. Our findings reveal that neural networks consistently outperform ARIMA and SARIMA by capturing non-linear patterns and complex temporal dependencies within the dataset, resulting in lower forecasting errors. This research highlights the potential of neural networks to enhance predictive accuracy in healthcare applications, supporting better resource allocation, patient monitoring, and long-term health outcome predictions.展开更多
Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requir...Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.展开更多
Frozen shoulder(FS),also known as adhesive capsulitis,is a condition that causes contraction and stiffness of the shoulder joint capsule.The main symptoms are per-sistent shoulder pain and a limited range of motion in...Frozen shoulder(FS),also known as adhesive capsulitis,is a condition that causes contraction and stiffness of the shoulder joint capsule.The main symptoms are per-sistent shoulder pain and a limited range of motion in all directions.These symp-toms and poor prognosis affect people's physical health and quality of life.Currently,the specific mechanisms of FS remain unclear,and there is variability in treatment methods and their efficacy.Additionally,the early symptoms of FS are difficult to distinguish from those of other shoulder diseases,complicating early diagnosis and treatment.Therefore,it is necessary to develop and utilize animal models to under-stand the pathogenesis of FS and to explore treatment strategies,providing insights into the prevention and treatment of human FS.This paper reviews the rat models available for FS research,including external immobilization models,surgical internal immobilization models,injection modeling models,and endocrine modeling models.It introduces the basic procedures for these models and compares and analyzes the advantages,disadvantages,and applicability of each modeling method.Finally,our paper summarizes the common methods for evaluating FS rat models.展开更多
文摘Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across various domains.However,the deployment of such models in resource-constrained environments presents a unique set of challenges that require innovative solutions.Resource-constrained environments encompass scenarios where computing resources,memory,and energy availability are restricted.To empower sentiment analysis in resource-constrained environments,we address the crucial need by leveraging lightweight pre-trained models.These models,derived from popular architectures such as DistilBERT,MobileBERT,ALBERT,TinyBERT,ELECTRA,and SqueezeBERT,offer a promising solution to the resource limitations imposed by these environments.By distilling the knowledge from larger models into smaller ones and employing various optimization techniques,these lightweight models aim to strike a balance between performance and resource efficiency.This paper endeavors to explore the performance of multiple lightweight pre-trained models in sentiment analysis tasks specific to such environments and provide insights into their viability for practical deployment.
文摘Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.
基金National Key R&D Program of China(No.2020AAA0108702)National Natural Science Foundation of China(Grant No.62022027).
文摘With current success of large-scale pre-trained models(PTMs),how efficiently adapting PTMs to downstream tasks has attracted tremendous attention,especially for PTMs with billions of parameters.Previous work focuses on designing parameter-efficient tuning paradigms but needs to save and compute the gradient of the whole computational graph.In this paper,we propose y-Tuning,an efficient yet effective paradigm to adapt frozen large-scale PTMs to specific downstream tasks.y-Tuning learns dense representations for labels y defined in a given task and aligns them to fixed feature representation.Without computing the gradients of text encoder at training phrase,y-Tuning is not only parameterefficient but also training-efficient.Experimental results show that for DeBERTaxxL with 1.6 billion parameters,y-Tuning achieves performance more than 96%of full fine-tuning on GLUE Benchmark with only 2%tunable parameters and much fewer training costs.
文摘We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable parameters.Extensive experiments conducted on various datasets demonstrate the effectiveness of our approach.
基金Science and Technology Innovation 2030-Major Project of“New Generation Artificial Intelligence”granted by Ministry of Science and Technology,Grant Number 2020AAA0109300.
文摘In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.
文摘Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained Language Models(PLMs)offers new possibilities.PLMs excel at contextual learning,potentially simplifying many natural language processing tasks.However,their application to NER remains underexplored.This paper investigates leveraging the GPT-3 PLM for NER without fine-tuning.We propose a novel scheme that utilizes carefully crafted templates and context examples selected based on semantic similarity.Our experimental results demonstrate the feasibility of this approach,suggesting a promising direction for harnessing PLMs in NER.
基金This project is supported by National Natural Science Foundation of China(NSFC)(Nos.61902158,61806087)Graduate student innovation program for academic degrees in general university in Jiangsu Province(No.KYZZ16-0337).
文摘The Coronavirus Disease 2019(COVID-19)is wreaking havoc around the world,bring out that the enormous pressure on national health and medical staff systems.One of the most effective and critical steps in the fight against COVID-19,is to examine the patient’s lungs based on the Chest X-ray and CT generated by radiation imaging.In this paper,five keras-related deep learning models:ResNet50,InceptionResNetV2,Xception,transfer learning and pre-trained VGGNet16 is applied to formulate an classification-detection approaches of COVID-19.Two benchmark methods SVM(Support Vector Machine),CNN(Conventional Neural Networks)are provided to compare with the classification-detection approaches based on the performance indicators,i.e.,precision,recall,F1 scores,confusion matrix,classification accuracy and three types of AUC(Area Under Curve).The highest classification accuracy derived by classification-detection based on 5857 Chest X-rays and 767 Chest CTs are respectively 84%and 75%,which shows that the keras-related deep learning approaches facilitate accurate and effective COVID-19-assisted detection.
基金supported by the Project of Stable Support for Youth Team in Basic Research Field,CAS(grant No.YSBR-018)the National Natural Science Foundation of China(grant Nos.42188101,42130204)+4 种基金the B-type Strategic Priority Program of CAS(grant no.XDB41000000)the National Natural Science Foundation of China(NSFC)Distinguished Overseas Young Talents Program,Innovation Program for Quantum Science and Technology(2021ZD0300301)the Open Research Project of Large Research Infrastructures of CAS-“Study on the interaction between low/mid-latitude atmosphere and ionosphere based on the Chinese Meridian Project”.The project was supported also by the National Key Laboratory of Deep Space Exploration(Grant No.NKLDSE2023A002)the Open Fund of Anhui Provincial Key Laboratory of Intelligent Underground Detection(Grant No.APKLIUD23KF01)the China National Space Administration(CNSA)pre-research Project on Civil Aerospace Technologies No.D010305,D010301.
文摘Sporadic E(Es)layers in the ionosphere are characterized by intense plasma irregularities in the E region at altitudes of 90-130 km.Because they can significantly influence radio communications and navigation systems,accurate forecasting of Es layers is crucial for ensuring the precision and dependability of navigation satellite systems.In this study,we present Es predictions made by an empirical model and by a deep learning model,and analyze their differences comprehensively by comparing the model predictions to satellite RO measurements and ground-based ionosonde observations.The deep learning model exhibited significantly better performance,as indicated by its high coefficient of correlation(r=0.87)with RO observations and predictions,than did the empirical model(r=0.53).This study highlights the importance of integrating artificial intelligence technology into ionosphere modelling generally,and into predicting Es layer occurrences and characteristics,in particular.
文摘Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism remains unknown.Therefore,experimental models of neuromyelitis optica spectrum disorders are essential for exploring its pathogenesis and in screening for therapeutic targets.Since most patients with neuromyelitis optica spectrum disorders are seropositive for IgG autoantibodies against aquaporin-4,which is highly expressed on the membrane of astrocyte endfeet,most current experimental models are based on aquaporin-4-IgG that initially targets astrocytes.These experimental models have successfully simulated many pathological features of neuromyelitis optica spectrum disorders,such as aquaporin-4 loss,astrocytopathy,granulocyte and macrophage infiltration,complement activation,demyelination,and neuronal loss;however,they do not fully capture the pathological process of human neuromyelitis optica spectrum disorders.In this review,we summarize the currently known pathogenic mechanisms and the development of associated experimental models in vitro,ex vivo,and in vivo for neuromyelitis optica spectrum disorders,suggest potential pathogenic mechanisms for further investigation,and provide guidance on experimental model choices.In addition,this review summarizes the latest information on pathologies and therapies for neuromyelitis optica spectrum disorders based on experimental models of aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders,offering further therapeutic targets and a theoretical basis for clinical trials.
文摘This paper presents a comparative study of ARIMA and Neural Network AutoRegressive (NNAR) models for time series forecasting. The study focuses on simulated data generated using ARIMA(1, 1, 0) and applies both models for training and forecasting. Model performance is evaluated using MSE, AIC, and BIC. The models are further applied to neonatal mortality data from Saudi Arabia to assess their predictive capabilities. The results indicate that the NNAR model outperforms ARIMA in both training and forecasting.
基金the University of Transport Technology under the project entitled“Application of Machine Learning Algorithms in Landslide Susceptibility Mapping in Mountainous Areas”with grant number DTTD2022-16.
文摘This study was aimed to prepare landslide susceptibility maps for the Pithoragarh district in Uttarakhand,India,using advanced ensemble models that combined Radial Basis Function Networks(RBFN)with three ensemble learning techniques:DAGGING(DG),MULTIBOOST(MB),and ADABOOST(AB).This combination resulted in three distinct ensemble models:DG-RBFN,MB-RBFN,and AB-RBFN.Additionally,a traditional weighted method,Information Value(IV),and a benchmark machine learning(ML)model,Multilayer Perceptron Neural Network(MLP),were employed for comparison and validation.The models were developed using ten landslide conditioning factors,which included slope,aspect,elevation,curvature,land cover,geomorphology,overburden depth,lithology,distance to rivers and distance to roads.These factors were instrumental in predicting the output variable,which was the probability of landslide occurrence.Statistical analysis of the models’performance indicated that the DG-RBFN model,with an Area Under ROC Curve(AUC)of 0.931,outperformed the other models.The AB-RBFN model achieved an AUC of 0.929,the MB-RBFN model had an AUC of 0.913,and the MLP model recorded an AUC of 0.926.These results suggest that the advanced ensemble ML model DG-RBFN was more accurate than traditional statistical model,single MLP model,and other ensemble models in preparing trustworthy landslide susceptibility maps,thereby enhancing land use planning and decision-making.
文摘Modeling HIV/AIDS progression is critical for understanding disease dynamics and improving patient care. This study compares the Exponential and Weibull survival models, focusing on their ability to capture state-specific failure rates in HIV/AIDS progression. While the Exponential model offers simplicity with a constant hazard rate, it often fails to accommodate the complexities of dynamic disease progression. In contrast, the Weibull model provides flexibility by allowing hazard rates to vary over time. Both models are evaluated within the frameworks of the Cox Proportional Hazards (Cox PH) and Accelerated Failure Time (AFT) models, incorporating critical covariates such as age, gender, CD4 count, and ART status. Statistical evaluation metrics, including Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), log-likelihood, and Pseudo-R2, were employed to assess model performance across diverse patient subgroups. Results indicate that the Weibull model consistently outperforms the Exponential model in dynamic scenarios, such as younger patients and those with co-infections, while maintaining robustness in stable contexts. This study highlights the trade-off between flexibility and simplicity in survival modeling, advocating for tailored model selection to balance interpretability and predictive accuracy. These findings provide valuable insights for optimizing HIV/AIDS management strategies and advancing survival analysis methodologies.
基金supported by the National Key R&D Program of China(No.2021YFB0301200)National Natural Science Foundation of China(No.62025208).
文摘Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.
文摘The UK’s economic growth has witnessed instability over these years. While some sectors recorded positive performances, some recorded negative performances, and these unstable economic performances led to technical recession for the third and fourth quarters of the year 2023. This study assessed the efficacy of the Generalised Additive Model for Location, Scale and Shape (GAMLSS) as a flexible distributional regression with smoothing additive terms in forecasting the UK economic growth in-sample and out-of-sample over the conventional Autoregressive Distributed Lag (ARDL) and Error Correction Model (ECM). The aim was to investigate the effectiveness and efficiency of GAMLSS models using a machine learning framework over the conventional time series econometric models by a rolling window. It is quantitative research which adopts a dataset obtained from the Office for National Statistics, covering 105 monthly observations of major economic indicators in the UK from January 2015 to September 2023. It consists of eleven variables, which include economic growth (Econ), consumer price index (CPI), inflation (Infl), manufacturing (Manuf), electricity and gas (ElGas), construction (Const), industries (Ind), wholesale and retail (WRet), real estate (REst), education (Edu) and health (Health). All computations and graphics in this study are obtained using R software version 4.4.1. The study revealed that GAMLSS models demonstrate superior outperformance in forecast accuracy over the ARDL and ECM models. Unlike other models used in the literature, the GAMLSS models were able to forecast both the future economic growth and the future distribution of the growth, thereby contributing to the empirical literature. The study identified manufacturing, electricity and gas, construction, industries, wholesale and retail, real estate, education, and health as key drivers of UK economic growth.
基金supported by Warren Alpert Foundation and Houston Methodist Academic Institute Laboratory Operating Fund(to HLC).
文摘Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein function or structure,understanding their genetic basis is crucial for accurate diagnosis and targeted therapies.To investigate the underlying pathogenesis of these conditions,researchers often use non-mammalian model organisms,such as Drosophila(fruit flies),which is valued for their genetic manipulability,cost-efficiency,and preservation of genes and biological functions across evolutionary time.Genetic tools available in Drosophila,including CRISPR-Cas9,offer a means to manipulate gene expression,allowing for a deep exploration of the genetic underpinnings of rare neurological diseases.Drosophila boasts a versatile genetic toolkit,rapid generation turnover,and ease of large-scale experimentation,making it an invaluable resource for identifying potential drug candidates.Researchers can expose flies carrying disease-associated mutations to various compounds,rapidly pinpointing promising therapeutic agents for further investigation in mammalian models and,ultimately,clinical trials.In this comprehensive review,we explore rare neurological diseases where fly research has significantly contributed to our understanding of their genetic basis,pathophysiology,and potential therapeutic implications.We discuss rare diseases associated with both neuron-expressed and glial-expressed genes.Specific cases include mutations in CDK19 resulting in epilepsy and developmental delay,mutations in TIAM1 leading to a neurodevelopmental disorder with seizures and language delay,and mutations in IRF2BPL causing seizures,a neurodevelopmental disorder with regression,loss of speech,and abnormal movements.And we explore mutations in EMC1 related to cerebellar atrophy,visual impairment,psychomotor retardation,and gain-of-function mutations in ACOX1 causing Mitchell syndrome.Loss-of-function mutations in ACOX1 result in ACOX1 deficiency,characterized by very-long-chain fatty acid accumulation and glial degeneration.Notably,this review highlights how modeling these diseases in Drosophila has provided valuable insights into their pathophysiology,offering a platform for the rapid identification of potential therapeutic interventions.Rare neurological diseases involve a wide range of expression systems,and sometimes common phenotypes can be found among different genes that cause abnormalities in neurons or glia.Furthermore,mutations within the same gene may result in varying functional outcomes,such as complete loss of function,partial loss of function,or gain-of-function mutations.The phenotypes observed in patients can differ significantly,underscoring the complexity of these conditions.In conclusion,Drosophila represents an indispensable and cost-effective tool for investigating rare neurological diseases.By facilitating the modeling of these conditions,Drosophila contributes to a deeper understanding of their genetic basis,pathophysiology,and potential therapies.This approach accelerates the discovery of promising drug candidates,ultimately benefiting patients affected by these complex and understudied diseases.
文摘Precise and accurate rainfall simulation is essential for Tanzania, where complex topography and diverse climatic influences result in variable precipitation patterns. In this study, the 31st October 2023 to 02nd November 2023 daily observation rainfall was used to assess the performance of 5 land surface models (LSMs) and 7 microphysics schemes (MPs) using the Weather Research and Forecasting (WRF) model. The 35 different simulations were then evaluated using the observation data from the ground stations (OBS) and the gridded satellite (CHIRPS) dataset. It was found that the WSM6 scheme performed better than other MPs even though the performance of the LSMs was dependent on the observation data used. The CLM4 performed better than others when the simulations were compared with OBS whereas the 5 Layer Slab produced the lowest mean absolute error (MAE) and root mean square error (RMSE) values while the Noah-MP and RUC schemes produced the lowest average values of RMSE and MAE respectively when the CHIRPS dataset was used. The difference in performance of land surface models when compared to different sets of observation data was attributed to the fact that each observation dataset had a different number of points over the same area, influencing their performances. Furthermore, it was revealed that the CLM4-WSM6 combination performed better than others in the simulation of this event when it was compared against OBS while the 5 Layer Slab-WSM6 combination performed well when the CHIRPS dataset was used for comparison. This research highlights the critical role of the selection of land surface models and microphysics schemes in forecasting extreme rainfall events and underscores the importance of integrating different observational data for model validation. These findings contribute to improving predictive capabilities for extreme rainfall events in similar climatic regions.
基金the National Natural Science Foundation of China(Grant Nos.61751201 and 61672162)the Shanghai Municipal Science and Technology Major Project(Grant No.2018SHZDZX01)and ZJLab。
文摘Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next,we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.
文摘Time series forecasting is essential for generating predictive insights across various domains, including healthcare, finance, and energy. This study focuses on forecasting patient health data by comparing the performance of traditional linear time series models, namely Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA, and Moving Average (MA) against neural network architectures. The primary goal is to evaluate the effectiveness of these models in predicting healthcare outcomes using patient records, specifically the Cancerpatient.xlsx dataset, which tracks variables such as patient age, symptoms, genetic risk factors, and environmental exposures over time. The proposed strategy involves training each model on historical patient data to predict age progression and other related health indicators, with performance evaluated using Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) metrics. Our findings reveal that neural networks consistently outperform ARIMA and SARIMA by capturing non-linear patterns and complex temporal dependencies within the dataset, resulting in lower forecasting errors. This research highlights the potential of neural networks to enhance predictive accuracy in healthcare applications, supporting better resource allocation, patient monitoring, and long-term health outcome predictions.
文摘Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.
基金National Key R&D Program of China,Grant/Award Number:2021YFC2502100,2023YFC3603404 and 2019YFA0111900The National Natural Science Foundation of China,Grant/Award Number:82072506,82272611 and 92268115+7 种基金The Hunan Provincial Science Fund for Distinguished Young Scholars,Grant/Award Number:2024JJ2089The Hunan Young Talents of Science and Technology,Grant/Award Number:2021RC3025The Provincial Clinical Medical Technology Innovation Project of Hunan,Grant/Award Number:2023SK2024 and 2020SK53709The Provincial Natural Science Foundation of Hunan,Grant/Award Number:2020JJ3060The National Natural Science Foundation of Hunan Province,Grant/Award Number:2023JJ30949The National Clinical Research Center for Geriatric Disorders,Xiangya Hospital,Grant/Award Number:2021KFJJ02 and 2021LNJJ05The Hunan Provincial Innovation Foundation for Postgraduate,Grant/Award Number:CX20230308 and CX20230312The Independent Exploration and Innovation Project for Postgraduate Students of Central South University,Grant/Award Number:2024ZZTS0163。
文摘Frozen shoulder(FS),also known as adhesive capsulitis,is a condition that causes contraction and stiffness of the shoulder joint capsule.The main symptoms are per-sistent shoulder pain and a limited range of motion in all directions.These symp-toms and poor prognosis affect people's physical health and quality of life.Currently,the specific mechanisms of FS remain unclear,and there is variability in treatment methods and their efficacy.Additionally,the early symptoms of FS are difficult to distinguish from those of other shoulder diseases,complicating early diagnosis and treatment.Therefore,it is necessary to develop and utilize animal models to under-stand the pathogenesis of FS and to explore treatment strategies,providing insights into the prevention and treatment of human FS.This paper reviews the rat models available for FS research,including external immobilization models,surgical internal immobilization models,injection modeling models,and endocrine modeling models.It introduces the basic procedures for these models and compares and analyzes the advantages,disadvantages,and applicability of each modeling method.Finally,our paper summarizes the common methods for evaluating FS rat models.
