Communication between people with disabilities and people who do not understand sign language is a growing social need and can be a tedious task.One of the main functions of sign language is to communicate with each o...Communication between people with disabilities and people who do not understand sign language is a growing social need and can be a tedious task.One of the main functions of sign language is to communicate with each other through hand gestures.Recognition of hand gestures has become an important challenge for the recognition of sign language.There are many existing models that can produce a good accuracy,but if the model test with rotated or translated images,they may face some difficulties to make good performance accuracy.To resolve these challenges of hand gesture recognition,we proposed a Rotation,Translation and Scale-invariant sign word recognition system using a convolu-tional neural network(CNN).We have followed three steps in our work:rotated,translated and scaled(RTS)version dataset generation,gesture segmentation,and sign word classification.Firstly,we have enlarged a benchmark dataset of 20 sign words by making different amounts of Rotation,Translation and Scale of the ori-ginal images to create the RTS version dataset.Then we have applied the gesture segmentation technique.The segmentation consists of three levels,i)Otsu Thresholding with YCbCr,ii)Morphological analysis:dilation through opening morphology and iii)Watershed algorithm.Finally,our designed CNN model has been trained to classify the hand gesture as well as the sign word.Our model has been evaluated using the twenty sign word dataset,five sign word dataset and the RTS version of these datasets.We achieved 99.30%accuracy from the twenty sign word dataset evaluation,99.10%accuracy from the RTS version of the twenty sign word evolution,100%accuracy from thefive sign word dataset evaluation,and 98.00%accuracy from the RTS versionfive sign word dataset evolution.Furthermore,the influence of our model exists in competitive results with state-of-the-art methods in sign word recognition.展开更多
In this study,a new algorithm was proposed for edge extraction of greenhouse strawberry leaf in natural light based on the 4-level daubechies 5(‘db5’)wavelet decomposition.This algorithm adopts different segmentatio...In this study,a new algorithm was proposed for edge extraction of greenhouse strawberry leaf in natural light based on the 4-level daubechies 5(‘db5’)wavelet decomposition.This algorithm adopts different segmentation methods for the reconstructed images at different scales to erase the external background and the internal leaf vein interference.There were two advantages of this method.One was that it can provide the abstraction from different spaces to express a same image.The other one was that some image features are hard to be acquired in some scale spaces,while the features are easy to be obtained in other scale spaces.In this image process methods,the Otsu threshold segmentation was to obtain the binary image areas,and the Canny segmentation is to obtain the accurate gradient edges,then the morphological methods and the logical calculus methods were to avoid the fragments inside the leaf area and the adhesions outside the leaf area.Since the strawberry leaf images were different respectively,and the greenhouse optical radiation and reflection may cause local non-uniform illumination of leaf image,the pseudo canny edges of leaf image ere divided into three categories in this research.The first category was the external pseudo canny edges area of the first layer reconstructed leaf image,the second category was the internal pseudo canny edges area in highlight of the third layer reconstructed leaf image,the third category was the internal pseudo canny edges area of significantly different grayscale of the third layer reconstructed leaf image.The different processing methods were constructed for the three kinds of different texture features based on the multi scale reconstructed images,then the complete and the accurate leaf edges without interference were obtained.Finally,the multi scale method was simplified and a remarkably effective segmentation algorithm was deduced for the greenhouse strawberry leaf in natural light.展开更多
基金This work was supported by the Competitive Research Fund of The University of Aizu,Japan.
文摘Communication between people with disabilities and people who do not understand sign language is a growing social need and can be a tedious task.One of the main functions of sign language is to communicate with each other through hand gestures.Recognition of hand gestures has become an important challenge for the recognition of sign language.There are many existing models that can produce a good accuracy,but if the model test with rotated or translated images,they may face some difficulties to make good performance accuracy.To resolve these challenges of hand gesture recognition,we proposed a Rotation,Translation and Scale-invariant sign word recognition system using a convolu-tional neural network(CNN).We have followed three steps in our work:rotated,translated and scaled(RTS)version dataset generation,gesture segmentation,and sign word classification.Firstly,we have enlarged a benchmark dataset of 20 sign words by making different amounts of Rotation,Translation and Scale of the ori-ginal images to create the RTS version dataset.Then we have applied the gesture segmentation technique.The segmentation consists of three levels,i)Otsu Thresholding with YCbCr,ii)Morphological analysis:dilation through opening morphology and iii)Watershed algorithm.Finally,our designed CNN model has been trained to classify the hand gesture as well as the sign word.Our model has been evaluated using the twenty sign word dataset,five sign word dataset and the RTS version of these datasets.We achieved 99.30%accuracy from the twenty sign word dataset evaluation,99.10%accuracy from the RTS version of the twenty sign word evolution,100%accuracy from thefive sign word dataset evaluation,and 98.00%accuracy from the RTS versionfive sign word dataset evolution.Furthermore,the influence of our model exists in competitive results with state-of-the-art methods in sign word recognition.
基金This work was supported by the Beijing‘Urban agriculture project group’program and was undertaken by China Agricultural University.
文摘In this study,a new algorithm was proposed for edge extraction of greenhouse strawberry leaf in natural light based on the 4-level daubechies 5(‘db5’)wavelet decomposition.This algorithm adopts different segmentation methods for the reconstructed images at different scales to erase the external background and the internal leaf vein interference.There were two advantages of this method.One was that it can provide the abstraction from different spaces to express a same image.The other one was that some image features are hard to be acquired in some scale spaces,while the features are easy to be obtained in other scale spaces.In this image process methods,the Otsu threshold segmentation was to obtain the binary image areas,and the Canny segmentation is to obtain the accurate gradient edges,then the morphological methods and the logical calculus methods were to avoid the fragments inside the leaf area and the adhesions outside the leaf area.Since the strawberry leaf images were different respectively,and the greenhouse optical radiation and reflection may cause local non-uniform illumination of leaf image,the pseudo canny edges of leaf image ere divided into three categories in this research.The first category was the external pseudo canny edges area of the first layer reconstructed leaf image,the second category was the internal pseudo canny edges area in highlight of the third layer reconstructed leaf image,the third category was the internal pseudo canny edges area of significantly different grayscale of the third layer reconstructed leaf image.The different processing methods were constructed for the three kinds of different texture features based on the multi scale reconstructed images,then the complete and the accurate leaf edges without interference were obtained.Finally,the multi scale method was simplified and a remarkably effective segmentation algorithm was deduced for the greenhouse strawberry leaf in natural light.
