School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology |
| Supervisor | Prof. Vanessa NG |
| Areas | Intelligent Applications |
| Year of Completion | 2019 |
One of the crucial difficulties that visually impaired people face in daily life is recognizing the value of banknotes. They cannot identify the value of banknotes by eyes but can sometimes identify the value by hands touching the accessibility features on banknotes. However, as time goes by, the accessibility features on banknotes will get damaged and become unreliable. Identifying values on banknotes becomes one of the biggest challenges of them.
The aim of this project is to develop a mobile application to help visually impaired people recognize the value of current Hong Kong banknotes. The application should (1) show the value of banknotes with (2) speech synthesis and (3) cell phone vibration coupled with (4) machine learning.
To achieve the aim, the main objective of the project is to develop a mobile application that can recognize the 2003 series and 2010 series Hong Kong banknotes which are the highest circulation banknotes. The project has also defined a number of sub-objectives as follows:
System architecture
Technique used for training the neural network for predicting more accurate results in banknote prediction
To enhance the accuracy of the neural network in recognizing Hong Kong banknotes, data sampling, data augmentation, and the technique of comparing different models have been applied.
Data sampling
In order for neural network model to extract informative features and learn the patterns of banknotes, high-quality training dataset are required. The training dataset must contain variant forms of banknotes' images; therefore, discrimination, i.e. can only recognize under certain distance or brightness, will not be occurred in the neural network.
Data augmentation
To avoid overfitting, which is a common problem in training machine learning model, data augmentation has been applied. Data augmentation is a powerful way for image classification with a limited number of data to generate more data from the existed dataset.
Figure 2: Data augmentation
Selecting a model with the best validation accuracy
To get a more accurate result, we had compared different models and eventually chose the best model with high validation accuracy.
The process will begin with a captured image. Then the image starts the pre-processing and feature extraction.
The image will first be passed through to the binary classifier, which is used for determining whether an image is a Hong Kong banknote. If the image is not a Hong Kong banknote, the application will send the non-banknote object feedback. Otherwise, the image will be passed through the multi-class classifier.
When the multi-class classifier has a result, the application will check the confidence of the result. If the confidence of the result is not high enough, the result will be ignored, and the system will require the user to try again. Otherwise, the result will be shown in the application with verbal feedback and vibration.
Figure 3: Flow chart of processing and analyzing images
Figure 4: Recognising the front side of HKD$100 banknote
Figure 5: Recognising the rear side of HKD$100 banknote
Figure 6: Recognising the font side of HKD$20 banknote
Figure 7: Recognising the rear side of HKD$20 banknote
| Time | |
| Test 1 | 41ms |
| Test 2 | 77ms |
| Test 3 | 74ms |
| Test 4 | 91ms |
| Min | 41ms |
| Max | 91ms |
| Avg | 70.75ms |
Table 1: Speed of test 1-4
4 recognition tests have been performed. All of them finished under 100ms. The minimum time was 41ms and the maximum time was 91ms. The average time was 70.75ms.
Compared to recognition of banknotes by hands, the time has been improved by 60000%, from 1 minute to 100ms. In contrast with the recognition by Note-measuring Template, the time has been improved by 30000%, from 30 seconds to 100ms.
The aim of this project has mostly been satisfied. The prototype mobile application can show the value of banknotes with speech synthesis and cell phone vibration coupled with machine learning. As we collected a wide variety of banknotes' patterns images for machine learning, our trained prototype's model has returned correct values for banknote pictures, and the feedback system has worked well. The prototypical mobile application has been created with high accuracy and high speed.
In the future, we will improve our work by the development of identifying other countries' banknotes. We can adapt the model used in the prototype and change the datasets with other countries' banknotes so that we can help more visually impaired people from other countries to recognize their banknotes.
