School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Science with Honours in Computing |
| Supervisor | Dr. TS Li |
| Areas | Innovative Mobile Applications |
| Year of Completion | 2014 |
The main objective is to develop a mobile application to recognize the outdoor flowers, and return flower information to the users automatically. The sub-objectives are:
There are wide range types of outdoor flower in Hong Kong, the mobile application is going to recognize the flowers in which people are interested in it. Also, the flower object should be clear enough so that the camera can capture it. Therefore, the flowers with obvious petals and attractive appearance become the recognition target. The herbaceous plant which has small petals and scattered appearance will not be recognized in the application. An example is shown below:
Flower object is not symmetric in three dimensional systems. The appearance of the flower is not same if users capture it in different views. It is not possible to recognize a flower object in different views because the scale of the database will be too large so that recognition accuracy is reduced. Therefore, the application is designed under the following two conditions:
(1) There should be at least one isolated flower exist in the image. In this case, isolated flower means that the object must not overlap with other flower.
(2) The front side of the flower should be captured. All petals should be captured clearly.
Instead of using flower boundary, I use the color of the flower to detect the flower region in the image. Once the flower region is found, the common edge detection method can be applied on the flower region to work on the next recognition processing.
The concept of finding flower color is to count the times of the color represented by every pixel in the image, maximum counts of the color will be considered as flower color. In the below image, assume that at least one flower object exist in the image, and the maximum color value counted from the pixels is light blue. Therefore the color “light blue” will be defined as the flower color.
The noise in the image is the regions which are out of focus or being darkness. These regions have low Saturation and Value in HSV model. The value of the darkness pixels is usually lower than 0.25 (between 0-1), and the saturation of the dim pixels is lower than 0.2 (between 0-1). The pixels under this range will be considered as noise and its hue will not be counted. The pixels having enough lighting and saturation are more likely belong to the flower object region. Refer to the above example, it is expected that the shadow region will not be counted after modification has been made.
The concept is to annoy all green Hue pixels in the image. Every green pixel is considered as the leaf region, it will not be counted as the flower color. It means the application could not able to handle the flower with green color. According to the Hong Kong 100 common outdoor flowers category published by PRODUCE GREEN FOUNDATION, the percentage of the green flower is smaller than 5%. The number is acceptable because the application is not assumed to cover every flower at all.
The petals number of the flower worked out due to the flower boundary. In the flower boundary, there are large changes of the direction between each petal. The number of these changes is equal to the petals number. Therefore, the petals number can be conducted by counting the changing positions number. In fact, the boundary is represented by a set of consequence points. There is a unique changing pattern of the turning point between petals. The distance between the turning point and the centre should be the smallest between petals. Also, the distance between the near points and centre increase gradually. By checking every boundary point is matching this pattern or not, the turning points between petals can be conducted.
Changing pattern: (d0 >d1) and (d1>d2) and (d2<d3) and (d3<d4) The changing pattern will be applied on each boundary point; d2 is the examined distance each time. It should be noticed that I have reduced 70% of the points in the boundary. It reduces the density of the boundary points. If the points are too dense, the different between d0, d1, d2 and d3 is not significant and the turning point may not be found. Also, to avoid wrongly define the boundary defection as a turning point, I use two more distance d5 and d6 to ensure the changes between the points are strong enough.
In summary, I have already explained the methods to distinguish flower and extract its variables, the illustration of the operations is shown below.
The system is developed for Android platform so that the prototype will be implemented by the Java. OpenCV (Open Source Computer Vision Library) is an open source library to handle image processing. OpenCV library provide a lot of basic image processing algorithm to achieve the low level problems. Developers can implement the specific functions base on the algorithm provided by OpenCV. Weka is the tool mentioned before which is providing Learning Machine algorithms API to handle the flower clarification task.
The illustrations below show the operation of the mobile application:
To measure the effectiveness of the application, the way is to look on the recognition accuracy. I choose 20 flower types with a total of 311 images to evaluate the recognition accuracy of the system. To make the evaluation become fair, there are some flowers with similar color, shape and petals numbers so that the recognition task will not become too simple. The 20 types of flower are shown below:
In the test, 196 out of 311 instances are classified correctly. Which means the recognition accuracy is around 63%. The recognition result of the Cross-validation test worked out by Tree-J48 classifier on 311 instances is shown below:
After looking on the general recognition accuracy of all the instances, I look into the recognition performance of each category. I would like to investigate which flower types are not likely to be recognized and find out the cause of the problem. The figure below shows the recognition details of each class:
The TP Rate (True Positive Rate) in the figure shows the recognition accuracy of each flower class. The accuracy of the classes between 6 and 10 is significantly lower than the average accuracy 63%. The flower types of these five classes are shown below:
The aim of this project is achieved. The mobile application is possible to help the users getting the information of the outdoor flowers in real time. The key to success is considering the application function practically. It is found that there is a wide range of flowers grow in the outside, and there are large variations on the flower object. The flower recognition functions may not be able to handle the recognition tasks properly.
The smart idea to solve the problem is that reduces the difficulties of the recognition tasks, and increase the possibility of returning correct result to user. To reduce the difficulties, a customized camera function is suggested in the objective. The camera can capture the flower objects only so that the recognition problems become more simple and solvable. Also, the multiple results displayer contributes a lot on returning correct result to the user without consuming user so much times.
The future work on the application is to modify the variables extraction method of the flowers. The short term goal is to introduce a new flower variable to increase the combinations of the variables so that the similar flower can also be identified. It is suggested that the new variable should be a template of flower petal. The first reason to use petal template as a variable is that template matching give out more accurate comparison result rather than using numeric. The second reason to do so is that the petal templates can be found easily by following petal counting process.
Copyright Au Chung Ning and TS Li 2014
