School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology Bachelor of Science with Honours in Computer Science |
| Supervisor | Dr. Alin Liu |
| Areas | Intelligent Applications |
| Year of Completion | 2023 |
Garbage classification is essential for Hong Kong as a cosmopolitan city to achieve environmentally sound waste treatment and sustainable resource use. Proper classification and management can maximize recycling and reuse, reduce disposal costs, and minimize environmental pollution. Improper segregation can lead to missed opportunities for recycling and contamination of bins. Despite efforts to promote waste separation, productivity has been moderate.
Weak awareness and knowledge, high implementation costs, and labor cost problems hinder waste separation in Hong Kong. Traditional sorting methods are time-consuming and inefficient. IT technology can improve efficiency and awareness, but existing apps have low accuracy rates and cannot dynamically update their database. Few apps cater to mixed piles of garbage.
To improve waste separation in Hong Kong, an application called Green Star will be developed to detect four waste categories and allow user-uploaded images and categories. The model’s accuracy will improve as the database is updated based on user images. Multiple models will be provided for different scenarios, making Green Star superior to existing apps.
Developing an application for waste sorting and recycling can have a significant positive impact on society and the environment. The app can educate users, reduce pollution, conserve natural resources, and minimize landfill waste. By providing a user-friendly tool, the app supports and complements environmental initiatives, making it easier for individuals to contribute to a greener
future. The project’s value proposition is to help people sort waste quickly through AI and support environmentalists in waste sorting actions.
Demonstration Video
Presentation Video
The YOLOv7 model structure has three parts: the Backbone Network, Bottleneck Network, and Head network. It was selected for the AI module in garbage disposal due to its fast and efficient detection and classification methods. YOLOv7’s speed and accuracy in the range of 5 FPS to 160 FPS exceeded all known object detectors, making it the most advanced target detection method.
The YOLOv7 model scaling function simplified the training task by scaling the model according to the user device’s performance. The re-parameterization module reduced the training cost by avoiding the need to train another model for difficult-to-detect objects. YOLOv7’s GIoU loss function considered location, size, shape, and other factors for better detection results. Stochastic gradient descent (SGD) was used as the optimizer, with different weight values set for different layers during the training process to achieve more fine-grained parameter tuning.
Architecture Design of Green Star
The Green Star application architecture had three parts: client, model, and server. The latest model training results were updated on the server side, and the server responded to client requests with JSON data. Uploaded XML and JPG files were saved in the MySQL database. Organized datasets were used to further train the models for higher accuracy and user involvement in model training.
Client Design
The client design had three functions: ‘RECOGNITION’, ‘UPLOAD’, and ‘MODEL DETAIL’. The ‘RECOGNITION’ function helped users identify the type of garbage, while the ‘UPLOAD’ function allowed users to upload pictures of garbage to expand the dataset and improve model accuracy. The ‘MODEL DETAIL’ function showed the current accuracy of each model and helped users choose which model to use for recognition.
Model Design
Object detection models were used for waste identification in our project. Three models were considered: YOLOv7-v1, YOLOv7-v2, and YOLOv7-tiny. YOLOv7-v2 had high accuracy but low FPS, YOLOv7-tiny had higher FPS but lower accuracy, while YOLOv7-v1 had both high accuracy and FPS, making it recommended for users in the recognition function.
Server Design
Our server framework was the Spring Framework, known for its simple coding, automatic configuration, and ease of deployment. The MVC software pattern was used to enhance server-side functionality. The server automatically saved client data in the MySQL database, and when the data volume was large enough, the data was extracted and organized into datasets for further model training.
Upon entering the application interface, the main page displayed two buttons: “USER GUIDE” and “GET START” (Figure 1, left). Clicking “USER GUIDE” provided detailed information on how to use the app, while clicking “GET START” led to the main menu page with three function buttons: “MODEL DETAIL”, “RECOGNITION”, and “UPLOAD” (Figure 1, middle). “MODEL DETAIL” displayed the accuracy rate of each model (Figure 1, right), while “RECOGNITION” used real-time detection to identify garbage type (Figure 2, left). “UPLOAD” allowed users to upload images and label the type of garbage to enlarge the dataset for model training (Figure 1, middle and right).
Figure 1
The image dataset for our project was sourced from TrashNet and Huawei databases, divided into four categories and 21 sub-categories, with over 600 pieces of data per sub-category. The server framework used was the Spring Framework, which provided comprehensive infrastructure support for server development, including object dependency control, transaction management, and testing. The server architecture consisted of four layers: data, logic, service, and transaction. MySQL was chosen as the database for its speed, robustness, and ease of use. The database had three tables: “imgFile” and “xmlFile” for storing user-uploaded data, and “modelDetail” for storing model training results. The “imgFile” table stored uploaded pictures, while the “xmlFile” table stored the coordinates of the label box and the types of garbage selected by users. The “modelDetail” table stored the results of our training model, saved as JSON files and updated to the database via the POST method. When the user needed to check the model detail, the server extracted the data from the “modelDetail” table and returned it to the user. Overall, our project used a combination of image dataset and server architecture to provide accurate garbage identification and improve model accuracy through user contributions.
The client-side of our project used the “Canvas” class to implement the function of user labeling for uploaded pictures. The idea was to use the entire uploaded image as a canvas, allowing the user to draw a box around the garbage using a paintbrush, and then importing the coordinate data of the box into the database for image data processing. A new thread was created when a user uploaded data or requested model detail from the server. The “NDK” tool was used for native development to control the underlying hardware layer of the device, allowing us to implement the “RECOGNITION” function. As Java could not interact with the underlying hardware layer, C/C++ code was used in our Android application. The “UploadRepository” class was created for the upload function to request data from the server, and the “request()” method was used to upload the file to the server. Overall, our client-side implementation used a combination of “Canvas” class, thread creation, and NDK tool to provide accurate garbage identification and user labeling for uploaded pictures.
Figure 2
This evaluation report compared two YOLO models, YOLOv7-tiny and YOLOv7-v2, trained using the Huawei garbage classification dataset. The YOLO algorithm was used for real-time target detection, and the models were evaluated based on runtime and accuracy measurements. YOLOv7-tiny had a faster runtime, processing an average of 12 frames per second, while YOLOv7-v2 could only process 4 frames. However, YOLOv7-v2 outperformed YOLOv7-tiny in terms of precision and recall, achieving a mean accuracy (mAP) of 0.543, while YOLOv7-tiny achieved only an mAP of 0.361. The value of GFLOPS for YOLOv7-v2 was higher than that of YOLOv7-tiny, indicating better performance. The choice of model depended on the specific requirements of the project, and balancing accuracy and performance was an important consideration. If real-time performance was critical, then YOLOv7-tiny may have been the better choice, but if accuracy was the primary consideration, then YOLOv7-v2 would have been the preferred model. Overall, the evaluation showed that both YOLOv7-tiny and YOLOv7-v2 had their advantages and disadvantages, and the performance and efficiency of models directly affected the reliability and user experience of the system.
The user experience evaluation report included feedback from four participants who tested the Green Star app for garbage sorting and detection. Participants found the app to be convenient, practical, and easy to use. The interface design was praised for being simple and clear, and the app’s unique feature of allowing users to upload garbage images and tag them to provide data sets for the model was appealing. The accuracy and computation speed of the YOLOv7-v1 model were also praised. However, some participants noted that the accuracy of the identified garbage was not always very accurate, and the image editing function could be optimized further. The upload function for participating in model training was found to be interesting, but some details may have needed to be adjusted to make it more user-friendly. Overall, participants had a good user experience with the app, but some suggested improvements such as adding knowledge of garbage sorting and making the app available on different systems.
The project successfully achieved its objectives of developing three garbage classification models using the YOLO algorithm and implementing them in an app to help users recognize and sort garbage. The models achieved a certain level of accuracy, and the app allowed users to upload garbage images to the model database to improve accuracy. The accuracy and performance of the models were evaluated, and the basic garbage detection and recognition were found to be in an ideal state, detecting garbage in real-time and labeling it with its name and category. The opinions of four test users were collected, and while the implementation of functions was relatively complete, there were some shortcomings in the user interaction part that needed to be upgraded and adjusted in the future.
The project used Python, Java, C++, HTML, SQL, and other languages to develop the model, server, and client. The YOLO algorithm was learned, improved, and trained on labeled data to export different model results. Alibaba Cloud was used as the platform for server development, and the client-side development mainly used Java to write the interface and functions. The challenge of user involvement in labeling was addressed by using Canvas and Paint to set the image as a canvas. The challenge of importing model details into the app was addressed by using HTML combined with JSON to present the details of the three models to the user after continuous debugging. Overall, the project successfully developed a garbage classification app with three models and user involvement in improving accuracy.
While the app received positive feedback from test users, there were still some limitations that needed to be addressed. The current model was still in a low state of accuracy, and more data was needed for training. The design and artwork of the interface needed to be unified, and there were some unclear aspects of user interaction that needed to be improved. Future work included building a larger dataset from user-uploaded data to improve the accuracy of the models and collaborating with industry and government to utilize the trained models. The ultimate goal was to reduce incorrect garbage disposal and raise awareness of the impact of garbage on the environment. If the project could reduce incorrect garbage disposal by 1%, it was equivalent to reducing air pollutants in factories by half a year, demonstrating the potential of machine learning for addressing environmental and lifestyle issues.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
