School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology |
| Supervisor | Mr. Tommy Chok-Pang Kwok |
| Areas | Intelligent Applications |
| Year of Completion | 2022 |
Project Aim
This project aims at developing an application which can formulate the real-time utilization rate of various washrooms in a shopping mall. After the system analysis which includes considering the potential users' genders and their current floors, the application will suggest the best choice of washroom for them.
The project also helps the management of the shopping malls to relieve the pressure on the usage of some specific washrooms. In a large shopping mall, different levels may have different visitor density. For example, the floor which connects to the MTR station will usually have the highest crowd and the washrooms on that floor will be more crowded than the others. The project can assist to ease the flow of visitors by suggesting them to use the washrooms on the other levels.
Project Objectives
Investigate the frequency of washrooms on different floors of the public places.
Determine the applicable population and main purpose of the application to be developed.
Design a complete model based on the application scenario.
Design the algorithm based on rational allocation of available toilet grids and preliminary application framework.
Develop mobile applications for helping users quickly find an available washroom grid and reduce the pressure of specific washrooms.
Evaluate the usability of the application and the accuracy of the availability of washroom grid continuously.
Demonstration Video
Presentation Video
Object detection
the technology for detecting the object which can detect the person's movement in the digital videos or images. Besides, some APIs are already in the market, such as Face API which is provided by Microsoft Azure. Liu OC.A. et al.(2020), claimed that object detection will detect the objects based on the captured images from the camera. So, the aim of object detection in this project is to detect the people who are in the queue to count noses via the camera. Zhong-Qiu Zhao (2019) described that object detection aims to locate and classify existing objects. It seems that object detection analyzes the type of object such as: human or other things. Victor Lempitsky (2010) points out that the counting algorithm needs to count the real-valued functions of pixel grids which can count the actual amount in the images. When the application goes to object detection for counting the total number of people who queued at this moment, the programme can calculate the total waiting time for the user who has just arrived and wants to queue via the real-time image.
Figure 1 shows the demo of object detection.
Figure 1: Demo of object detection
AI technology(Artificial Intelligence)
the technology to simulate some human thinking which can be learned how to solve problems. In Lee stated that the devices and machines can conduct the data exchanges with the global network supported by the Internet of Things (IoByWith the Integrate artificial intelligence, IoT and data, the project can provide some advice for the user to go to another washroom or just wait in the same queue. According to Dong et al. (2020) showed that detecting faces through video analysis can be supported by advanced AI technology.
Figure 2 shows the demo of AI technology.
Figure 2: Demo of using AI technology
Evaluation board
a piece of hardware based on Python which can support some sensors to finish object detection. Shah, P., & Vyas, T. (2014). provided with the serial communication of the evaluation board which the signal will be sent out so can detect the object within the camera. The evaluation board (PyAI-K210) developed by 01 Studio can detect the person in a specific area. It can count the total number of people in the specific area within the washroom’s waiting area. Also, PyAI-K210 can cooperate with the camera component to achieve object tracking and video analysis. Moreover, to deal with the privacy issues with using the camera outside the washroom, according to the “Guidance on CCTV Surveillance and Use of Drones” from the Office of the Privacy Commissioner for Personal Data, Hong Kong said that if CCTV is used to monitor traffic flow or crowd movement, generally no need to record carefully the appearance. In other words, the CCTV of this project needs to set a lower resolution to deal with the privacy issues. Figure 3 shows the demo of the Evaluation board.
Based on those supporting technologies, the project can describe the user stories through the use-case diagram (Fig. 4). As a user, I want to know the information about the nearby washroom so that I can go to the washroom immediately.
Figure 3: Demo of using Evaluation board
Figure 4: Use-case Diagram
Functional and non-Functional Requirements
| Non-functional requirements | |
| Waiting time | Washroom occupied average time for each people |
| Maintainability | Target user |
| Physical location | Capacity, Growth and Scalability |
| Functional requirements | |
| Object detection | AI technology（Artificial Intelligence） |
| Evaluation board (PyAI-K210) | Algorithm |
| Camera component | Android application |
| MQTT | Wi-Fi microchip (ESP-8266) |
Key Supporting Technology
Given the project aim and objectives, our group has developed some methods to shorten the queuing time by providing real-time information to the visitors of the large shopping malls. This project will be divided into three components to accomplish the goal.
1. Algorithm
The algorithm was written in Python programming which evaluates the total queuing time for the newcomers based on the age and the gender of the users in front of them. Furthermore, the programme will consider the real-time locations (i.e. the x and y coordinates) of the users so as to provide the optimal suggestions on whether they should use another washroom to save time.
2. Object detection
An evaluation board with a camera, PyAI-K210 (Fig. 9) will be installed above the washroom door (Fig. 10) which collects real-time information about the total number of people who are queueing. The data collected is significant for the algorithm to ensure the accuracy of estimating the waiting time. On the other hand, object detection can also distinguish between persons and other matters/species such as dogs and cats. Only when the detector has detected that the object is a person (rather than the other matters) will it increase the total number of counts by one
Figure 5: Evaluation Board with a camera (PyAI-K210)
Figure 6: Location of Detector
3. Mobile application
The mobile application was designed by Android Studio. When comparing the algorithm and object detection, the mobile application will be the most user-friendly UI interface for the user. The mobile apps will provide some suggestions for the user which show the queuing time for washrooms at different levels. Given the updated information, users can make their choices to go to the other washroom or just wait in the same queue. Figure 7 shows the preliminarily UI design for the user to choose their age, floor and gender.
Figure 7: The UI design for mobile application
For the mobile application to become more user-friendly and provide more functions, such as providing three different languages which are English, Traditional Chinese, and Simplified Chinese. Those functions are very handy for people who are using “Washroom Easy”.
Another function of the application is to check network status via a mobile application that can identify the location of the user automatically. The application will check the Basic Service Set Identifier, called BSSID. BSSID is the MAC physical address of the access point or router. The mobile application also facilitates the checking of the network status and whether the user's mobile device has connected to the shopping mall’s Wi-Fi or not. Also, each BSSID will pair up with the corresponding floor. It means that the one BSSID belongs to one floor.
4. System Design to Bridge the Gap
According to the Component Diagram (Fig. 8), the component of the project includes the analytics and the queueing which can analyze the usage of washrooms on different floors. The Camera Processor and AI technology component are important for evaluating the usability of the application and the accuracy of the availability of washroom grid continuously. These components can also help to design a complete model. Lastly, the algorithm is designed based on the rational allocation of available toilet grids and preliminary application framework.
Figure 8: Component Diagram
According to the Data-flow Diagram (Fig. 9), the first data is to collect the newest waiting time for checking the arrival time of the latest people on the queuing line. The second data is to collect the queuing status to calculate the total number of queuing people for building the queue list. The third data is to collect the shopping mall floor plan for viewing the floors list and checking the structure of the shopping mall to find out the details of washrooms' physical location. They can help with building a suitable path for users to find the other occupied washrooms. Therefore, the user can know the usage of washrooms.
Figure 9: Data-flow Diagram
For the algorithm part, a class diagram can help illustrate how it works. When processing the waiting time for the washroom, the algorithm calculation will consider the location of the user (i.e. the x and y coordinates of the user). Figure 10 below shows the details.
Figure 10: Class Diagram for algorithm operation
For the object detection part, a while loop flowchart can facilitate the explanation. When the detector via camera has detected an object, the programme will distinguish if the object is a person or not. If true, that means it is a person, so the counter of the programme will add 1 to the total number of queueing people and the waiting time will be extended. If the object is not a person, the counter will not add 1 and miss the object in front of the camera. Figure 11 below shows the process of object detection specifically.
Figure 11: Flowchart
Evaluation Methods and Design
For the user evaluation part, it will separate two parts. The first part(user evaluation #1) is mainly testing the algorithm and detector. For each criteria, the full mark is 10 marks for each user which the user can give 0 mark as the lowest score. The questionnaire for user evaluation #1 shown in Table 3.
| Tasks | score (The full mark for each user is 10 marks) | Total score | |||||||||
| Analyze waiting time via algorithm | |||||||||||
| Distinguish the person object via the detector(PyAI-K210) with Camera | |||||||||||
Table 3. The Questionnaire for User Evaluation #1
In user evaluation #2, the survey is going to test the mobile application coordination ability for object detection and algorithm parts which mainly focus on mobile application testing. The questionnaire for user evaluation #2 shown in Table 4.
| Tasks | Score (The full mark for each user is 10 marks) | Total score | |||||||||
| Change different languages | |||||||||||
| Set the location via network status | |||||||||||
| Update the real time information via the application | |||||||||||
| Calculate the waiting time through an algorithm and the detector can show up in the application user interface correctly in the actual situation | |||||||||||
Table 4. The Questionnaire for User Evaluation #2
It is to be observed that the assessment criteria and results for user evaluation #1 and #2 will be discussed in Results-Evaluation Part
| Tasks | Technologies |
| Combine Python and Java within Android Studio | Chaquopy The Python SDK for Android |
| Detector detect the unseen object as person | Object detection KPU provided by PyAI-K210 and YOLO2 algorithm |
| Send the real-time data provide by the detector to the Android studio | MQTT |
| Display the real-time information | Android Studio |
Highlight of the Key Function
The Key Function of Washroom Easy is to display the situation of each floor's washroom. By comparing the approximately time of each washroom, Washroom Easy will provide the suggestion to the user. Figure 12 and 13 show the demonstration of each washroom.
Figure 12: Details data of Female Washroom
Figure 13: Details data of Male Washroom
When the user slides down this page, the data will be updated. Figure 14 and 15 show the updated data of each washroom.
Figure 14: Updated data of Female Washroom
Figure 15: Updated data of Male Washroom
To comfort the user, when the user clicks the box of the washroom, the map will appear and show the location of the specific washroom as a yellow star. Figure 16 and 17 display the map below.
Figure 16: Location of the L3 Female Washroom
Figure 17: Location of the L3 Male Washroom
The UI design provides three different languages which are English, Traditional Chinese, and Simplified Chinese. It seems that the application will become more convenient for different users with different languages. So, the user can search for the most useful information in their familiar language. Figure 18 displays the UI of select different languages below.
Figure 18: The UI design for different languages
Results – User Evaluation #1 (Algorithm and detector)
The user evaluation #1 mainly to check satisfaction of the tasks via user experience. The task is focused on an algorithm and the detector. This part recorded ten user’s satisfaction with specific tasks. Those tasks without the application. The results shown in Table 5.
| Tasks | score (The full mark for each user is 10 marks) | Total score | |||||||||
Analyze waiting time via algorithm | 10 | 8 | 9 | 9 | 10 | 9 | 10 | 10 | 9 | 10 | 94/100 |
| Distinguish the person object via the detector(PyAI-K210) with Camera | 9 | 9 | 9 | 10 | 9 | 9 | 8 | 9 | 8 | 10 | 90/100 |
Table 5. Results of User Evaluation #1
According to user evaluation #1, the algorithm will be tested by the user. In the first task, the user needs to use the suggestion provided by the algorithm to find the washroom and check whether the waiting time is within the suggested time or not. The user needs to give out the mark based on his/her experience. The assessment criteria are to find the washroom successfully and the accuracy of suggestion waiting time. The full mark is 10 points per user. Due to the project having ten users to finish the surveys, the total score for each task is 100 marks. Within task 1 called “Analyze waiting time via algorithm”, the total score got 94 marks out of 100 marks. In other words, 94% of users are satisfied with the algorithm which provides correct waiting time.
The second task in user evaluation #1 is called “Distinguish the person object via the detector(PyAI-K210) with Camera”. The assessment criteria is that the user can hold different objects over the camera with a detector that can distinguish which object is a person. In other words, this task is going to test the capability of distinguishing the unseen object which one is the person object correctly. The full mark is 10 points per user. Due to the project having ten users to finish the surveys, the total score for each task is 100 marks. The total score of the second task got 90 marks out of 100 marks. To put it in another way, 90% of users are satisfied with detecting the “person” object.
In conclusion, the performance of task 1 and task 2 within user evaluation #1 is ably performed satisfactorily which is the object detection and algorithm part.
Findings – User Evaluation #1 and #2
To sum up with user evaluations #1 and #2, the performance of the algorithm and detector is quite good, except for working with the application. It is because the algorithm and detector have some limitations. If the future work has some progress done well, the algorithm and detector can be more flexible and accurate. So, the “Washroom Easy” application will be more workable with both algorithm and detector. Then, the satisfaction of the user with calculating the waiting time in the actual situation will be increased. The limitation and future work of “washroom easy” will be discussed in Conclusion.
Results – User Evaluation #2 (Application)
Based on user evaluation #1, the object detection and algorithm parts were satisfactory for most of the users. In user evaluation #2, the survey is going to test the mobile application coordination ability for object detection and algorithm parts which mainly focus on mobile application testing. This part also invited 10 users to use this application to find the washroom. Research whether the detector(PyAI-K210) can cooperate with the mobile application appropriately in the actual situation or not. In user evaluation #2, there are four tasks for the user to test with the “Washroom Easy” application. The result of user evaluation #2 shown in Table 6.
| Tasks | Score (The full mark for each user is 10 marks) | Total score | |||||||||
| Change different languages | 10 | 10 | 10 | 10 | 10 | 9 | 10 | 10 | 10 | 9 | 98/100 |
| Set the location via network status | 8 | 8 | 7 | 10 | 8 | 9 | 7 | 8 | 8 | 9 | 82/100 |
| Update the real time information via the application | 7 | 8 | 9 | 8 | 9 | 10 | 8 | 8 | 7 | 10 | 84/100 |
| Calculate the waiting time through an algorithm and the detector can show up in the application user interface correctly in the actual situation | 7 | 7 | 9 | 9 | 8 | 8 | 10 | 7 | 7 | 8 | 80/100 |
Table 6. Results of User Evaluation #2
According to user evaluation #2, the user interface of a mobile application will be tested by the user. The first task is to change different languages in the user interface. The assessment criteria is that the user can change three languages he/she likes immediately. The application provides three languages which are English, Traditional Chinese, and Simplified Chinese. The full mark is 100 marks per task. Since this task got 98 out of 100 marks, 98% of users were satisfied with changing different languages in the user interface.
The second task of user evaluation #2 is called “Set the location via network status”, the assessment criteria is checking the user location whether and the location automatically via network status is consistent or not. 82% of users are satisfied with setting the location automatically via network status which means that the location can be set automatically as the user location.
The third task in user evaluation #2 is testing to update the real-time information via the application. The user needs to use the scroll down to update real-time information immediately which the user is able to test for accessibility of this function. Most of the users felt user friendly when he/she updated the real time information as soon as possible. Also, there were ten users to join this testing. Luckily, 82% of users were satisfied with setting the location automatically via network status.
The last task for user evaluation #2 is to calculate the waiting time through an algorithm and the detector can show up in the application user interface correctly in the actual situation. It is the main task of user evaluation #2 which is testing the whole system of the “Washroom Easy”. The testing is about whether the application can cooperate with the object detection and algorithm appropriately in the actual situation or not. However, when adding the application that cooperates with the algorithm and detector, 80% of users are satisfied with the algorithm, detector ,and application for calculating the waiting time in the actual situation.
Washroom Easy is designed to provide an optimal solution for a visitor in a shopping mall to use the appropriate washrooms. The application is composed of three main components, namely, the algorithm, the object detection and the mobile application.
The main purpose of the algorithm is to identify the most suitable washroom and to compute the necessary waiting time by considering a series of relevant factors such as the gender, the age, the location and the instant usage rate of the washrooms on different floors. On the other hand, the algorithm is also designed to take in account the user’s specific walking speed according to his gender and age before giving the final advice to the user.
The main purpose of object detection is to detect whether the unlabeled object is a human or not. The counter is set to focus on counting the total number of people only. When an unlabeled object is a person, the head count will be increased by one, which indicates that the total waiting time will be increased accordingly. The total number of people waiting at the queue will be reported to the mobile application through the MQTT system.
In the aspect of mobile application, the user interface will be provided by the application which is the UI design for users to input their ages, floors and genders. Also, the users are also allowed to change the languages, which include English, Traditional Chinese and Simplified Chinese, of the application. On the other hand, users can scroll down the window to update the real-time information immediately. A map which displays the locations of the washrooms is also available for users' easy reference.
To sum up, Washroom Easy is composed of the three main elements known as the algorithm, the object detection and the mobile application. First, an unlabeled object will be checked to determine whether it is a person or not. If the object is a human, the headcount will be added by one and the algorithm behind will estimate the total waiting time for the newcomer. After that, the result will be shown to the users through the mobile application.
According to the user evaluation #1 and #2, there is still room for improvement for the overall performance and some limitations of the application have been discovered.
With regard to the algorithm for analyzing the waiting time, the limitation is that it may only be specific to one shopping mall. A new algorithm may be required if the system is applied to another shopping mall which has a different internal building structure.
For object detection, the project team may consider to use more labeled datasets for training purposes in order to enhance the accuracy for human determination. If the object detection has a higher accuracy, the capability for distinguishing human beings from the other kinds of objects via the detector(PyAI-K210) will be improved.
Lastly, for the application part, Suining He (2015) described that using the similarity metric by deterministic algorithms can distinguish online signal measurement and fingerprint data. For improvement of this application, Wi-Fi fingerprint-based indoor positioning and the k-nearest neighbors algorithm (KNN algorithm) may be used to locate the user’s location by identifying his/her x and y-axis in grid coordinates. This location part will collect the Basic Service Set Identifier(BSSID) and WiFi Received Signal Strength Indicator (RSSI). The RSSI is the value of the RF signal. With stronger signals, the connection quality will be better.
To achieve the same, the WiFi fingerprint would be collected when the user is offline. The RSSI and BSSID should be collected by Wi-Fi fingerprint. When the user is online, the fingerprint will use a matching algorithm and the KNN algorithm would be used. Then, the x and y-axis would be found through those algorithms.
Last but not least, when the algorithm, object detection, and mobile application parts become more mature in the future, “Washroom Easy” will be more accurate. The project team has set out the limitations as well as suggested works in the future in Table 7 below.
| Tasks | Limitation | Future Work |
Algorithm: Analyze waiting time via algorithm | ● The algorithm can analyze on specific shopping mall only ● The algorithm can change the ways of calculation according to the age group of the user | ● The algorithm should flexible on every shopping mall area ● The algorithm should be modified for different ages instead of the age group. |
Object detection: Distinguish the person object via the detector(PyAI-K210) with Camera | ● Sometime, the detector will fail detect the object which the object will not defined as person | ● Use more labeled training data to build a model which can predict the unlabeled data more accurately |
Mobile application: Set the location via network status | ● BSSID can available for specific shopping mall only ● BSSID of the user can affect the other BSSID in the same location | ● Use the KNN with the BSSID and RSSI to set the location of the user to become more accurately ● Use machine learning to define the location for the BSSID and RSSI |
Algorithm+Object detection+Mobile application: Washroom Easy | ● Calculate the waiting time through algorithm and detector can show up in the user interface may occur some error | ● With the optimized algorithm and objection, the waiting time for waiting washroom will become more accurately |
Table 7. Details of Limitation and Future Work of Washroom Easy
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
