School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology |
| Supervisor | Prof. Vanessa Ng |
| Areas | Intelligent Applications |
| Year of Completion | 2021 |
Dementia patients will use cameras to capture real-world images and transmit them to object recognition and facial recognition functions. Object recognition will retrieve photos from the database to see whether it has been matched and return the result. While the facial recognition function will store facial images into the user database. The memoir function will use the facial recognition function to detect faces inputted by both caretakers and patients and display them to the patients with a grid layout. The location of the dementia patients will send to their relatives through Google Maps API.
Figure 1: Overall system design diagram
Data Flow Diagram:
The database will send photos and names to the game. After users finish the game, they will go to the result screen and show the result. In the meantime, the score will be stored in the database.
Figure 2: Data Flow Diagram of Object recognition
Raspberry Pi will be used as an input device to send video capture to the machine learning server for recognition. At the same time, the database will send all belongings photos to the server in order to match the capture. After successfully matching, the server will send the latest screenshot to the database. Once the user requires the latest position of his/her belonging, the application will retrieve the result from the database through the application.
Figure 3: Data Flow Diagram of facial recognition
Raspberry Pi will be used as an input device to send video capture to the machine learning server for recognition. At the same time, the database will send all previously matched faces with names to the server in order to match the new one. After successfully matching, the server will send the screenshot with the name of the person identified to the database. Once the user requires the name and relationship of someone, the application will retrieve the result from the database through the application.
Figures 4: Data Flow Diagram of Memoir
When dementia or caretaker user uploads photos to the application. The photos will be sent to the Machine Learning Server. Then the server will request all previously matched faces with names from the database. After matching faces and adding name tags to the photos, the server will update the memoir in the database. Once the Dementia user requires the memoir, the database will send the photos to the application.
Figures 5: Data flow diagram of GPS sharing
Google API will be used to detect the location of dementia patients and send it to their relatives.
Correctness Analysis
In order to test the accuracy and the rate of incorrect detection of facial recognition and object recognition, we have used 40 testing data for facial recognition functions and 30 testing data for the object recognition function to test the functions.
In the test of facial recognition, we have used 4 photos which the people in the photos who are not in the trained data set, and 35 normal photos. The results of detecting the unknown faces are 80%. The program has successfully detected the unknown faces in the 4 photos. Those photos will not show in the application, but they will be saved in the database. There is 1 photo with no tag which means no face in that photo has been detected. The results of detecting the known face (Trained already) are 82.8%. In those 35 photos, the program has returned correct results which means a photo with a correct name tag and relationship has appeared in the application. After testing, we realized that the people cannot be recognized by the program if they are not facing the camera since the program had returned no faces detected in 3 photos. Also, 3 photos returned the wrong result. Those faces mismatched the name and relationship.
In the test of object recognition, we used 10 photos that did not contain glasses, keys, and wallets and 20 photos that had different positions and groupings of glasses, keys, and wallets. In those 10 photos, there 6 photos identified no glasses, keys, and wallets but 4 photos show either glasses or keys. In recognizing the unknown belonging, the accuracy is 60%. For the other 20 photos, 14 photos successfully detected glasses, keys, or wallets. We discovered that the belonging if were overlapped would fail to detect. Also if the belongings are too far from the camera, the program could not detect the belongings. In recognizing the known belonging, the accuracy is 70%
In conclusion, the accuracy of facial cognition is around 80% – 83.3%. The accuracy of object cognition is around 60% -70%.
To sum up, the implementation of EYEDENTIFY is fruitful. This project has successfully implemented a mobile application which concentrates on the needs of Dementia patients.
The implementation of face recognition and memoir functions has been done smoothly by using Face recognition (DLIB). In the face recognition functions, a real-time face recognition system is realized, the user identifies their family and friends. By using the Raspberry Pi with the camera module, the photo will pass the Face recognition (DLIB) program. After face detection and matching, the results will be shown in the application. Dementia users can remember the identity of the people they are meeting. In the memoir function, both dementia users and caretaker users can upload photos. After matching faces by Face recognition (DLIB), dementia users can see the photo with the name tag of their family and friends and share the same memories with them.
The implementation of the object recognition function has been completed with Tensorflow. A function like lost and found has been created for Dementia users to find their personal belongings e.g. glasses. As Dementia users lose their belonging, they can find the last seen photo of that belonging and locate it. The last seen photo was taken by Raspberry Pi and processed by Tensorflow. Since Dementia users can find their belongings by themselves, fewer controversies with family and friends will happen.
The implementation of the GPS sharing function has been accomplished favorably with Google Map API. Both Dementia users and Caretaker users can see each other’s last seen location which brings an equal relationship.
To ensure the application remains competitive and stands the test of time, further development should be done. Both hardware and software can be upgraded to provide a better service.
In hardware development, Raspberry Pi is the core hardware of the whole system. First, the physical size of Raspberry Pi should be reduced to increase portability. The Raspberry Pi must operate with a power bank now. Both items should be combined to reduce the volume. Second, the camera in Raspberry Pi should be upgraded in order to increase the resolution of the photos. A higher resolution can increase the accuracy of the face and object recognition.
In software development, several developments should be completed in the future. First, the existing functions should be enhanced. For the object recognition functions, more items of target objects should be added. There are only four items that can be detected in the current function. Users should be able to add any belonging that they need to find to the function in the future. For the GPS sharing function, a direction feature should be built to show the fastest way to go to the current location of each user. Second, some new features should be created to boost the comprehensiveness of the mobile application. An instant messengers function can be added into the application to close the relationship as users can talk to each other at anytime and anywhere. A ‘GO-HOME' function can be implemented into the application. As dementia patients will lose their way easily, a function shows the fastest way to go back home and notify their caretaker to ensure their safety.
To ensure the application remains competitive and stands the test of time, further development should be done. Both hardware and software can be upgraded to provide a better service.
In hardware development, Raspberry Pi is the core hardware of the whole system. First, the physical size of Raspberry Pi should be reduced to increase portability. The Raspberry Pi must operate with a power bank now. Both items should be combined to reduce the volume. Second, the camera in Raspberry Pi should be upgraded in order to increase the resolution of the photos. A higher resolution can increase the accuracy of the face and object recognition.
In software development, several developments should be completed in the future. First, the existing functions should be enhanced. For the object recognition functions, more items of target objects should be added. There are only four items that can be detected in the current function. Users should be able to add any belonging that they need to find to the function in the future. For the GPS sharing function, a direction feature should be built to show the fastest way to go to the current location of each user. Second, some new features should be created to boost the comprehensiveness of the mobile application. An instant messengers function can be added into the application to close the relationship as users can talk to each other at anytime and anywhere. A ‘GO-HOME' function can be implemented into the application. As dementia patients will lose their way easily, a function shows the fastest way to go back home and notify their caretaker to ensure their safety.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
