School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology |
| Supervisor | Dr. Keith Lee |
| Areas | Virtual Reality and Augmented Reality Applications |
| Year of Completion | 2021 |
The aim of the project is to help some primary students who may have poor emotional management by using the Virtual Reality System to simulate some emotional occasions and analyze their reaction and response in the test section.
Help poor emotional management students
Simulate Emotional Occasions
Test section
In addition, for emotional Occasions. We will provide a function for exception handling. If users fail to control their emotions, the operator will stop the application until the users calm down and learn to tell their emotions in the right way. Also, when the system did not receive any response in a limited time, the system will determine that the user is out of control and change to relax occasion mode to reduce their anger or nervousness.
To achieve the project aim, the project has defined a number of sub-objectives as follows:
Essential key technologies.
About the application engine, we use Unity Engine.
We use the HTC Vive focus plus to test and develop the VR environment. HTC Vive focus plus is an all-in-one Virtual Reality headset with the Android Operating System.
We use the Mongodb Nosql online database from Mongo Atlas Cloud Service to store the dataset for the feedback system.
These are compatible for us to develop our project.
Figure 1: Hardware Requirements
Functionality Design
Figure 2: Functionality of our application
Main Menu & Account
The application will provide login and create an account for users to use our application. Users can create their only name and the password. The password for the account will be encrypted. Login and create function will access an online database for checking the username and adding the account information into the database.
Figure 3: Main menu screen
Figure 4: Main menu screen
Introduction and Instruction
The introduction and instruction will show our project aim and teach users how to interact with it easily.
History
Feedback History on the main menu will get the dataset such as the timestamp and total score from the online database to show the record. Users can watch their recent records if they want.
Figure 5: Instructions screen
Figure 6: History screen
Occasion Design
Scene Design
To increase the professionalism and rationality of our occasions and feedback system. We especially asked a primary school teacher to evaluate our situation design and feedback system design based on his professional knowledge and rich experiences in contact with primary students. Here is the design of all scenes.
For the marking scheme of our grading and feedback system in occasion design. The marks range between -10 to +15. The criteria are depending on the severity or performance of the user. If the choice is most suitable to show emotional management skills, higher marks will be obtained. If the choice is showing that the user can not control well in emotional management and involves some violent element, lower marks will be obtained.
School – Classroom Scene:
Situation 1).
The first situation in the classroom scene is called “teacher punishment of the performance after the lesson”. This situation talks about the user who has not done the preparation in the class and talks too much in the class. Class teacher finds the user to have a short meeting about that bad performance. She wants to punish the users. This situation mainly examines the emotional management skills when being blamed. The characters in this situation included the class teacher and classmates. The module corresponds to the teacher and student model.
Figure 6: Environment and 3D model in scene ‘School – Classroom’
Figure 5: Design of School – Classroom Scene: Situation 1
Voice Guidance
Voice guidance will be provided on every occasion such as the conversation with NPC, reflection of user choices. Voice guidance will include human actors and online AI voice.
Figure 7: Subtitles in scene ‘School – Classroom’
Relax Scenario Design
We designed the Relax Scenario for users to take a break after they try the scene. The environment for the scene will place it in a natural place with animation and music that can let the user take a break. We designed different ways for users to go to the Relax Scenario in case immediately or enter by manual control in the menu. Also the user can leave the scene or back to the previous playing scene.
Figure 8: Environment of Relax Scene
Feedback System
Feedback Result Menu will show all of the dataset after they finished such as score and total of score. The dataset will be obtained from the local database and sent to online database storage. The suggestion will be stored in the local storage and it will relate to the question that the user may choose the best choice or the worst. They can watch their whole history in the Feedback history menu again after they finish.
Figure 9: Score and User data on Feedback
Database Design
The dataset of the application is not complicated so we use Nosql format to store it. It will store all user data in user account and feedback history. The user Account will be stored in the ‘Account’ in Figure 10. The dataset included the username and password will insert and update while the user is trying to login or create an account. The user dataset of all scores will be stored into the online database ‘User’ in Figure 10 after the user finishes all scenes. About the structure of the table, each answer has its own score. Totalscore will convert to be the grade in Pass or Fail and give the related comment to the user.
Figure 10: Design of Online Nosql database in user account and user record
Evaluation Methods and Design
Functionality testing. We found some classmates who are familiar with game operations to test our application performance and fluency.
User satisfaction: Questionnaire. For the game design, the primary students are the target audience of our application. The objective of the project is to find out the students who are the potential person to have emotional management problems. We only need to find the primary students to test the application. We plan to invite around 20 students and to play our application in the first round test originally. For the COVID 19 factor, we need to keep a social distance and avoid body contact. It enhances the difficulty to get permission from the parents to play the game. For safety reasons, it has the probability to change the strategy of evaluation temporarily.
Finally we decided to use an online survey to finish the questionnaire section. The respondents do the questionnaire after they watch our full functional demonstration video.
User satisfaction: Focus group interview. Then we would ask for the focus group to have an interview. e.g. the respondents would think the app is high relative to their need for emotional management. The second principle is the user who finished the questionnaire and who has the interest to give our detailed feedback. The qualitative data would be observed and recorded, e.g. the characteristics of an emotional management application. We also ask the respondents to give some suggestions and discuss the shortcomings of our application so that we can make improvements for future needs based on the information recorded.
Results & Discussion
Usefulness of Functions
Figure 11: Survey question no.2
Figure 12: Survey question no.3
Our project aim is to develop a VR Application for primary students to improve emotional management via simulating emotional occasions and analyze their reactions and responses in the feedback system section. We successfully built a VR platform for students to practice their emotional management and analyze their performance. Second, we have an emotional occasion function to provide different scenarios. We created three emotional occasions with six situations. We add some conflict elements and communication scenarios into our virtual situation similar to real life. Third, we have a professional feedback system to analyze the users' choices and estimate the students who have poor emotional management. For the objectives section, we have seven objectives designed in the early stage. Most of the objectives are focused on design and implementing our main systems. The final objective is evaluating the user satisfaction of the project. We finished developing all of the functions and evaluated our application. We successfully finished all the stages of the progress of development and satisfied our project aim and objectives for the base requirement.
The flow process of the user to achieve the improvement in emotional management. First, our application developed for the VR platform and increased the immersion via VR technology. Second, the emotional occasions and interactive mini games provide good practices for the user. Third, the user gains a professional feedback and performance rating from the feedback system. Finally, the user achieves emotional management improvement through our virtual social practices.
The performance and effectiveness of our application are also in line with our expectations. We hope that the user can learn new knowledge or skills when we design the application. Most of the respondents agree that our application can improve their ability of emotional management and agree our feedback system provides a good reference for them to know their performance in the questionnaire section. For the result In the group interview of the evaluation section, the user can learn different bits of knowledge other than emotional management skills. They easily understand other people’s feelings and won't get angry more easily. Some respondents said that they are more patient when listening to others. Second, they can get more experience to cope with their difficulties in real life and enhance their communication skills. To further demonstrate that our application is innovative and meaningful in the mental health and education area.
Although we have a good idea to develop this application, we still have a long-range to be a perfect product. We found some limitations when we developed the application and we integrated the suggestions from the evaluation group interview section. We got poor feedback in graphic and media design because of the budget limit and time limit. It is difficult to find some 3D models to have a perfect match with our situation design. Imperfect 3D model can reduce the story expression and reduce the immersion of the gameplay. For the virus and hygiene problems, it is difficult to find some professional actors to do the character voice work for our application. Improving emotional management of primary students using VR experience is a serious game. If we can improve the character 3D model design and be professional in character voice in future, enhance the professionalism and credibility of our application.
We only have three emotional occasions at the current stage. I think we can create more emotional occasions in the future. Increasing the base number of VR scenes can help our feedback system to be more accurate to analyze the grade of user performance.
Third, some of the respondents thought that our interactive mini-games were not enough. Surprisingly most of the respondents were impressed by our interactive mini-game on occasion. The interactive mini-games let the user invoke more on the occasion and increase their immersion and have a good connection with the emotional occasion. We planned to create some interactive mini-games similar to the housework tasks in the home scene. The gameplay of the mission or task mode is a good way to achieve the immersion. We hope that interactive mini-games can be implemented in different situations and choices.
Regarding our application, an extension for the near future is the use of interactive mini-game and more professional multimedia design in order to improve.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
