School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology
Bachelor of Science with Honours in Computer Science |
| Supervisor | Prof. Andrew Lui |
| Areas | Intelligent Applications |
| Year of Completion | 2022 |
The aim of this project is to achieve interview automation through developing a mobile application coupled with an interview chatbot for HR interviewing first-round interview in a time-efficient and less-resourced approach
To achieve the aim, the project has defined a number of sub-objectives as follow:
Demonstration Video
Presentation Video
The supporting technologies that are helpful to this project include the following technologies.
In this project, we will need to build servers, a chatbot, and a mobile application.
A cloud platform allows us to deploy our servers and provide services with various benefits.
Databases allows us to store data and be accessed by our server or application.
An IDE allows us to develop our mobile application for the project.
To build a chatbot for interviewing in a voice-based approach. We will need to adopt a conversational agent that delivers and responds in audio. Speech-To-Text and Text-To Speech techniques are required. Also, machine learning tools for classification is needed to process answer text.
For programming languages, Node.js can be used in writing servers, Python can be used in the development of the chatbot, and Java can be used in mobile Android application development.
Architecture or High-Level System Design
Major Components are Android Client, the Server for machine learning and NLP and databases.
An interviewee will be using the Android Client for interviewing and receiving questions from the server and sending answers to the server. The server will process the interviewee's resume for generating questions, receive answers from an interviewee and update database data after processing the answer. During the interview, the android client will take photos of the interviewee as evidence of the real applicant. After the interview, the server will generate a report stored in the database.
Figure 1: Components of the system
There are 5 stages in the first-screening interview which will be focusing on this project.
The first stage requires an interviewee's resume and generates a token for authentication.
The second stage describes the process of question generation.
The third stage is the flow of validation. How the interviewee receives the token and how to track the real applicant.
The fourth stage is the interview process. Mainly conducting Q&A conversations for times. Interviewees will read their answer as an input to the chatbot. A report will be generated after the interview. Photos of the interviewee will be taken occasionally and stored for post-interview review.
The fifth stage is the output stage after the interview. HR could read the reports with filter options provided.
Stage 6 and 7 are second-screening and third-screening where interviewees selected by the HR might attend.
Figure 2: Information flow and interview model
Our user evaluation is inviting some people as interviewees to use our chatbot and mobile application. We will be reviewing the report generated after the interview. The evaluation procedures are as follows.
We have discovered several issues during the user evaluation.
First, users were confused when using the application. They started the interview without reading the instruction. Users responded directly after the chatbot without using the buttons for recording audio at first. Users got lost when the interview was started, they did not know what to respond.
Second, there were some recording issues and unsuccessful speech recognition. Since we conducted the user testing online. Recording quality is highly correlated to the interviewee's recording equipment. Bad audio quality caused the failure of speech recognition and classification. Also, users' accent and poor articulation may cause unsuccessful speech recognition.
Third, there were some issues of chatbot. The chatbot sometimes could not classify the label of “repeat” and “not sure” and could not understand a topic in further. Also, the response time of the chatbot is long due to lengthy operations such as Google text-to-speech, speech-to-text, zero-shot classification, and uploading the file.
Lastly, we observed that users tend to respond in short instead of long sentences when talking to the chatbot. We have concluded few possible reasons of it. First, interviewees did not know the capability of the chatbot on handling audio. So, they were not sure how much they should talk. Second, they had no experience of chatbot interview, so they were not used to speaking to a chatbot. Hence, they could not anticipate how the conversation would go and did not perform normally as in-person interview. Another reason might be that they try to interact with the chatbot as interacting with AI assistant. Therefore, they tried to answer questions concisely sometimes.
In our user survey. We have asked the following questions. Some are the common questions; some questions are only asked to users who have experience in using chatbot for interview.
Most of our users have never been interviewed by a chatbot.
In both situation of before or after using the chatbot, most of the users did not feel boring, nervous, or difficult. They think chatbot interview is time efficient, work efficient, and effective. One of the users felt nervous when using the chatbot.
About the functionality of the chatbot, our users agree that the voice from the chatbot is clear, the user interface works well.
However, some of the users think that the chatbot cannot hear their answer clearly, the chatbot cannot understand their answers. Also, they think the chatbot does not response promptly.
For the design of the interview, our users think the follow-up questions is asked reasonably.
According to one example of our survey results. One of the users have used other interview chatbot before, but that chatbot does not provide follow-up question based on interviewee's answer which our does. Also, that user consider our chatbot is a more capable than the chatbot he used before.
Lastly, our users would like to have chatbot interviews in the future and agree that chatbot interview will be a trend.
| User Survey Questions | |
| Interviewed by chatbot before | Never interviewed by chatbot |
| [common]
1. Have you ever been interviewed by a chatbot? |
|
| [used before]
1. How many times have you been interviewed by chatbots? |
|
| [common]
2. What are your first impressions of chatbot interviews? (before) 3. What was your impression after the chatbot interview? 4. Please rate the functionality of the interview chatbot. 5. Any other problems/impressive features? |
|
| [used before]
2. Which features of the interview chatbot you used before are included. (Not including this chatbot) 3. Compare to the chatbot you used before, please rate the functionality of the interview chatbot. |
|
| [common]
6. After this experience, what type of interviews would you like to have in the future. 7. Do you think interview chatbots will be a trend in the future? 8. Any comment or suggestion. |
|
Table: User Survey Questions
Interview automation has been accomplished successfully and demonstrated in this project. The aim and objectives of this project have been satisfied. Despite first-screening interview can be automated and chatbot could play a role of an interviewer, we do not try to explore the replacement of human interviewer or claim all interviews can be automated using a chatbot.
In automating interview, we have developed a chatbot and a server for hosting the interview, an android application as a client of the chatbot server. Besides, we have developed a web application for collecting resume information and visualizing collected data for reviewing. All developed programs are integrated as a prototype system which could function properly and carry out the interview in an automated and voice-based approach.
Though our prototype system contains limitations, we believe the system can surely demonstrate the idea of automated chatbot interview and suggest the structure of applying chatbot in interview.
For collecting interviewee resume and data. A more powerful tool might be used. Alternatively, an online resume form could be provided to interviewees. As the form could be designed in any structure to collect required information and regulate the result to ensure consistency.
For the classification model, we may train the model in further based on the original model to attain a better performance in understanding interviewees. In the chatbot, there is a function for summarizing answer text if the text is long. However, according to our observations, users tend to answer in short sentences instead of lots of words. The summarization function might be useful in some cases but remains unused in most of the time. So summarizing techniques could be adopted when long response is expected.
In using the mobile application, a better user guide or tutorial for interviewee could help in better user interaction.
In practice, we might want the chatbot could handle concurrent access so more interviews can be conducted at the same time. If the interview is conducted in a company, appropriate equipment including headphone, microphone with good quality, and tranquil environment for recording might be provided by the company to ensure the recording quality to avoid speech recognition problem. The chatbot model could be modified to suit different types of interviews of various aspects or job types.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
