School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Science with Honours in Computing |
| Supervisor | Dr. Vanessa Ng |
| Areas | E-learning and Intelligent Tutoring Systems |
| Year of Completion | 2010 |
The aim of the project is to integrate both teaching assessments and learning into a single real-time system architecture. Feedback is provided immediately to act as a reflective tool for students. As aforementioned, student-driven concept will be a key part of this project. Therefore, the main objective of this project is to design and implement a web based assessment system to engage students self-learning through the dynamic questions generated by the system.
In this paper, an online assessment system is proposed to engage students to have self-learning through the use of dynamic questions. The system will automatically design and generate questions for individual students according to their learning ability. Students can take the assessment at anytime and anywhere. Questions are not only prepared by the teachers in advance (which are stored in a database), but are also generated by the system automatically known as 'parameterized questions'. As a result, students can perform self-assessment on a particular subject according to their needs. This would be considered as a self-learning activity for the students who are weak at particular subjects and they can learn through practice and exercises.
The proposed online assessment system will include the following sub-parts:
Although a variety of online assessment systems (OAS) have been proposed in the past few years, most of these systems are result-driven that focused on the assessment results only, they are not able to ensure effective learning of the students. In other words, the existing online assessments are basically equivalent to pen-and-paper assessment with questions prepared by the teachers.
The existing OAS is simply employed as a tool for teachers to perform online assessment but not a tool for students' learning. Just like written assessment, this is a unidirectional assessment that is guided by teacher. This does not make full use of the information technology. In other words, existing OAS is basically equivalents to written assessment. It is still devoid of measurements of student's learning effectiveness. As a result, student-centred assessment system with parameterized question approach should be introduced to solve these problems.
In this project, different programming language, Database Management System, texting mining tools and wordnet have been studied and the final configuration are listed below. The details are described in the following:
JSP chosen as the programming language of the project is twofold. Since JSP is part of the Java technology family and shares similar programming syntax with java program, it is easier for me to employ this programming language in this project. Moreover, JSP technology allows web developers to create dynamically generated web pages with html and xml which is a crucial part in developing the online assessment system.
Background research on the database management system was performed to see which systems best suit this current project. Features between PostgreSQL, MySQL and Oracle 9 were compared. At last, MySQL Server was employed for its free and robust database management system which will contribute to my database management.
Text-mining tool such as the part of speech tagger and wordnet must be applied in order to generate question of a proper English text. A number of text-mining tool and wordnet API had been studied before the development of the question generating system. At last, Stanford POS Tagger was decided to use in this project. One of the most important reasons is the 97.28% accuracy tagging the correct and unknown words. Moreover, Java library and API are also provided in detail which is easier to use and has a high extensibility in its feature. However, it is true that the higher the accuracy, more time is need for tagging the sentence. Rita Wordnet is selected because it is open source and freely available over the internet. It is java library that can be integrated with my system easily. At the same time the API can provide adequate method to retrieve lexical data easily which is useful in generating English grammar question.
The online assessment system is based on client server architecture. It consists of two major modules: the central web server which acts as the repository for data stored and the system services provider. The web server is accessed in the client side by the web browser. The web server is the core module of the system which provides the following main tasks: database connection, authentication, questions generation, giving feedback and generating summary reports. The online assessment system architecture is shown below:
The system consists of three subsystems: Administrator Subsystem, Instructor Subsystem and Student Subsystem which is based on a central database component and includes a group of system modules. These subsystems provide the system interfaces for different users. The system functional component diagram is represented below:
The main features of the system developed include:
The Dynamic Question Generating System is applying Rita WordNet, Stanford Log-linear Part-Of-Speech Tagger and Java Math parser. Basically, questions about English grammar and primary level of mathematic are generated. Rita Wordnet is applied as a source of lexical recourses, Stanford Log-linear Part-Of-Speech Tagger is used to indicate the word class of the words in the English sentence and Java Math parser is used to generate the step by step solution for the feedback system. Question patterns and details are gathered, analyzed and used to generate the parameterized questions with answer automatically.
Grammar exercise is usually a set of patterns covering different grammatical items. It can be generated easily by prompting in a candidate sentence. Using POS tagger, the part of speech of each word can be tagged indicating the syntax of that sentence. Question can then be constructed by analyzing the syntactic relations and replacing the target words in the sentence with those in the WordNet of database. The flow of the question generating system is shown below:
The following shows an example of the sentence with the POS Tagger:
The following shows an example of generalizing parameterized expression for mathematical questions:
For the Automatic Critiquing System, submitted answer is marked and graded by the system automatically. System will able to evaluate the answer as correct, incorrect. This can be simply achieved by comparison of the submitted answer and the model answer. However, it may require some advanced computation technology for matching answer like unordered word and algebraic expression.
For the Feedback and Performance Evaluation, performance report of the students will be evaluated for reference after the assessment. In addition, a summary report of the assessment would be created if modification of the question is required. This report includes difficulty, selection of topics or numbers of various types of question. The workflow is shown below:
| Achievement | |
|---|---|
| A secure login system | Multi-Login is disable resolving the un-authorized access. |
| Examination part | Create and publish assessment through the system. |
| Add question into database. | |
| Self-Assessment part | Dynamic Question Generating System |
| Automatic Critiquing System | |
| Feedback and Performance Evaluation |
An experiment has been performed to measure the feasibility and the efficiency of the question generation of English Grammar question. 100 tense questions in level 1 are generated because generating tense question will spend most time due to the system design. Time spend on each generation is record and the experiment has repeated for 10 times a more stable result. Time spent on each attempt is shown below:
| Attempt | Time Spent |
|---|---|
| 1 | 23.46s |
| 2 | 23.19s |
| 3 | 19.14s |
| 4 | 29.64s |
| 5 | 28.32s |
| 6 | 25.67s |
| 7 | 29.52s |
| 8 | 27.66s |
| 9 | 35.32s |
| 10 | 29.50s |
Therefore, the average time used on generating 1 tense question is around 0.27s. It is no doubt that computer question generation is more efficiency than the manually design.
Another experiment has also been performed to measure the feasibility and the efficiency of the question generation of Mathematics question. 100 mixed type questions are generated for 10 times to achieve a more stable result. Mathematics question are formula-based that are generated directly from the parameterized question pattern. The time spent on generation is much less than generating grammar questions. Indeed, it is almost generated instantaneously. It is no doubt that it is more efficient to generate questions by computer than those by manual design. During the experiment, a total number of 1000 Mathematics questions are generated by the dynamic question generating system. Over 83% of the parameterized question patterns can be converted into a qualified question with the expectable parameter without any extreme cases.
The goal of the evaluation is to measure the objective and the subjective value of the proposed system as a learning tool. To determine the objective value of the proposed system, performance evaluation of the main component has been evaluated and stated in the previous chapter. In order to find out the effectiveness of learning of using a self-assessment system, a questionnaire is conducted at the final phase of the project. The questionnaire included a range of questions focused on the system's features and its usage. The sample questionnaire is shown below:
| Question | 0 | 1 | 2 | 3 | |
|---|---|---|---|---|---|
| 1 | Do you think it is a good idea to use this system for learning? | ||||
| 2 | What do you think about the overall performance? E.g. Loading time , speed | ||||
| 3 | What do you think about the system interface? | ||||
| 4 | What do you think about the Question Generating System that can generate unlimited questions automatically? | ||||
| 5 | Is it too easy of the generated question? | ||||
| 6 | Do you agree to increase more question type? | ||||
| 7 | What do you think about the overall feedback and performance evaluation??Is it useful? | ||||
| 8 | Do you agree that the feedback generated is useful?? | ||||
| 9 | Do the feedback is helpful to your study? | ||||
| 10 | Are you willing to use the system again? |
Results show that over 83% responders were positive about the system. More than 80% said that the question generating system is powerful and over 40% agreed to increase the question type of the generated system. About 60% responders agreed that the generated feedback is useful. Finally, over 60% of responders are willing to use the system again.
In this project, the background of existing online assessment system and their lacking has been describe and reviewed. So far the proposed system has met its goals in integrating both teaching assessments and learning into single real-time system architecture. A student-centred and effective framework is established in which dynamic feedback is generated and can be given to students for facilitating their learning. The dynamic feedback generation and parameterized question approach are successfully integrated into the system such that student are able to achieve the mastery on different topics in the self-assessment context. In addition, the problems of the existing online assessment system are also solved. Questions source is not the problem again, workload of teachers are thus decreased.
There are also a number of ongoing extensions to this project. One is the creation of new question types asking phrasal verb knowledge. It is one of the most frequently asked questions in grammar exercise in primary school. Another is getting the web resources such that the sentence are automatically capture from the newspaper website such as South China Morning Post. Since the source of sentences is dynamically updated to the sentence database, not only lower the workload of the administrator and instructor, passage type of grammar test item can be generated. In addition, I am looking at ways to enhance the feedback system for generating a humanistic feedback to student.
Copyright Fung Ka Kui and Vanessa Ng 2010
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
