School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology Bachelor of Science with Honours in Computer Science |
| Supervisor | Dr. Jeff Au Yeung Siu Kei |
| Areas | Intelligent Applications |
| Year of Completion | 2023 |
The project, initially aimed at developing an autonomous robotic car, has evolved after several development phases. The team realized it could automate trolleys, leading to project updates for efficiency and success rate improvement, including possibly abandoning the inertial measurement unit (IMU) implementation.
To accomplish the goal, the project has outlined several sub-objectives as follows:
The project aims to be completed with the following features:
Demonstration Video
Presentation Video
Our solution leveraged existing hardware but innovated in software aspects, including backend scripts and user-facing programs.
Our solution incorporated the following design patterns and primary components:
Supporting technical components included:
The Raspberry Pi 4’s GPIO pins controlled the robotic car manually by sending pulse width modulation (PWM) signals to the DC motor, enabling speed control. The Python-written motor driver in ROS received messages from the publisher, enabling precise power control and signal transmission.
Figure 1: The GPIO Layout of the Raspberry Pi 4
To simplify tasks that could not be controlled from our web app and required server console control, we created an all-in-one wrapper script using BASH shell scripting. We also utilized a command line tool called gum to enhance the visual appeal of our command line interface (CLI) tool.
A LiDAR scanner scanned the environment to create a cloud-point map, providing depth information for navigation stacks. To extract useful information and generate motor commands, a standard diagram should have been followed.
Figure 2: A diagram of the relationship between publisher and subscriber in a navigation stack
The robotic car, equipped with a low-cost A1M8 RPLIDAR Scanner, created a 2D cloud-point map of indoor areas, highlighting features like walls and obstacles. This map was stored on a server for future use. As per the official documentation, the fundamental specifications of the scanner were as follows:
Figure 3: A capture of the dimension of A1M8 from the official site of SLAMTEC
Cartographer was a robotic operating system enabling real-time localization and mapping. It integrated multiple platforms and sensor configurations, benefiting custom robotic cars. It divided the SLAM system into Global and Local SLAM. As per the official description, the functions of these two subsystems were outlined below:
Global SLAM:
Local SLAM:
The Indoor Positioning System (IPS) located objects or people using a network of devices. Ultra-Wideband (UWB), a type of IPS, offered high accuracy (<30 cm) and comprised two types of hardware chips: station and label. UWB could be used in hospitals to quickly locate critical equipment like pacemakers, improving efficiency and potentially saving lives. Other Types of Positioning Systems and Their Disadvantages were listed below:
Figure 4: Flowchart of UWB Navigation System
The project used UWB to implement 2D coordination using one label and three or more stations. Distances d1, d2, and d3 were the station signal outputs, and R1, R2, and R3 were the 2D coordinates of the stations. By drawing three circles with these distances as radii and the coordinates as centers, the intersection point (UWB label) gave the resulting 2D coordinate.
Figure 5: UWB 2D coordination
UWB Navigation involved using the resulting 2D coordinate to navigate a robotic car from a starting point to a target point. The car, equipped with a UWB label, received commands such as “forward”, “turn right”, “turn left”, and “stop”. The process continued until the car reached the target, determined by a preset value x indicating the car’s arrival within the target circular area.
Figure 6: decision tree in navigation
The blind point situation arose when the UWB entered a no-signal area, usually obstructed by an object. In this scenario, the car switched to a mode based on machine learning, which had been trained to approach the target from the last known UWB position. This process involved predicting the blind point area to locate the car when a signal loss occurred, using a pre-trained ML network to continue moving towards the target without UWB coordination, and finding the shortest path as data input for the ML after UWB signal loss.
The project employed the NEAT algorithm for reinforcement learning in navigation. An agent, equipped with five detectors representing sensors like LiDAR and infrared, was used. The training map was handmade, with black symbolizing walls and white representing the floor. The lengths of the detectors served as network inputs for each agent. The agent car disappeared upon hitting a wall or the target. Rewards were given when the car hit two checkpoints or the target, with the target reward being the highest.
Figure 7: RL training
The architecture of the demo application comprised ROS, server, and client components, which communicated via WebSocket and REST API. The server employed Node.js, MongoDB, and Shell Script, while the client provided APIs and documentation for self-integration, along with a demo web app. The data flow diagram demonstrated how data from sensors was utilized to generate maps, visualize them, and store them for future use, such as analysis and AI training. In this setup, the ROS component underwent minimal modification.
Figure 8: Data flow diagram
The diagram below illustrated the design of the prototype system:
The prototype system was designed to meet the following requirements:
Our Prototype System Design was shown below:
Our Assembled Prototype System Result was shown below:
The prototype system successfully met our project development requirements. The features successfully developed and incorporated into the prototype system included:
Motor Driver
The motor driver, written in Python, was unique compared to other programs in the package. It operated as a stand-alone program that could be executed with commands. It interfaced with the ROS core, receiving commands from other packages to power the motors via GPIO pins. The performance of the motor driver during testing and evaluations met our expectations, enabling the robotic car to move smoothly based on the generated commands. With the motor driver, the robotic car gained enhanced mobility, including the ability to move forward with automatic speed adjustment and to turn left or right with adjusted angles. This ensured the car avoided crashes or unnecessary shifting, enhancing its overall performance and reliability.
RPLiDAR
Throughout the testing and demonstration phases of the project, we fully realized the potential of the RPLiDAR scanner, proving it to be a key component of the prototype. It was instrumental in drawing maps using the Cartographer algorithm and navigating the car using both the Cartographer and navigation stacks. The successful deployment of the RPLiDAR scanner and associated programs enabled several key features in the prototype, including cloud-point map drawing, path finding, and obstacle avoidance, significantly enhancing the functionality and effectiveness of the prototype.
Ultra-Wideband (UWB)
The algorithms implemented in the project performed well in their respective roles as outlined in the previous sections. The ultra-wideband (UWB) system effectively received signals and guided the robotic car to its target. However, there was a regrettable limitation in the current setup. The Reinforcement Learning (RL) navigation system, a crucial component for autonomous movement, unfortunately, was unable to complete the final integration step. This was an area that required further attention and development to fully realize the potential of the autonomous robotic car.
Demo Application
Our application primarily consisted of existing ROS features, so our testing and validation process mainly involved comparing the functionality of these features in ROS and our application. The results were largely similar or closely matched. Additional features, such as logging, were also tested and performed as expected. To evaluate the user interface (UI) and user experience (UX), we conducted blind tests with volunteers from various technical backgrounds. The aim was to ensure that our application was accessible and easy to learn, regardless of the user’s prior knowledge of the subject. The feedback we received was generally positive, and we used this feedback to make necessary updates to the UI, thereby enhancing the overall user experience.
The project, initiated in August 2022, aimed to provide a solution for investigating unknown areas using map drawing, route planning, and AI-enhanced positioning. The objectives were met, with achievements including ROS package and program development, RPLiDAR and IPS implementation, and web application development.
ROS Implementation (Package and Program Development)
The robotic car used a Raspberry Pi for operations such as motor control and sensor data integration. Despite Raspbian OS meeting basic needs, ROS was chosen for its standardization in deploying and executing software operations in robotics. Despite the learning curve, ROS became the project’s fundamental platform.
RPLiDAR Implementation and Development
The RPLiDAR scanner, a key project component, scanned the environment and generated data for a cloud-point map, enabling users to review visualized mapping data. This data aided in obstacle avoidance, route planning, and map review in the web application.
IPS Implementation
The prototype was capable of locating itself in an indoor area using the Indoor Positioning System (IPS).
Demo Application Development
An API set was developed for easy ROS management/deployment. A responsive web app allowed users to control the car and scaled automatically to the user’s screen size. Original ROS features were migrated to the app, with additional features added to enhance the user experience.
ROS Launch Script
Some features, not originally part of the plan, were added due to their unavailability in the demo.
The current design of our solution had a few limitations. Firstly, the car did not move smoothly when it needed to make a turn at an angle larger than 30 degrees. This impacted the overall maneuverability of the car. Secondly, there was not enough memory to effectively run the IPS analysis program, which was crucial for the car’s positioning system. Lastly, there were technical gaps in generating a live map in the web application for regular users, which affected user experience.
To address these issues, we suggested several improvements for future work. We could develop the motor control program with more precise PWM settings and formulas to improve the car’s movement. We could also develop a more lightweight algorithm for the IPS analysis program to reduce memory usage. Additionally, we could create a better script for resource management to optimize the system’s performance. Finally, we could develop a package with related programs and APIs that could transmit the cloud-point map data to the web application, enhancing the live map feature.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
