School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Science with Honours in Web Technologies |
| Supervisor | Dr. Raymond Pang |
| Areas | Intelligent Applications |
| Year of Completion | 2013 |
| Award | IEEE Macau Student Project Competition 2013 Second Runner-Up |
The aim of the project is to develop a Kinect-based 3D scanner system to scan object for e-commerce website and analyze the possibility of such a system in e-commerce marketing. Currently, online shopping is popular all in the world. So many companies want to join in the online auction market. But usually most of online shop website only displays the image, for example, a shoes, figure, electronic device and accessories etc. That makes the customer without confident to buy the product because they can't view the details of product. The 3D scanner will help the sellers to present the visual 3D product to the customers. Then it can increase the attraction of products to the customers. Also, the 3D system will provide simple steps to produce 3D object and output file and then collaborate with the e-commerce website. The website will be built in 3D component system for user to upload the 3D product easily and display the good quality of product.
The project has defined a number of sub-objectives as follows:
In the Initial stage, we tested on the ReconstructME SDK for our project but we found that we cannot enhance the sdk easily and lack of the color on the model. As we concerned about the problem of ReconstructME, we are going to develope the whole 3D reconstruction engine on our own. Our approach is different from the ReconstructionMe, in which a triangular mesh model is generated. While, in our 3D reconstruction system, the output model is represented in point cloud. The point cloud contains the color information, therefore we improved that the lack of the color on the model. Moreover, our system is provided with better tracking of the scene and the upload function for uploading the output to the webhost, it helps the user to scan the object and management on their own product more easily. Before the description of the 3D reconstruction system, the procedure of our 3D reconstruction system is shown below:
The both saturations on above will be work by the 3D reconstruction system. In the saturation 1, the Kinect captured the color image of the car and the depth map in the view 1, after that, the Kinect is translated and rotated to the position of View 2 and captured, the point cloud data of these 2 views are different and the 3D reconstruction system will be registry for these point cloud data into the 3D model. The difference of the Saturation 2 is using the rounded table and rotate the car model instead of rotate the Kinect, the point cloud data that captured when rotating the model is different, and the 3D reconstruction system will be align them together. After understanding how our system works, the coming paragraph will showing the main procedure about the 3D reconstruction system as below:
As you see the overview of 3D reconstruction system, there are 3 procedures in the system as below:
Procedure 1: Model Capture
The model capture procedure is used to obtain the point cloud data from the RGB-D data. OpenNI is implemented in our system and RGB-D data are stored in RGBImage[width*height] and Depth[width*height] by default, therefore I can matching them into our own point cloud object easily in next procedure.
Procedure 2: Processing on Point Cloud Data
The RGB-D data is obtained by previous step, and now I am going to mapping them and converting to our point cloud object. Before the conversation on the RGB-D data, the mapped RGB-D data is in projection model, therefore it has to be convert to the world model. The OpenSceneGraph is implemented for the point cloud viewer.
Procedure 3: Point Cloud Registration
There are 5 steps of our point cloud registration and the briefing on each step will be shown below:
The following shows the details:
For the e-commerce website, our target is to create the C2C Web Store for consumer to do online auction. So we have design the Web store tree map is showed below:
The beta of web store index figure is showed below:
To 3D visualization functional beta versions of the site are created and every aspect of it is tested thoroughly before site launch. Use XB Point Stream and JavaScript to create 3D product rendering page embedded the product detail page. And then provide control feature bar to increase the user interaction and experiment. The product status showed behind the 3D product, product description and other related product under the above content.
The 3D product is showed below:
We have built a prototype system with Intel i5-3.4GHz CPU to realize our idea. The scanning is done by manually moving the Kinect camera around the target object. The system will record every frame that through the ARTookit label setup scan area and its point cloud obtained and display on screen interactively with the real viewfinder as shown below:
There testing products are used in our experiments, they are the bunny, bottle, and Astroboy. The scanned results are list in the table below:
The post-processing time is around 1-3 minutes depending on the size of the model (the total number of points). The process can be further improved by optimizing some of the implements in the current prototype system. The scanned model can be freely rotated, zoom in and out within the web browser environment so as to view the object under different viewpoints as illustrated below:
The accuracy on the matched model may be affected by the noise; therefore we have designed the evaluation on different conditions. The scanned results are listed in a table below:
There are different amount of noises in below three environments. In the similar shape, it included unnecessary unmatched point which reconstructed inaccurate products. Because the limitation on the curvature evaluation by PCA. When under strong light intensity, the senor cannot capture enough depth data to reconstruct the perfect 3D feature. Because infrared sensor will be affected by strong light confuse to the infrared reflection. Compared to these environments, the explicit shape reconstructed better 3D feature and more matched points.
In the web host layer, data transfer speed is very important for user. So the 3D model needs to be good quality and appropriate size to represent the detail. The testing results are listed in a table below:
From left to right side, the first one product size is larger than others because there is only one frame but included unnecessary point cloud as noise and background without match processing. The second product size is filtering out a lot of noise and background but the shape quality is unacceptable. Compared to previous two products, the last one quality and size is better. Because it through matching and optimization processing that size is small is less noise and filter out the background.
In this research, we demonstrate that a consumer level 3D object scanning device has significant potential to evolve the current habits of e-commerce because of the portability, affordable cost and user-friendliness for C2C sellers to capture 3D geometry of goods for sale; instead of using traditional 2D images. On the other hand, low-cost 3D reconstruct techniques can be applied to other application areas such as modeling in animation production. In addition, we would like to improve our result from 180 to 360 degree 3D product. This would provide more complete product to the user get more good experience and interaction.
Copyright Cheng Wing Yeung, Geoffrey Poon, Yeung Yu Yin and Raymond Pang 2013
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
