School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Science with Honours in Web Technologies |
| Supervisor | Dr. Jeff Tang |
| Areas | Intelligent Applications |
| Year of Completion | 2013 |
The aim of this project is to simulate a real room environment in a better way, which allows the users to experience the outcome of possible furniture arrangement and evaluate whether the outcome is acceptable. To achieve the aim, the Augmented Reality technique is applied. The main objective of the project is to measure the dimensions of a real room by a depth camera, which is provided by the Microsoft KINECT sensor. It converts the depth data for our system, and uses those data to estimate the available space in the real room. Afterwards, the users can select the 3D virtual furniture from the system database and put them into the room. Finally, our system enables the user to customize their favorite size or color of furniture and give some suggestion for the user such as whether the space is enough for placing furniture.
The project has defined a number of sub-objectives as follows:
The figure below shows how the system flows. When the system start, the system is going to get the depth and the color data by KINECT, and measure a real distance and identify the coordinate for estimation of the wall skirting. The sizes of the floor is estimated from the depth information and the furniture is scaled to real dimensions according to distance with respect to the camera.
And the followings are actually for the user. User can select the furniture what they want from the furniture list and put them into the AR scene. Then, the user can depend on their idea to manual edit the position of furniture. In additional, the system can automatically arrange the selected furniture by prioritizing their shapes. Finally, user can see how the style is after put the furniture in there and save the result for future reference. The use-case diagram is shown below:
In the system, user can through KINECT sensor to get depth data and convert the data to generate AR scène. Application allow user to adjust the floor level when the deviation happened.
Moreover, user can selects different type of 3D furniture and put them into AR scene. When the furniture is selected, system will show the furniture information to user. Also, application allows user using hit detection to control the position of 3D furniture model in an AR scène. Finally, the output AR scene can be saved for the user’s future reference.
The Augmented Reality automatic furniture fitting system is going to be implemented in C# language. The system is equipped with a computer vision module, which is enabled by the KINECT sensor device. According to the literature study, we found most of the related applications that used this device were developed in C#, C++. And by comparing with C# and C++, C# is more simple and stable when writing a GUI program and the project is written in WPF, which defines an interface in XML format so the developer can edit the UI components easily and in an organized way, although C++ is more powerful when writing console applications.
Also, we use KINECT SDK to develop our application. KINECT SDK provides the tools and APIs, both native and managed, that supports applications built with C++, C#, or Visual Basic using Microsoft Visual Studio 2010 or 2012. So we develop KINECT-enabled applications for Microsoft Windows. Another SDK is OpenNI which is an open source used for the development of 3D sensing middleware libraries and applications but it is more complex to identity depth map in our system. So we use KINECT SDK.
Windows Presentation Foundation uses data points to create a mesh. 3D graphics in WPF require many data points for a smooth and clear image. To prevent issues like blurry in 3D model, Foundations of WPF, which the best methodology for programming in 3D with WPF is to use a 3D designer package to create models and meshes, and then import them into WPF. The following shows the details:
For import 3D model into a C# program of WPF, that only support XAML file. We should convert 3D model (.3ds) into XAML. Most of the 3D modeling software has 3D Model to XAML plug-in. Then WPF can import a 3D model is displayed in Windows Form.
About the furniture to a given coordinate, we should set model property of translating and rotate transforms (i.e. transform the model on the floor and spinning it to face the proper direction, respectively). Since we want all of our furniture to be on the floor and not floating in the sky or buried underground, we need concern ourselves with applying the proper x and z translations as well as the proper rotation about the y axis. The following shows the details:
To pick furniture and translate/rotate it in the 3D space, we implemented the “Hit detection” in the system, which enables the user interacting with the selected 3D furniture models with mouse events. The following shows the details:
The scaling of furniture model need match distance changing between KINECT and wall. We use the max detection of KINECT (531.7 x 409cm) to compare the distance between KINECT and skirting. It can the ratio in detected range and use it to scale the ratio of furniture. The following shows the details:
The 3D furniture models are put into 5 levels according to their widths. The longest width is level 5 and starts from it. And put it to back to the wall at the skirting. Then, put the following furniture next to it if there is enough space. Otherwise, put and rotate it at left hand side and checking for other available area to put more another model. The following shows the details:
The Interface of the Main Software simply divided into three parts.
The following shows the details:
To evaluate the system, we find some user trying to use the application. And complete a questionnaire for giving us some feedback. In the evaluation, we visited a total of 20 non-technical users. Since the system is need to work with the KINECT device, but not everyone had. So when doing the evaluation, we taken a video of system operation, the respondent may require watching the video to answer a list of question. Also, we setup the system in the school and find other student or teacher from other classes or field to do the experience. This can get a more fair feedback.
In evaluating the following three aspects, those are the system usability, flexibility, accuracy by the user after experience with our system. The following shows the details:
For the results, the overall of evaluation is good and satisfied. The question 6 is highest in evaluation. Absolute majority of interviewer represented that the system is usable in Hong Kong and through the system can helpful to their home design. In the unexpected results, the question 3 and 4 is lowest in evaluation. Some of users think the skirting in the system is not accurate and not flexible. The reason may be between the KINECT and skirting have a noise affect the detection and the KINECT detection is not 100% accuracy. Also the system cannot handle the distance between skirting and KINECT with barrier in present. In addition, the result of question 2 and 7 are pleased. Most of respondents said that the system is easily to use and the results of automatic fitting is satisfied. The following shows the details:
Conclusion of evaluation result, we can find that our system is more helpful furniture buyer and suitable for Hong Kong. Also, most of people like using Augmented Reality (AR) technique to put 3D furniture in the scene of a real room to design their home. This proves that our system has a great potential.
In our project, we want to provide an effective and valuable way to do home design in Hong Kong. It is to use KINECT to measure room size to get a real room environment that allows users to examine furniture placement and evaluate the overview. It is our automatic furniture fitting system.
In this system, there are 6 main objectives we satisfy. After all objective was finished, we can distribute two part of this: depth map analysis and auto fitting arrange. Depth map analysis can adjust all model moving and their size contain skirting analysis, furniture model scaling, Horizontal and vertical adjustment of model. The other part is Auto fitting arrange is main function of system which involve almost objective.
Also we find some limitations in our system. First the detection range of KINECT is from 0.8m to 4 m. So we recommend user can start analyses form the corner. The next one is deviations appearing when KINECT scanning ranges increase. We are checking by average points to reduce data fluctuation.
With the future, we propose the system can find the distance between skirting and KINECT with barrier. It is make it more effective and practical to handle more case.
Also, the system can allow user to import furniture in application they want which provide a great diversity of furniture and increase liberty to use the system.
Copyright Chan Kwun Kit, Lau Wan Man, To Kwok Ho and Jeff Tang 2013
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
