School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Science with Honours in Web Technologies |
| Supervisor | Dr. Jeff Tang |
| Areas | Intelligent Applications |
| Year of Completion | 2013 |
The aim of this project is to simulate a real room environment in a better way, which allows the users to experience the outcome of possible furniture arrangement and evaluate whether the outcome is acceptable. To achieve the aim, the Augmented Reality technique is applied. The main objective of the project is to measure the dimensions of a real room by a depth camera, which is provided by the Microsoft KINECT sensor. It converts the depth data for our system, and uses those data to estimate the available space in the real room. Afterwards, the users can select the 3D virtual furniture from the system database and put them into the room. Finally, our system enables the user to customize their favorite size or color of furniture and give some suggestion for the user such as whether the space is enough for placing furniture.
The project has defined a number of sub-objectives as follows:
The figure below shows how the system flows. When the system start, the system is going to get the depth and the color data by KINECT, and measure a real distance and identify the coordinate for estimation of the wall skirting. The sizes of the floor is estimated from the depth information and the furniture is scaled to real dimensions according to distance with respect to the camera.
And the followings are actually for the user. User can select the furniture what they want from the furniture list and put them into the AR scene. Then, the user can depend on their idea to manual edit the position of furniture. In additional, the system can automatically arrange the selected furniture by prioritizing their shapes. Finally, user can see how the style is after put the furniture in there and save the result for future reference. The use-case diagram is shown below:
In the system, user can through KINECT sensor to get depth data and convert the data to generate AR scène. Application allow user to adjust the floor level when the deviation happened.
Moreover, user can selects different type of 3D furniture and put them into AR scene. When the furniture is selected, system will show the furniture information to user. Also, application allows user using hit detection to control the position of 3D furniture model in an AR scène. Finally, the output AR scene can be saved for the user’s future reference.
The Augmented Reality automatic furniture fitting system is going to be implemented in C# language. The system is equipped with a computer vision module, which is enabled by the KINECT sensor device. According to the literature study, we found most of the related applications that used this device were developed in C#, C++. And by comparing with C# and C++, C# is more simple and stable when writing a GUI program and the project is written in WPF, which defines an interface in XML format so the developer can edit the UI components easily and in an organized way, although C++ is more powerful when writing console applications.
Also, we use KINECT SDK to develop our application. KINECT SDK provides the tools and APIs, both native and managed, that supports applications built with C++, C#, or Visual Basic using Microsoft Visual Studio 2010 or 2012. So we develop KINECT-enabled applications for Microsoft Windows. Another SDK is OpenNI which is an open source used for the development of 3D sensing middleware libraries and applications but it is more complex to identity depth map in our system. So we use KINECT SDK.
Windows Presentation Foundation uses data points to create a mesh. 3D graphics in WPF require many data points for a smooth and clear image. To prevent issues like blurry in 3D model, Foundations of WPF, which the best methodology for programming in 3D with WPF is to use a 3D designer package to create models and meshes, and then import them into WPF. The following shows the details:
For import 3D model into a C# program of WPF, that only support XAML file. We should convert 3D model (.3ds) into XAML. Most of the 3D modeling software has 3D Model to XAML plug-in. Then WPF can import a 3D model is displayed in Windows Form.
About the furniture to a given coordinate, we should set model property of translating and rotate transforms (i.e. transform the model on the floor and spinning it to face the proper direction, respectively). Since we want all of our furniture to be on the floor and not floating in the sky or buried underground, we need concern ourselves with applying the proper x and z translations as well as the proper rotation about the y axis. The following shows the details:
To pick furniture and translate/rotate it in the 3D space, we implemented the “Hit detection” in the system, which enables the user interacting with the selected 3D furniture models with mouse events. The following shows the details:
The scaling of furniture model need match distance changing between KINECT and wall. We use the max detection of KINECT (531.7 x 409cm) to compare the distance between KINECT and skirting. It can the ratio in detected range and use it to scale the ratio of furniture. The following shows the details:
The 3D furniture models are put into 5 levels according to their widths. The longest width is level 5 and starts from it. And put it to back to the wall at the skirting. Then, put the following furniture next to it if there is enough space. Otherwise, put and rotate it at left hand side and checking for other available area to put more another model. The following shows the details:
The Interface of the Main Software simply divided into three parts.
The following shows the details:
To evaluate the system, we find some user trying to use the application. And complete a questionnaire for giving us some feedback. In the evaluation, we visited a total of 20 non-technical users. Since the system is need to work with the KINECT device, but not everyone had. So when doing the evaluation, we taken a video of system operation, the respondent may require watching the video to answer a list of question. Also, we setup the system in the school and find other student or teacher from other classes or field to do the experience. This can get a more fair feedback.
In evaluating the following three aspects, those are the system usability, flexibility, accuracy by the user after experience with our system. The following shows the details:
For the results, the overall of evaluation is good and satisfied. The question 6 is highest in evaluation. Absolute majority of interviewer represented that the system is usable in Hong Kong and through the system can helpful to their home design. In the unexpected results, the question 3 and 4 is lowest in evaluation. Some of users think the skirting in the system is not accurate and not flexible. The reason may be between the KINECT and skirting have a noise affect the detection and the KINECT detection is not 100% accuracy. Also the system cannot handle the distance between skirting and KINECT with barrier in present. In addition, the result of question 2 and 7 are pleased. Most of respondents said that the system is easily to use and the results of automatic fitting is satisfied. The following shows the details:
Conclusion of evaluation result, we can find that our system is more helpful furniture buyer and suitable for Hong Kong. Also, most of people like using Augmented Reality (AR) technique to put 3D furniture in the scene of a real room to design their home. This proves that our system has a great potential.
In our project, we want to provide an effective and valuable way to do home design in Hong Kong. It is to use KINECT to measure room size to get a real room environment that allows users to examine furniture placement and evaluate the overview. It is our automatic furniture fitting system.
In this system, there are 6 main objectives we satisfy. After all objective was finished, we can distribute two part of this: depth map analysis and auto fitting arrange. Depth map analysis can adjust all model moving and their size contain skirting analysis, furniture model scaling, Horizontal and vertical adjustment of model. The other part is Auto fitting arrange is main function of system which involve almost objective.
Also we find some limitations in our system. First the detection range of KINECT is from 0.8m to 4 m. So we recommend user can start analyses form the corner. The next one is deviations appearing when KINECT scanning ranges increase. We are checking by average points to reduce data fluctuation.
With the future, we propose the system can find the distance between skirting and KINECT with barrier. It is make it more effective and practical to handle more case.
Also, the system can allow user to import furniture in application they want which provide a great diversity of furniture and increase liberty to use the system.
Copyright Chan Kwun Kit, Lau Wan Man, To Kwok Ho and Jeff Tang 2013
