AUGMENTED REALITY : AN APPLICATION FOR ARCHITECTURE

By Anish Tripathi
atripathi@hotmail.com
A Thesis Presented to the
FACULTY OF THE SCHOOL OF ARCHITECTURE
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the Requirements for the Degree
MASTER OF BUILDING SCIENCE (Building Science)

Advisors
Prof. Marc Schiler
Karen Kensek
Douglas Noble

ABSTARCT

Augmented reality (AR) works on the same principles as virtual reality. Yet, unlike VR where the user is immersed in a completely virtual environment, augmented reality overlays virtual objects and information over the real world. This is usually achieved by the use of see-through head mounted displays and tracking devices. Another area of computing that has seen substantial progress is wearable computing. Architecture is one of the many professions that can benefit and grow with the development of virtual reality technologies. This thesis develops a prototype augmented reality and mobile computing system for application to facilities management. It evaluates the system in a simple case study of the Master of Building Science laboratory.

CONTENTS

Acknowledgments

Part I. Background

Part II. System Design

Part III Case Study.

Part IV Conclusions and Analysis

ACKNOWLEDGEMENTS

Thanks are in order for my advisors, committee, parents, professors, friends, and fellow building science students. Without their support, this thesis would never have been completed in such a short amount of time.

My thanks especially go to Professor Marc Schiler, Director, MBS program, not just for guiding and encouraging me but also for going out of his way for ensuring resources for the project and also making accommodations in the schedule. Without his pushing and support this thesis would not have been completed on time. Thanks also to Professor Doug Noble and Karen Kensek, who were always there guiding me and providing me with new ideas, every time I was in a fix and it was the discussions with them which actually brought about the idea for the project.

I am also gratified to Tim Eilers for having lent his personal I-glasses for the project, without which the project wouldn't have started even.

I would like to thank my parents and my brother for having supported me through the project and special thanks to Neha for keeping my spirits up in rough times.

Thanks to Enrique for all the hardware support. And, thanks to all my friends and fellow students who contributed with valuable ideas and suggestions for the project.

Back To Contents

Part 1 Background

Virtual reality (VR) can be used as a powerful, three-dimensional method to interface with computers. By wearing a head mounted audio-visual display, position and orientation sensors, and tactile interface devices, one can actively inhabit an inclusive computer generated environment. With increasing computing power allowing for the processing of huge amounts of information in real time, VR technology has become more effective. Also with new advancements in the display technologies, virtual environments are coming closer to real environments and the VR technology has entered a period of public attention.

Augmented reality (AR) works on the same principles as virtual reality. Yet, unlike VR where the user is immersed in a completely virtual environment, augmented reality overlays virtual objects and information over the real world. This is usually achieved by the use of see-through head mounted displays and tracking devices. The critical problem with present augmented reality systems is the lack of real-time and accurate tracking. Since the information has to overlap with the real world, smallest errors in tracking information are detected by the human eye. Any mismatch between augmented objects and real objects can be discomforting and also result in incorrect information being given to the user.

Another area of computing that has seen substantial progress is mobile computing. With computing devices diminishing in size and with options like wireless networking, a user is no longer limited to his physical desktop. Wearable computers are the next generation of mobile computing devices. In general, a wearable computer may be defined as a computer that is subsumed into the personal space of the user, controlled by the wearer and has both operational and interactional constancy, i.e. is always on and always accessible (1). A typical wearable computer may be composed of a computer processor and battery mounted on a belt or backpack, a head mounted display (HMD), wireless communications hardware and an input device such as touchpad or chording keyboard or voice input capabilities.

Architecture is one of the many professions that can benefit and grow with the development of virtual reality technologies. A lot of experimentation and research is going on for the use of VR in the architectural design process as well as presentation. However, architecture will also be affected by the AR technology on another level with the AR based systems becoming more powerful. An augmented reality based system combined with a wearable computer can become a powerful new tool with a wide range of applications for architecture. This thesis is an attempt to experiment with some of these applications.

Back To Contents

Ivan Sutherland, the computer graphics pioneer, is largely credited with the concept of Augmented Reality (2). The Head-Mounted Display, which was prototyped by Ivan Sutherland in the late 1960’s, started out as an Augmented Reality viewing device. It was so heavy that the device was called the "Sword of Damocles" because it had to be suspended from the ceiling to off set most of the weight from the head of the user (Figure 1.1). Since then Augmented Reality systems have come a long way and their progress has increased rapidly through the 1990's. The following sections take a look at some of the augmented reality applications that have been developed or are under research.

The use of augmented reality for visualization purposes has been restricted due to speed limitations in realistic real-time rendering. However, flat shaded renderings have been employed to use augmented reality for spatial visualization.

1. Collaborative Interior Design

The collaborative design system developed at ECRC (3) is a demonstration of interactive graphics and real-time video for the purpose of interior design. The system combines the use of a heterogeneous database system of graphical models, an augmented reality system, and the distribution of 3D graphics events over a computer network (4).

The scenario for this application consists of an office manager who is working with an interior designer on the layout of a room. On a computer monitor they can see a picture of the room from the viewpoint of the camera. By interacting with various manufacturers over a network, they select furniture by querying databases using a graphical paradigm. The system provides descriptions and pictures of furniture that is available from the various manufactures who have made models available in their databases. Pieces or groups of furniture that meet certain requirements such as color, manufacturer, or price may be requested. The manager chooses pieces from this "electronic catalogue" and 3D renderings of this furniture appear on the monitor along with the view of the room. The furniture is positioned using a 3D mouse. Furniture can be deleted, added, and rearranged until the users are satisfied with the result; they view these pieces on the monitor as they would appear in the actual room. As they move the camera they can see the furnished room from different points of view (Figure 1.2).

Figure 1.2 Collaborartive interioir design

The users can consult with colleagues at remote sites who are running the same system. Users at remote sites manipulate the same set of furniture using a static picture of the room that is being designed. Changes by one user are seen instantaneously by all of the others, and a distributed locking mechanism ensures that a piece of furniture is moved by only one user at a time. In this way groups of users at different sites can work together on the layout of the room. The group can record a list of furniture and the layout of that furniture in the room for future reference.

1.2.3 Augmented Reality for Maintenance/Repair Instructions

Various industries have been funding development of Augmented Reality systems for instruction/ maintenance.

1. Augmented Reality in Architectural Construction, Inspection, and Renovation

The Computer Graphics and User Interfaces Lab at Columbia University has worked on two augmented reality systems for use in structural engineering and architectural applications. The first, called "Architectural Anatomy," overlays a graphical representation of portions of the building’s structural systems over a user’s view of the room in which they are standing (5). A see-through head-mounted display provides the user with monocular augmented graphics and tracks the position and orientation of their head with an ultrasonic tracking system (Figure 1.3).

The above application can be used effectively in teaching architectural technology. The other augmented reality testbed system addresses spaceframe construction (6). The spaceframe is assembled one component (strut or node) at a time. For each step of construction, the augmented reality system:

• Directs the worker to a pile of parts and tells which part to pick up by playing a sound file containing verbal instructions.

• Confirms that the worker has the correct piece by reading a barcode on the component.

• Directs the worker to install the component. A virtual image of the next piece, with a textual description fixed near it, indicates where to install the component, and verbal instructions played from a sound file explain how to install it.

• Confirms that the component is installed by asking the worker to place hand at a particular location on it (denoted by the barcode), then determining the position of the hand.

This research demonstrates the potential of augmented reality's x-ray vision and instructional guidance capabilities for improving architectural construction, inspection, and renovation.

1. KARMA (Knowledge-based Augmented Reality for Maintenance Assistance)

KARMA is a prototype system that a see-through head-mounted display to explain simple end-user maintenance for a laser printer. Several Logitech 3D trackers (the small triangles in the Figure 1.5a) were attached to key components of the printer, allowing the system to monitor their position and orientation (7).

 

Figure 1.5 (a) Triangular trackers attached to various parts (b) Showing how to remove paper tray

2. AR in Aircraft industry

The Integrated Media Systems Center at University of Southern California is working with McDonnell Douglas engineers to create an AR system that will display text and graphics so aircraft assemblers at its Douglas Aircraft Co. facility in Long Beach can build planes more quickly and accurately. Such a system would help workers by relieving them of the need to refer back and forth to blueprints or instruction manuals.

At Boeing, David Mizell is using a grant from the Defense Advanced Research Projects Agency to try to use AR to simplify the process of bundling hundreds of wires. Traditionally, workers use foam boards with complicated pre-printed diagrams to lace the wires into a bundle. AR might allow a worker to use a blank board and rely on graphics in a head-mounted display to show where each wire should go.

1. MARS :A Mobile Augmented Reality Systems for Exploring the Urban Environment

This prototype system developed by the Computer Graphics and User Interfaces Lab, Columbia University acts as a campus information system, assisting a user in finding places and allowing to query information about items of interest, like buildings, statues, etc. (8). The user carries a backpack computer with a wireless network and wears a head-mounted display. The position of the user is tracked by differential GPS while orientation data is provided by the head-mounted display itself. As the user looks around the campus, the see-through headworn display overlays textual labels on campus buildings. The user can interact with the system to bring up related information about any building.

2. GRIDS (Geospatial Registration of Information for Dismounted Soldiers)

GRIDS is an Augmented Reality system currently under research, that can offer an intuitive, natural way for dismounted soldiers to understand electronic information (9). An infantryman wears a see-through Head-Mounted Display (HMD) hat which overlays computer graphics directly upon his view of the surrounding environment. The graphics are spatially registered with objects in the environment. A soldier wearing an HMD sees information labels from the tactical database directly superimposed over seen or unseen individuals and objects. The crucial requirement is that the graphic labels be properly registered to the correct objects in the environment. Accurate registration requires accurate tracking of the user's location and direction of gaze.

1. Mann, S., Smart Clothing: The "Wearable Computer" and WearCam, Personal Technologies, Vol. 1, No. 1, March 1997, Springer-Verlag [ ]
2. Sutherland, I., A Head-Mounted Three Dimensional Display Proc. Fall Joint Computer Conference 1968, Thompson Books, Washington, DC, 1968, 757–764. [ ]
3. This system was developed by the User Interaction and Visualization group at the European Computer-Industry Research Centre (ECRC), which has now been transferred to the Fraunhofer Project Group for Augmented Reality in ZGDV. [ ]
4. Ross T. Whitaker, Chris Crampton, David E. Breen, Mihran Tuceryan and Eric Rose, Object Calibration for Augmented Reality, ECRC-95-04, 1995. (Also appeared in Eurographics '95 Proceedings, pp. C-15 - C-27, September 1995). [ ]
5. Feiner, S., Webster, A., Krueger, T., MacIntyre, B., and Keller, E., Architectural anatomy, In Presence, 4(3), Summer 1995, 318-325. [ ]
6. Webster, A., Feiner, S., MacIntyre, B., Massie, W., and T. Krueger., Augmented reality in architectural construction, inspection and renovation, Proc. ASCE Third Congress on Computing in Civil Engineering, Anaheim, CA, June 17-19, 1996, 913-919. [ ]
7. KARMA, Computer Graphics and UI Lab, Columbia University, Jan 2000 http://www.cs.columbia.edu/graphics/projects/karma/karma.html
[ ]
8. Feiner, S., MacIntyre, B., Höllerer, T., and Webster, T., A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment, In Proc. ISWC '97 (Int. Symposium on Wearable Computers), October 13-14, 1997, Cambridge, MA. [ ]
9. GRIDS, Computer Graphics and Immersive Technologies Lab, University of Southern California, Jan 2000, http://www.usc.edu:80/dept/CGIT/ [ ]

Back To Contents

Part 2. System Design

The goal of the project was to develop an augmented reality system which uses a see through display combined with a tracking device to overlay upon the display, the services elements in the space occupied by the user and to provide links to an information database about the services. The database may provide the user with an element’s properties, its maintenance history, operation/repair instructions, etc. The services may be visible to the user, such as a lighting fixture, or may be hidden behind the building infrastructure, such as air-conditioning ducts behind a false ceiling. To better understand the project, consider the following scenario –

A facility manager is standing in a room wearing a see-through head-mounted display with a portable computer in his backpack and a small keyboard mounted on his wrist. When he turns on the headset display he gets icons on the screen for different services like electrical, lighting, HVAC, etc. Upon choosing an icon, for example, lighting, outlines of all elements related to the lighting system in the room are drawn on the head display overlapping over the actual view. As he turns around the display is continuously updated such that the outlines drawn on the display overlap the real world elements. He can click on any lighting fixture outline to determine its fixture schedule, the manufacturer’s contact information, etc. He can view mounting/dismounting instructions for the fixture. He can view the maintenance record to see when was the bulb last replaced. He can make any annotations for that fixture, which would be saved and can be retrieved any time in the future. He can determine how the wiring is running behind the wall and find out any other devices, e.g., a dimmer, which the fixture might be connected to. He can also immediately log in any maintenance work he carries out.

This location aware information can become a useful tool in various facility management/maintenance related applications. It can also be used as an educational tool. Rather than two-dimensional drawings, by observing how different services are running behind the infrastructure in a real space, students can have a better understanding of the building systems.

For implementing the project two separate modules were developed. One module for creating the 3D database and the second to view it on the augmented reality system. Figure 2.1 shows the flow diagram for the system. The first application interacts with a CAD application (AutoCAD in this case) to extract information from a drawing and link it to a database. The second application is used on the mobile computer. It extracts the geometrical information for the given space from the three-dimensional database of the building. After the geometry is loaded the position and direction of look for the user are continuously calculated based on the data from the tracker. Simultaneously, the geometry is transformed using transformation matrices based on location and direction data to produce an image that aligns with the objects in the real view. The following sections discuss in detail the various considerations in the choice hardware and the software design.

Back To Contents

The scope of the research did not include the design of any hardware. Commercially available products were integrated with the program. The critical components for any mobile computing project are the display unit, a tracking device, a mobile computer and an appropriate input device. Following sections deal with each element in greater detail.

Humans use the visual sense more than any other sense to process information. Hence the capabilities of the display device becomes more critical. The display device has to be able to provide necessary quality at an acceptable cost while minimizing the impact on the user. The main factors to be considered in choosing a display device for an augmented reality application are:

1. See through capability – A large number of HMD’s are available in the market today. However, the specific requirement for this project was a "see through" capable display. See through displays use special optics to allow for the viewer to be able to use the projected images as well as the surrounds. The transparency should be sufficient enough to allow the person to make out things easily and clearly under normal lighting conditions.
2. Display resolution – Resolution is the number of pixels in the horizontal and the vertical directions making up an image. The display resolution directly affects the quality of the graphics output. Smaller resolutions will result in pixelated images and would make it difficult for the user to distinguish between two separate entities.
3. Field of View – Field of View is defined as the angle that the horizontal and vertical edges of the display subtend with the eye. Ideally the field of view of a display device should be equal to the field of view f the human eye (around 180 degrees). However field of view for most current systems varies from 30 degrees to 60 degrees.
4. Colors – The display device for this application should ideally be color capable to allow greater flexibility and readability in the interface design.
5. Ergonomics - Since the device has to be worn by the user on the head for a long duration, it must be ergonomically designed so that it does not inconvenience the user, especially while carrying out any maintenance related work. It shouldn’t be too heavy to exhaust the user.
6. Durability – Since the implementation of the project is to be for maintenance applications, the display device should be rugged and be able to take jerks and perform well under dirty conditions.
7. Cost – The display unit should be able to provide the required functionality and quality in a reasonable cost for it to be commercially viable. Current HMD’s range from $399 up to $100,000.
8. Power Consumption – Since the system is mobile it has to operate on batteries, which makes the power consumption a very crucial factor.

A wide range of commercially available HMD’s were considered for the project. Such displays range from high-end, expensive products such as ProView 50 from Kaiser Electro Opticals, which have prices around $40,000 to $55,000, to medium-price systems such as the I-Glasses Protec or the Sony Glasstron700 which have prices above $1,000 but less than $5,000, to quite inexpensive consumer product displays costing less than $1,000, including the I-Glasses- LC, and VFX1. Monocular displays (1) have also been used in a variety of augmented reality applications. However, for the objectives of this project a see through display is ideal.

For the project an older model of I-Glasses LC from IO Display Systems was used as the HMD unit (Figure 2.3). The choice of I-Glasses as the display unit was mainly in order to keep the prototype systems cost as low as possible. While VGA resolution HMD’s are in the range of $2500 and higher, the I-Glasses used in the project ship for under $500. These glasses support NTSC mode and can be connected to the computer using a VGA to NTSC converter (shipped with the glasses). These glasses offer a field of view of 30 degrees with a resolution of about 180,000 pixels per eye, which roughly calculates to a resolution of around 256x230 pixels (2). Another positive feature of I-Glasses is the light weight (8 ounces) combined with a fairly rugged frame. Though the resolution is poor for a final application, it was sufficient to use the system as a testbed.

Tracking, also called Position and Orientation Tracking, is used where the orientation and the position of a real physical object is required. For augmented reality applications both position and orientation tracking of the viewer are critical. The First step in tracking is to locate the user in the space using position tracking. This requires the user’s Cartesian coordinates (x, y, and z) with respect to a reference point. Once the viewer has been located in the space the next step is to determine which direction the viewer is looking in. This requires the orientation to be specified by three angles known as pitch (elevation), roll, and yaw (azimuth). The complete position of the viewer is hence described. (Figure 2.4) For the system to be successful it should be able to determine these values continuously as the user may alter his position/orientation. This is called as six degrees of freedom (DOF).

Trackers are used to measure the motion of the user's head or hands, and sometimes eyes. Different technologies are available for tracking depending upon the application. For example, in the case of magnetic sensors, a receiver is placed on the user's head so that when the head moves, so does the position of the receiver. The receiver senses signals from the transmitter, which generates a low frequency magnetic field. The user’s head motion is sampled by an electronic unit that uses an algorithm to determine the position and orientation of the receiver in relation to the transmitter. In addition to magnetic head trackers there are mechanical, optical, acoustic (ultra-sonic), and inertial head trackers. These types of trackers also can be mounted on glove or body suit devices to provide tracking of a user’s hand or some other body part. Eye trackers work by measuring the direction at which the users’ eyes are pointed out of the head. This information is used to determine the direction of the user’s gaze. Eye trackers use electroocular, electromagnetic, or optical technologies.

One of the biggest challenges for augmented reality is the registration problem in correctly aligning real and virtual objects. The human eye easily catches even the slightest error in registration. Also, there is a lag in the time interval between measuring the head location and superimposing the corresponding graphic images on the real world because of which virtual objects may appear to swim around real objects.

The critical characteristics for a tracking device are -

• Resolution : Measures the exactness with which a system can locate a reported position. It is measured in terms of inch per inch of transmitter and receiver separation for position, and degrees for orientation.

• Accuracy . The range within which a reported position is correct. This is a function of the error involved in making measurements and often it is expressed in statistical error terminology as degrees root mean square (RMS) for orientation and inches RMS for position.

• System responsiveness . Comprises:

The tracking device shipped with the I-glasses was used as the orientation tracking device for the project. This tracker provides 3DOF (3 directions of freedom). It feeds the host computer application with the yaw, pitch and roll values for the glasses (Figure 2.5). All orientation descriptions are from the perspective of someone actually wearing the Tracker:

• Positive yaw is defined as a left head rotation.
• Positive pitch is an upward head tilt.
• Positive roll is a left head tilt.

The coordinate system used is +Y up, +Z out, and +X right (The positive axes can be formed with the right-hand index, middle finger, and thumb: a right-handed coordinate system (4).

The Tracker communicates with the host computer via an RS-232C 3-wire serial interface (TXD, RXD, GND). It can run at 1200, 2400, 4800, 9600, and 19200 bps and queried and tested using a standard ASCII terminal program. The tracker uses a magnetometer to measure the yaw and a two-axis inclinometer to measure the pitch and roll.

For lack of financial resources no position tracking device was used in this project. To work around the problem of locating a user, the application was setup to let the user calibrate his/her position in the space by using the cursor keys to move around till the augmented view matches the real view. Once calibration for a position is done the user can turn around that stationary point and experience the augmented view from that position.

As mentioned earlier the ideal computing device for this application is a wearable computer which provides maximum freedom to the user in terms of movement, ruggedness and ergonomics. Wearable computing is an area of current research at many places like the MIT Wearables lab, Georgia Tech Wearables, U. of Washington HIT lab. The main features of a wearable computer as compiled by the MIT Wearable lab are -

Portable while operational: The most distinguishing feature of a wearable is that it can be used while walking or while performing any task. This distinguishes wearables from both desktop and laptop computers.

Hands-free use: Military and industrial applications for wearables especially emphasize their hands-free aspect, and concentrate on speech input and heads-up display or voice output. Other wearables might also use chording keyboards, dials, and joysticks to minimize the tying up of a user's hands

Sensors: In addition to user-inputs, a wearable should have sensors for the physical environment. Such sensors might include wireless communications, GPS, cameras, or microphones.

Attention-getting: A wearable should be able to convey information to its user even when not actively being used. For example, if there is a new mail, the computer should be able to let the user know immediately you have new email and who it's from.

Always on: By default a wearable is always on and working, sensing, and acting. This is opposed to the normal use of pen-based "Personal Digital Assistants," which normally sit in one's pocket and are only woken up when a task needs to be done. (5)

A few commercial wearable computers like the Xybernaut are also available. (Figure 2.6). For this project, the focus was not on testing the ergonomics for the unit. So most of the development work was done on a Dell desktop computer with a 233 MHz Pentium II processor and an ATI Rage Pro graphics card with 8MB display RAM with OpenGl rendering capabilities. For mobile testing purposes a Compaq Armada 1750 notebook computer, with similar hardware specs, was used.

Though, no special input device was used for this thesis, the interaction methodology with the system forms a research topic in itself. The input device in a mobile computer gets very critical because it has to let the user interact with the system while he may be performing some other tasks. A few of the possible input devices that can be used are -

Keyboards – Two kinds of keyboards may be used in a mobile application

Arm/wrist mounted Keyboard – Small keyboards are available which can be easily mounted on the wrist. However, they are inconvenient due to the small size and only one hand can be used for typing.

Chordic Keyboards – Chordic keyboards have fewer number of keys and work by generating keys based on the combination of keys which are in pressed state. They are a little difficult to learn but work very well after practice. "Twiddler", a commercially available chordic keyboard has a built in tilt sensitive mouse as well.

Touchpad – A touch pad with handwriting recognition features can be another input device.

Speech – A good speech based input system can be very useful for this application. It can provide a completely hands free operation (6).

Software developed for the prototype system was broken into two modules. Figure 2 shows the overall structure for the software modules. The first module handles creation of the building geometry data and linking it to a database. The second module retrieves and displays this data based on the position and orientation of the user.

For maximum productivity it was best to integrate this module with an existing CAD application. AutoCAD is one of the most widely used CAD application and also provides support for customization using ADS or Active X. For this project ActiveX automation through Visual Basic was used. Though an ActiveX automation application is slower in execution than an ADS application, the time required for developing a ActiveX application is much shorter than that required for ADS (7). The choice of Visual Basic as the programming language was made for its RAD (Rapid Application Development) environment and robust database handling components. Microsoft Access 7.0 Database format was used for storing the database.

The program design was focused around a customizable database structure, which lets the user add/remove different element types and properties. Figure 2.11 shows the flow diagram for the working of the module.

On launching the application a user has the choice to start a new project or load an existing project (Figure 2.12). The program moves to the main data creation dialog with either an empty database or the database user had chosen to open. Figure 2.13 shows the configuration of the integration dialog.

The program also launches AutoCAD in the background and through ActiveX automation, opens the drawing file associated with the database. For a new project the program asks the user to choose the AutoCAD file that has to be associated with the project. A handle is created in Visual Basic for the AutoCAD document object. All subsequent interactions with AutoCAD are through the document handle. On successful launch of AutoCAD the user can proceed to creating the actual database.

The Project tree on the left side lists predefined categories for services. A user can add custom categories to it. By Clicking on the "Project" item in the tree the project’s details can be filled in. Project details including consultant information, contact information, etc. can be compiled here.

On clicking a service category the user gets an option to define a new element type or simply choose from a previously created database. That brings up the screen where data for the particular element can be viewed/edited. The data is divided into three tabs-General Properties, Details and Maintenance.

General Properties –

Properties of the element like dimensions, manufacturer, any contact information, remarks etc (Figure 2.16). To keep the interface customizable, the design for this tab is a simple two column grid where the left column identifies a property while the right column stores the property value. This gives the user freedom to define custom properties for any element. Currently there is a limit on the number of properties that can be defined for a single element. This value can be easily increased at a later stage and currently has been set to 50.

Details – The Details Tab shows directory and file trees. Using them the user can choose image files in .bmp, .gif, .jpg or .wmf file formats that would be displayed as a detail in the display module. The program currently supports these raster formats only except for .wmf. The advantage of raster files in this application would be that they can be rendered to the display much faster. However, .dwg or .dxf file support would be useful to provide greater flexibility to the user. Vector formats also power the user to be able to zoom into the details that can be useful on low-resolution display glasses.

Maintenance – The Maintenance tab has a simple three column grid control for date, comment and the supervisor filling in the log.

Because of their customizable nature the same tabs are used for all the service categories.

Once a service element type has been defined, the next step is to attach AutoCAD entities which have these properties. For example, once a fixture type has been defined the next step is to select all entities in AutoCAD which represent that fixture. There are 4 ways in which entities can be bound-

1. Bind By Block – Clicking on this button brings up a list of all the blocks that are defined in the file. After this all the insertions of that block in the drawing are bound to the current services element.
2. Bind By Attribute – This method again lets the user select a block to use and then brings up the blocks attributes. The user can choose a specific attribute value so that all the insertions of the block in the drawing with that attribute are bound to the current services element.



Figure 2.19 DataCreation Module - Binding Autocad Drawing Entities with Database

3. Bind by Layer – This option (as in Figure 2.19) brings up a list of the layers defined in the drawing. All objects on the chosen layer are bound to the current services element.
4. Bind by Selection – This option makes the application switch to AutoCAD and generates a selection prompt in AutoCAD. All the objects that are selected in AutoCAD are bound to the current element.

All bind buttons are additive, that means that each time any method is used the program still remembers the previously bound entities. There is also an unbind button to remove any entities bound to the current control. However, it cannot be used to remove selective entities, all entities are unbound.

Once the database has been filled in, clicking on the save 3d data button saves the database and the 3dmesh.

Rendering speed was the most critical factor in the display module to ensure minimum lag in updating the screen as the user changes position/orientation. For the 3D-development environment, OpenGl was chosen between OpenGl and DirectX, for its greater flexibility being platform independent. Since all the speed critical routines were in OpenGl, Visual Basic was taken as the front-end application. This allowed sharing of database handling sub-routines between the two modules.

This module loads the 3d geometry definition created by the data creation module and then starts polling the tracker. The user can use the cursor keys to position oneself accurately within the space. The program applies the view transformations based on the position as set by the user and the yaw, pitch and roll values as provided by the tracker. As the tracker keeps sending the new values the program keeps on updating the screen display. Because the objects are grouped by different service categories, for example, lighting, electrical, etc. the user can choose to turn on/off the display of a given object type. Also when the user clicks on a particular element, the program gets the information from the database file and displays it on the screen. Figure 2.20 shows the flow diagram for the display module.

To enable calibration of position, a keyboard handler routine has been defined. The user can move forward/backward and left or right with the cursor keys. Upward/Downward movement can be achieved using the cursor keys while holding gown the Ctrl key. For finer adjustments in the Yaw, Pitch and Roll the Y, P and R keys can be pressed along with shift to increase or decrease the value. Another factor that can be adjusted is the field of view. Best calibration with these glasses was normally achieved with a field of view of around 15 degrees. Figure 2.21 Shows an uncalibrated and calibrated view through the glasses.

Once calibration for a point is complete, the user can turn around and experience the augmented view. On the screen icons for the different service categories are also displayed which can be turned on or off by pressing the Tab key. The icons act as toggle buttons, where pressing an icon toggles display on/off for objects of that service category. Figure 2.22 shows the icons as they appear on the side of the screen.

When the user clicks anywhere on the display, the program searches through the display list to check which objects are close to the point of click. The closest object is then selected. The data for the object is pulled out from the Microsoft Access database and shown in the data properties dialog (Figure 2.23). Because of the low resolution of the glasses only a small amount of data which can be showed at a time. The icons for general information, maintenance history and details can be used to view the respective information.

Back To Contents

1. Monocular Displays are displays for one eye only. However, mostly they are "see-around" displays rather than "see-through". A small opaque display unit is positioned in front of one eye and the user can refer to images on that unit. See-around Monocular displays have been used in augmented reality systems which do not need to overlap any visual geometry over the real world but provide task/location based information. [ ]
2. The display resolution is 180,000 per eye. However, these displays use three color dots (triads) to make one pixel for 256x230 Computer addressable pixels. This resolution is good enough for the VGA 320x200 resolution, however for SVGA 640x480 small fonts are not readable. [ ]
3. Durlach, N. I., Psychophysical Considerations in the Design of Human-Machine Interfaces for Teleoperator and Virtual-Environment Systems, In Proceedings of Medicine Meets Virtual Reality II: Interactive Technology and Healthcare: Visionary Applications for Simulation Visualization Robotics, (pp. 45-47),1994, Aligned Management Associates. [ ]
4. For more information on coordinate systems see Computer Graphics, principles and practice, by Foley[ ]
5. Features of wearable computers taken from the Wearable Computing FAQ compiled by the Wearables Lab at MIT (http://wearables.www.media.mit.edu/projects/wearables/FAQ/FAQ.txt ), Jan 2000
[ ]
6. For an overview on practical uses of voice see Christopher Schmandt’s "Voice Communication with Computers," Van Nostrand Reinhold, New York [ ]
7. An ActiveX client application is a standalone Windows Application which uses ActiveX Automation interact with another windows application. ADS applications are native to AutoCAD and run within AutoCAD.
[ ]

Back To Contents

Part 3. Case Study

As the previous chapter outlines, the system design of the project was focused on creating an augmented reality experience in a given room. The goal was to provide the user with information relating to the various services-related elements in the room like air-conditioning ducts, lighting fixtures, etc. Once the system design had been evaluated and the component structure established, the next step was to choose a space that would act as the test bed. A 3d mesh of the space had to be made and linked with a database of the elements in the space.

3.1 Choosing the Case Study

For choosing the case study following factors had to be considered:

Service elements- The room should preferably have varied service elements in it to provide a challenging development environment and an interesting augmented experience. Multiple elements also allow for the testing of the system for issues relating to clarity of display, resolution and interface design.

Shape/form – Rectangular/cuboidal geometry for the room was considered favorable to keep the defining mesh simple. Simple geometry would also make it easier to the calibrate the users position when the application is started.

Size – The low resolution and field of view of the glasses make it necessary that the room be neither too big or too small. In a small room, because of the very small field of view the augmented area visible to the user would be too small and would make precise calibration difficult. In a large space the distant objects would appear closer. Due to the low resolution it would become difficult to differentiate between separate elements rendering the system ineffective.

Lighting – See-through glasses though fairly transparent, do darken the view through them. In a poorly lit room it would be difficult to see through clearly and would hinder proper evaluation of the system.

Availability of Information – Sufficient information to build a comprehensive database of the services present in the room should be available. This information could be in the form of architectural/ construction drawings, electrical drawings, lighting schedule, manufacturer catalogs, etc.

Access – The test room should have an easy access from the development station for convenience in tweaking the software during the test phase.

The computer lab for the Buildings Science program was chosen as the test space for the project. The rectangular room (41’x16’) offers a fair amount of complexity in terms of the lighting, electrical wiring and ductwork around the room. Also, since the program was being developed in the computer lab itself, it allowed immediate testing of the program in the same space. Any required tweaking and recompiling was easily possible thus saving time.

Keeping the above factors in mind, a few of the rooms in the architecture department were evaluated. The computer lab for the Building Science program was chosen as the test space for the project (Figure 3.1). The rectangular room (41’x16’) offers a fair amount of complexity with different lighting fixtures, electrical/ data cabling and ductwork around the room. The room is fairly well lit with natural as well as artificial light. With a recent renovation project, updated drawings for the ducting system were also readily available. Also, since the program was being developed in the computer lab itself, it allowed immediate testing of the program in the same space. Any required tweaking and recompiling was easily possible, thus saving time.

Figure 3.1 The MBS lab used as a case study

Th room has a false ceiling with some electrical ducts running above it. The main duct for air condition is behind the west wall, while a heating coil is located on the east wall. Electrical conduits also run through both the east and west walls. There are a few conduits which were recently added to provide connections for the various track/suspended lights and occupancy/daylight sensors.

Back To Contents

3.2 Creating the Database

Measurements for the room were taken in and drawings for various services like lighting, air conditioning, heating, etc. were collected. All this information was used to create a 3d model of the lab in AutoCAD (Figure 3.2). The geometry was kept as a bounding box encompassing the objet, to keep rendering loads to a minimum.

Figure 3.2 The 3D Model for the MBS Lab made in AutoCAD

The data creation module treats services under categories of– lighting, electrical, telecom, HVAC, plumbing, structural system and finishes. The elements in the lab were categorized based on this categorization defined by the module. The data module allows binding of entities to a database based on layer, block, block attribute or by selecting entities. Based on the kind of entities being used to create an element one can use the appropriate binding method. For example, elements that are created using blocks can be linked to a data type with the bind by block button. For elements like electrical ducts that are unlikely to be blocks, can be easily bound by keeping them in a separate layer and using the bind by layer feature.

For this project, all entities for lights were created in a layer "light". Different blocks were used to differentiate between the different elements within a layer. For example, all Downlights were created as a block "DL" while all tracklights were created as a block "TR". This allowed easy association of each element to the database using the bind by block feature in the data creation modules.

Back To Contents

3.3 Testing the System

The first step is the required calibration so as to achieve minimum error in overlap of the augmented and real views. To make the calibration process easier, a spot had been marked on the floor which corresponded with the viewpoint defined in the database. The initial phases of calibration required a lot of experimentation with the different viewing transformations possible depending upon the location, field of view and the three directional settings. Once the appropriate filed of view was determined calibration became easier with minor tweaking requirements. Minor calibration of Yaw, pitch and roll values was also required for different users because of the difference in the way the headset sits on different users heads.

Figure 3.3 Testing the system in the MBS lab

Once the system gets calibrated for the spot fairly accurate results were observed. On a complete turnaround through 180 degrees there was a slight mismatch because of the slight shift in position of the user. However, the shift wasn’t big enough to prevent association of the augmented objects and the real objects. The results were same for different users except for certain users who had a difficulty in calibrating the view.

Any input was very difficult without a mobile input device. There was also a difficulty in coordinating the focus between the mouse cursor and the background object, when attempting to select the object. The data that comes on selecting an object was difficult to read initially and the font sizes had to be increased substantially. Experimentation was also carried out with the thickness of the wireframe for the objects. With thiner wireframes the objects would tend to get lost in the background. Thicker wireframes made it easier to view the scene. However, because of the low display resolution it merged the display of close objects

.

Figure 3.4 Showing a thin wireframe view getting lost in backdrop

Figure 3.5 Showing a thicker wireframe

Back To Contents

Part 4. Conclusions and Analysis

4.1 Problems Encountered

Tracking

The lack of position tracking proved to be the biggest hindrance in an accurate overlap of the geometry over the real world. Without position tracking a user needs to spend time in calibrating their position in the room by moving the view position through the cursor keys. Without accurate position tracking such an application can result in giving inaccurate information to the user. The resolution of the tracking device was another factor. With a small field of view the trackers directional resolution was not sufficient to ensure a smooth motion as the user turns around. The slow access speed also produced a lag between a users movement and the updating of the display.

Display

The I-glasses are not designed for VGA viewing and therefore have a very poor display resolution. This implies that any text that has to be viewed on the display screen has to be larger than 14 point. This limits the amount of information that can be presented to the user at a time. The low resolution also affects the quality of drawings for that can be shown for maintenance/operation.

Speed

Though computing powers have grown manifold, in an augmented reality system the demands are very high. As the complexity of the mesh increase the rendering time increases which results in slower updates of the screen and hence a lag between the actual and augmented objects.

Back To Contents

4.2 Future Work

With better tracking technology available now fairly accurate and fast position tracking can be achieved providing the user with 6DOF (6 directions of freedom). This would avoid problems of the user having to calibrate their position in the space. The project should definitely be tested with a position tracking device.

Another area of further research is on the display of the 3d mesh. Experimentation can be carried out with different modes for displaying the geometry from wireframe to flat shaded to rendered meshes. The different modes can be compared with respect to the clarity of displayed information and the corresponding rendering time.

The user interface design for the software is another area which needs a tremendous amount of work. The way a user would input data or the way information from the database would be presented to the user is very critical for success of this application. How a user would navigate through the information becomes critical.

Other applications of such an augmented reality system in architecture also need to be explored. A few such applications could be – as a teaching aid, as a visualization tool on site by rendering a 3d model overlapping with the empty site, as a information resource for visitors in a public building like a museum, etc.

Back To Contents

4.3 Conclusions

The emphasis of the project had been on testing the feasibility of using augmented reality for an architectural application in facility management/maintenance. The prototype system developed in this thesis has shown demonstrated that such an application can be very successful. The system has achieved fairly accurate results while using existing low cost hardware. The data creation process also uses off the shelf software, AutoCAD that is widely used in CAD applications. Existing drawings of a project can be built upon and used for the data creation module.

However, the tests also show that the technology in its present form is still not completely ready for the application. The tracking speeds and accuracy must increase with a decrease in the cost and installation complexity. Also it would be feasible only after price for higher resolution displays comes to an affordable level.

Current research on augmented reality is focused more on fields other than architecture. One of the big sponsors for current research is the aircraft industry. The complexity of engineering drawings for engines makes it very difficult for the workers working on the repair or assembly of aircraft engines. In such a situation augmented reality can be a very useful tool to provide help to the workmen. This application comes very close to the application scenario discussed in this thesis. This is an application that would require very precise tracking and also issues relating to the interface design for presenting information to the user and getting his input without restricting his working capability. With the similarities of the aircraft industry application and the architectural application discussed in this project, use of augmented reality for architecture will gain from any advances in this field.

Another project GRIDS, discussed earlier in the report, also has relative applications for architecture. The GRIDS project is a military application, which is striving for getting accurate tracking in, an open space and providing information for the surroundings to soldiers dismounted in an alien territory. In architecture advancements in this technology will enable systems which can be used for site surveying, for determining location of underground pipes, cables etc. by using the tracking information combined with a 3d data for the underground infrastructure. It can also be used as a visualization tool where a user can be tracked on a site and a building projected on his display to visualize how the building responds to the surroundings.

In conclusion, augmented reality is a developing and still untapped field that can find many applications in architecture from visualization to facility management to restoration and architectural education.

Back To Contents