Dohi, T, Sakuma, I, & Liao, H. (Eds). (2008). Medical Imaging and Augmented Reality: 4^th International Workshop Tokyo. Berlin Heidelberg: Springer-Verlag.

Doswell, J.T. (2006). Augmented Learning: Context-Aware Mobile Augmented Reality Architecture for Learning. ICALT Proceedings of the Sixth IEEE International Conference on Advanced Learning Technologies. 1182-1182.

Guo, Y, Du, Q, Luo, Y, Zhang, W & Xu, L. (2008). Application of Augmented Reality GIS in Architecture. The International Archives of the Photogrammetry, Remote Sensing and Spacial Information Sciences. XXXVII (B5) 331-336.

Höllerer, T, H. & Feiner, S, K. (2004). Mobile Augmented Reality. In Karimi, H. & Hammad, A (Ed.), Telegeoinformatics:Location-Based Computing and Services. London: Taylor and Francis Books Ltd.

Milgram, P, Takemura, H, Utsumi, A, & Kishino, F. (1994). Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum. SPIE Proceedings: Telemanipulator and Telepresence Technologies, 2351, 282-292.

State, et al. (1994). Case Study: Onserving a Volume Rendered Fetus within a Pregnant Patient. Proceedings of IEEE Visualisation ’94, 364-368.

Webster, A, Feiner, S, MacIntyre, B, Massie, W & Krueger, T. (1996). Augmented Reality in Architectual Construction, Inspection and Renovation. Proceedings ASCE Third Congress on Computing in Civil Engineering. Anaheim, CA. 913-919.

Zudilova-Steinstra, E, Adriaansen, A & Van Liere, R. (2009). Trends in Interactive Visualization: State-of-the-Art Survey. London: Springer.

The Columbia Tourng Machine. Feiner, MacIntyre, Höllerer & Webster (1997)

Early Prototypes & Projects

The oldest tourism-related use of mobile augmented reality dates back to the end 90′s, when wearable computing and tracking devices became small enough to take them to the streets. On the left you can see a slightly more recent version of this early MARS prototype  that was designed as a 3D graphical tour guide for the Columbia Universtiy campus and provided its wearer with information about the various campus buildings and objects, including restaurants. The left image (a) shows the user, with his mobile wearable computing system and the right (b) displays what he sees through his goggles (Höllerer, 2004). Admittedly, small enough is rather relative and no tourist would be caught dead walking through an unknown city with such a device.

AR for Travel Preparation

AR on the Road

AR at the Destination

Romeiß-Stracke, F. (1989). Neues Denken im Tourismus. München: ADAC. s, 12.

Despite its ancient quality, many of my teachers are somehow fond of showing this image to explain to us what a holiday consists of and which sectors therefore need to be considered for as touristic services. Augmented reality Apps cover all of these fields and more. There are apps for traveling entire countries, for hotels, mountain viewing, cultural heritage, city guides (including a Monopoly Tour of London), tourist attractions, accommodation in general, the Berlin wall montioned before, some things in Japanese I can’t read, “Nice Spot“s, restaurants, cafe’s, clubs. The list goes on.

Conclusion

Whereas AR research from the 90′s focuses on pc’s with webcams and HMDs, the 21st Century brings forth a whole new branch of AR. The former is referred to as Marker based AR: the webcam of the HMD recognizes a special symbol (the marker) from which the computer generated imagery appears.  The latter is Location based (mobile) AR: users can observe the augmented reality through their mobile devices, without the need for markers. Mobile AR is not strictly bound to the use of mobile phones! Goggles (or possibly even lenses) can also be used. The point of Mobile AR is that markers become obsolete, because the augmented content is provided based on the geographical location of the user of the mobile device, whether mobile hand-held or head-worn displays.

To facilitate Mobile AR, specific hard- and software is needed, generally referred to as MARS:

“AR systems integrate virtual information into a person’s physical environment so that he or she will perceive that information as existing in their surroundings. Mobile augmented reality systems (MARS) provide this service without constraining the individual’s whereabouts to a specially equipped area” (Höllerer & Feiner 2004, p. 1).

One needs:

• a computational platform
• displays
• registration
• wearable input and interaction technologies
• wireless networking
• data storage and access technology

The details for these challenges are laid out Höllerer & Feiners (2004) article Mobile Augmented Reality. This gives an interesting insight into this phase of Mobile AR development, where the possibilities spoke to the imagination of many, but much progress was to be made. Still today, the Augmented World is far from perfect. However, much progress has been made and “widespread commercial adoption” is finally taking off. Below, recent developments in both Mobile hand-held, and head-worn displays are presented and discussed.

Mobile hand-held displays

Besides promoting equality or capitalizing on the hype, there are also those who contribute to Mobile AR development to unleash their creativity. Layar seems to be the central player in this field, apparently having conquered the terrible hurdles that lay before the 2004 generation of Mobile AR developers. Layar has been clever enough to address equity and creativity in their corporate conduct, providing the first AR browser for which creative individuals, but also companies large and small, publishers and advertising agencies can design their own Mobile AR apps. The platform functions on iphone and android operating systems and there are no licence costs involved creating a layer. There are already hundreds of apps available. The big internet names are in on the deal, with the youtube app showing nearby youtube video’s, the “Tweeps Around” app for locating nearby twitter users and entering geo-tagged tweets. But also more original, unexpected apps are to be found such as Swiss Paragliding and Excile on Your Street. Since the 18th of June 2010, Layar has deals around the globe to make its AR services compatible with one third of the worlds smartphones.

Alongside Layar, the open source community is also in the picture, providing various platforms that run on open source software for the development of Layar apps. Porpoise is made especially to convert data sets of POIs (Points of Interest) into Layar compatible data and Openhanced has made an @layarmobile starter kit, aiming to open up access to the world of Layar layer creation for even the less tech-savvy among us. From what one sees on the Layar catalog, most apps are of the kind that shows 1D (points) or 2D (image) related AR, like youtube video’s or tips for good restarants nearby. Of course, its the 3D, fully functioning immersed and interactive superman flying through our lives that everyone is waiting for. Until now, all I could find is the UFO flying through the Layar catalog page.

Mobile head-worn displays

Another mind-boggling idea is the modernization of old goggles-with-backpack to flashy sci-fi movie style goggles like this:

CAARS consumer goggles as depicted in Doswell (2006)

If you look closely you may notice the tiny camera’s at the sides. Speakers are also built in. The idea of course, is that, with these goggles, the user can see virtual imagery blended into the otherwise perceived environment (Doswell, 2006). When my professor mentioned this possibility in class, I thought it was a silly idea, because I could not imagine reading data from my own glasses. For others who have difficulty imagining how this technique could work, an Italian architecture student has provided the world with a enchanting forecast of the use of Mobile AR goggles in this video:

My ethics professor once told the class that “tourism is everything”. Is it also augmented reality? Read up on how the two come together to change travelling experiences and perhaps, the industry too in my final entry on this topic for now: AR in Tourism.

Medicine

A possible use of AR in the field 0f Medicine would be combining 3-D datasets of a patient with a real time view to assist doctors excecuting minimally-invasive surgery; a type of surgery that uses no or only very small inscisions making it difficult for doctors to see inside the patient. AR would solve this problem of visualisation by providing a 3-D internal view of the patients based on non-invasice sensors such as Magnetic Resonance Imaging (MRI) or Computed Tomography (CT). Recently, scientists have refined this technique for use in minimally invasive cardiac surgery (Dohi, Sakuma & Liao. (2008).

Most medical applications concern image guided surgery for example for general visualisation assistance such as targeting for an inscision and needle injection, or displaying instructions to medical students at work with a patient eliminating the need to glance aside in a manual, or viewing a fetus through HMD (Azuma, 1997; State, et al. 1994).

Marketing

Here is on of the most viewed video’s that explains augmented reality.

One might assume that such a helpful demonstration would come from either an educational institution or a AR technology developer. However, this shot clip posted by plugintothesmartgrid is actually provided by General Electric, as a means to lure those curious about augmented reality to their website on renewable energy.

Indeed, companies have caught on to the hype and flashy, so called ARM (Augmented Reality Marketing), is popping into the real-world like algae on the Baltic Sea. Whereas AR was originally explored by military, education and scientific institutions, businesses’ marketing departments are the ones to deliver AR uses fit for mass consumption.  Automobile companies have been especially busy: MINI marked the back of a magazine that, when held in from on a webcame, will display a 3D Mini.

Adidas, also had to be one of the first, placing a market on the front of their shoe, footware is know also used as a joystick. For an overview of these and other companies using ARM, check out this blog entry of webdistortion, an IT company that uses its prevalent high involvement in the theme to draw those interested in ARM, in turn, to their website.

On the topic of Marketing, a fellow blogger has noted that despite the hype of AR for consumer use, most companies’ applications lack an element of interaction deemed necessary to hold the consumers attention. The space3dsolutions blog puts it this way: “there are a lot of fairies turning on the top of fingertips”. Both name Total Immersion as a positive example because of their creation of holograms that the user can manipulate and interact with. Space3dsolutions explains that there are many technical problems related to the ability to track moving objects with the current quality of most webcams. The first author claims that gaming and multi-media will be the fields in which AR will be most attractive from a marketing perspective, because they can answer to the demand for interactive uses of AR that will grow steadily as the novelty of the technology wears off. However, a lot of hard work on perfectionizing the details will be necessary before the virtual imaging supplements the real-world environment smoothly enough to offer high-quality consumer products that remain entertaining in the long run.

Media

Other uses of AR that quickly gained attention are Esquire Magazine’s cover with an AR Marker. Throughout the magazine, multiple AR entries can be found. All are interactive, and can be controlled by the reader by changing the angle of the marker relative to the camera, or holding it up at a different point in time.

Where most uses I have mentioned entail the virtual entering the previously virgin real-world environment, there is one sector that moves in the opposite direction: gaming. In the Wearable Computer Lab at the School of Computer and Information Science at the University of South Australia researchers, under- and postgraduate students have are developing ARQuake. If you prefer spaying virtual paint over virtual bullets, TagDis is your game. With this AR app, you can leave a trace of your own virtual graffiti tags wherever you go. Tagdis treads slightly into the real of social gaming, as others may tag the same location by getting more views, kick you off your throne of king and owner of the territory. Combining motion capture technology, Flash and social networking,   Cannonballz from Zugara offers users the opportunity to collaboratively procrastinate by throwing cannonballs at their Facebook friends. The video gives a straightforward demo.

Soon, busy parents trying to get their children away from gaming and back to books, but lacking the time for a good bedtime story, may turn to the use of augmented reality for children’s books for support. That is, if they don’t mind the Magic Books reading themselves to their child. Probably, the next generation kids will be far from secretly reading books or playing games alone under their blankets. They might be chasing the monsters out from under their beds. Monsters they really do see in this case. All they need is a phone or a pair of goggles. Unlikely? Find out more about Mobile AR.

Architecture

Augmented Round Table for Architecture and Urban Planning

In the field or Arhcitecture and Urban Planning, using 3D models on a 2D interface is insufficient to grasp the subtle details of the buildings’ spacial relations. Therefore, students and professionals need real-world 3D models, which they meticulously assemble; a highly time and material resource intensive practice. Now, with the use of AR, the programmed structures can be visualized in 3D, with the use of HMDs to overlay sound and graphics to the planners natural senses. These overlays remain in place, even while the person moves around, because the HMDs allow for the location of the person to be tracked. Note that the model is interactive, enabling the planners to directly adjust and modify the structures in real-time. Also, the interior structure of buildings could be seen, much as if the viewer would have x-ray vision. (Webster, 1996; Guo, 2008)

Education

The field of education has likewise expressed high interest in developing AR for its purposes. Brett E. Shelton explores the potential for learning with AR,  for example using markers and HMD so that textbooks could gain an interactive pop-up quality (left). This way, students are able to interact with 3D models of their subject. Rather than having to figure out how all those lines on the page are supposed to be our solar-system in action, they can actually observe how the planets circle around the sun.  CSIRO has been researching AR for educational purposes, focusing on using virtual objects as the center of learning interaction (middle, right).

More recent research at CSIRO has been focused on using this technology for long distance interaction between people solving a task together. This is part of their research on Interactive Visualization (IV), a young research field created “to develop new scientific methods to increase person’s abilities to explore and understand the data, so that an increased awareness of meaning in the data is possible” (Zudilova-Steinstra, Adriaansen & Van Liere, 2009).

Arguing that the use of mobile AR would do more justice to equity, researchers at the Harvard Graduate School of Education, the University of Wisconsin at Madison and the Teacher Education Program at MIT, have joined forces in HARP: a Handheld Augmented Reality Project. The video below gives a short introduction to the project and the philosophy behind it.

The technique used in HARP is also known as augmented learning. In augmented learning, the students learning environment dynamically adapts to them, offering additional visual and/or audio information. Commonly, this information is shown as pop-ups on a computer screen. However, as in HARP, more and more researchers have taken an interest in the use of MARS’ (Mobile Augmented Reality System). Researcher Jayfus T. Doswell claims that “MARS has the potential to adapt to individual learning needs and dynamically distribute tailored instruction to improve learning performance for a lifetime” (Doswell, 2006). Wheather on a pc or a mobile device, a key aspect of augmented learning is that information is given to students on a need to know basis, as opposed to conventional learning approaches where information is memorised prior to excecuting a task. It may be argued that this could have negative effects on students memory skills. But, in a society where information is in abundance and knowledge is rapidly outdated, the augmented learning approach supplies students with experience in dealing with this high turnover of information,  equipping them for a future in which the skill of dealing with data will be in high demand.

Art

The playful, interactive element of AR makes it an attractive technology for artists who like to involve the public in their work, to let them discover and reflect upon different corners of contemporary society. In an exposition end 2009, artists at swissnex in San Fransisco explored the use of AR in this manner, adding “new experiences to everyday objects” such as storybooks and t-shirts. See what they came up with here:

Augmented Reality (AR) is a variation of Virtual Reality (VR) (also know as Virtual Environments (VE)). To „augment“ means to supplement. Accordingly, AR suppliments reality with virtual, computer-generated imagery. As opposed to VR, where the user sees only the virtual world, AR allows users to see virtutal objects in a real-world environment. „Augmented Reality enhances a user’s perception of and interaction with the real world“ (Azuma, 1997, p. 3). For those familiar with the movie „Who Framed Roger Rabbit“, one could say the well-functioning AR should achieve the level of fluent blending of the real with the virtual as exhibited in this movie. (Azuma, 1997)

Milgram’s reality-viruality continuum

In the famous paper „Augmented Reality: A Class of Displays in the reality-virtuality-continuum“, Milgram, Takemura, Utsumi & Kishino (1994) introduce the different components of what they call Mixed Reality, placing AR one step down the line moving from the Real Environment to a totally Virtual Environment, as shown in the figure below:

AR on the RV continuum

A visualisation of AR

Azuma, in his fundamental piece „A Survey of Augmented Reality“, uses the following figure to visualise what it may look like when synthetic 3-D objects are placed in a real 3-D environment, explaining that they are „combined in 3-D, so that the virtual lamp covers the real table and the real table covers parts of the two virtual chairs“ (Azuma, 1997, p. 2). In the figure, the lamp and table are virtual, whereas both the table and phone are real life objects.

Real desk with virtual lamp and two virtual chairs

Defining AR

In his survey, Azuma (1997, p. 2) defines AR as having „the following three characteristics:

1. Combines real and virtual
2. Interactive in real time
3. registered in 3-D“

He justifies this definition by explaining that it does not limit AR to specific technologies, such as Head-Mounted-Displays (HMDs), but at the same time rules out film or 2-D overlays because they either lack interactivity, such as the film „Jurrasic Park“, or the feature of having been combined with the real world in 3-D (demonstrated in the figure 2 above). With his definition, Azuma leaves the real of AR open to a number of technologies, such as monitor-based interfaces, monocular systems and see-through HMDs (Azuma, 1997).

What good is it to start „mixing realities“? Find out more about how AR can help people of various professions to perform real-world tasks in the diverse fields of application of AR.

This blog is made as an university assignment and may be useful as an introduction to AR for those priorly less informed. More in depth and expert information can be found through the Literature list.