augmented reality not to be confused with virtual reality virtual fixtures first ar system 1992 us air force wpafb augmented reality ar is an interactive experience of a realworld environment where the objects that reside in the realworld are augmented by computergenerated perceptual information sometimes across multiple sensory modalities including visual auditory haptic somatosensory and olfactory augmented reality is related to two largely synonymous terms mixed reality and computermediated reality the primary value of augmented reality is that it brings components of the digital world into a persons perception of the real world and does so not as a simple display of data but through the integration of immersive sensations that are perceived as natural parts of an environment the first functional ar systems that provided immersive mixed reality experiences for users were invented in the early 1990s starting with the virtual fixtures system developed at the us air forces armstrong laboratory in 1992 another example is an ar helmet for construction workers which display information about the construction sites augmented reality ar is used to enhance natural environments or situations and offer perceptually enriched experiences with the help of advanced ar technologies eg adding computer vision and object recognition the information about the surrounding real world of the user becomes interactive and digitally manipulable information about the environment and its objects is overlaid on the real world this information can be virtual augmented reality also has a lot of potential in the gathering and sharing of tacit knowledge augmentation techniques are typically performed in real time and in semantic context with environmental elements immersive perceptual information is sometimes combined with supplemental information like scores over a live video feed of a sporting event this combines the benefits of both augmented reality technology and heads up display technology hud technology a microsoft hololens being worn by a man hardware hardware components for augmented reality are processor display sensors and input devices modern mobile computing devices like smartphones and tablet computers contain these elements which often include a camera and mems sensors such as accelerometer gps and solid state compass making them suitable ar platforms display various technologies are used in augmented reality rendering including optical projection systems monitors handheld devices and display systems worn on the human body a headmounted display hmd is a display device worn on the forehead such as a harness or helmet hmds place images of both the physical world and virtual objects over the users field of view modern hmds often employ sensors for six degrees of freedom monitoring that allow the system to align virtual information to the physical world and adjust accordingly with the users head movements in january 2015 meta launched a project led by horizons ventures tim draper alexis ohanian boe optoelectronics and garry tan eyeglasses vuzix ar3000 augmentedreality smartglasses ar displays can be rendered on devices resembling eyeglasses versions include eyewear that employs cameras to intercept the real world view and redisplay its augmented view through the eyepieces hud headset computer see also headup display a headup display hud is a transparent display that presents data without requiring users to look away from their usual viewpoints a precursor technology to augmented reality headsup displays were first developed for pilots in the 1950s projecting simple flight data into their line of sight thereby enabling them to keep their heads up and not look down at the instruments neareye augmented reality devices can be used as portable headup displays as they can show data information and images while the user views the real world many definitions of augmented reality only define it as overlaying the information crowdoptic an existing app for smartphones applies algorithms and triangulation techniques to photo metadata including gps position compass heading and a time stamp to arrive at a relative significance value for photo objects a number of smartglasses have been launched for augmented reality due to encumbered control smartglasses are primarily designed for microinteraction like reading a text message but still far from more wellrounded applications of augmented reality in january 2015 microsoft introduced hololens an independent smartglasses unit brian blau research director of consumer technology and markets at gartner said that out of all the headmounted displays that ive tried in the past couple of decades the hololens was the best in its class contact lenses contact lenses that display ar imaging are in development these bionic contact lenses might contain the elements for display embedded into the lens including integrated circuitry leds and an antenna for wireless communication the first contact lens display was reported in 1999 the futuristic short film sight many scientists have been working on contact lenses capable of many different technological feats the company samsung has been working on a contact lens as well this lens when finished is meant to have a builtin camera on the lens itself the design is intended to have you blink to control its interface for recording purposes it is also intended to be linked with your smartphone to review footage and control it separately when successful the lens would feature a camera or sensor inside of it it is said that it could be anything from a light sensor to a temperature sensor in augmented reality the distinction is made between two distinct modes of tracking known as marker and markerless marker are visual cues which trigger the display of the virtual information virtual retinal display a virtual retinal display vrd is a personal display device under development at the university of washingtons human interface technology laboratory under dr thomas a furness iii several of tests were done in order to analyze the safety of the vrd in one test patients with partial loss of vision were selected to view images using the technology having either macular degeneration a disease that degenerates the retina or keratoconus in the macular degeneration group 5 out of 8 subjects preferred the vrd images to the crt or paper images and thought they were better and brighter and were able to see equal or better resolution levels the kerocunus patients could all resolve smaller lines in several line tests using the vdr as opposed to their own correction they also found the vdr images to be easier to view and sharper as a result of these several tests virtual retinal display is considered safe technology virtual retinal display creates images that can be seen in ambient daylight and ambient roomlight the vrd is considered a preferred candidate to use in a surgical display due to its combination of high resolution and high contrast and brightness additional tests show high potential for vrd to be used as a display technology for patients that have low vision eyetap the eyetap also known as generation2 glass captures rays of light that would otherwise pass through the center of the lens of the eye of the wearer and substitutes synthetic computercontrolled light for each ray of real light the generation4 glass handheld a handheld display employs a small display that fits in a users hand all handheld ar solutions to date opt for video seethrough initially handheld ar employed fiducial markers games such as pokmon go and ingress utilize an image linked map ilm interface where approved geotagged locations appear on a stylized map for the user to interact with spatial spatial augmented reality sar augments realworld objects and scenes without the use of special displays such as monitors headmounted displays or handheld devices sar makes use of digital projectors to display graphical information onto physical objects the key difference in sar is that the display is separated from the users of the system because the displays are not associated with each user sar scales naturally up to groups of users thus allowing for collocated collaboration between users examples include shader lamps mobile projectors virtual tables and smart projectors shader lamps mimic and augment reality by projecting imagery onto neutral objects providing the opportunity to enhance the objects appearance with materials of a simple unit a projector camera and sensor other applications include table and wall projections one innovation the extended virtual table separates the virtual from the real by including beamsplitter mirrors attached to the ceiling at an adjustable angle virtual showcases which employ beamsplitter mirrors together with multiple graphics displays provide an interactive means of simultaneously engaging with the virtual and the real many more implementations and configurations make spatial augmented reality display an increasingly attractive interactive alternative an sar system can display on any number of surfaces of an indoor setting at once sar supports both a graphical visualization and passive haptic sensation for the end users users are able to touch physical objects in a process that provides passive haptic sensation tracking modern mobile augmentedreality systems use one or more of the following motion tracking technologies digital cameras andor other optical sensors accelerometers gps gyroscopes solid state compasses rfid these technologies offer varying levels of accuracy and precision the most important is the position and orientation of the users head tracking the users hands or a handheld input device can provide a 6dof interaction technique networking mobile augmented reality applications are gaining popularity due to the wide adoption of mobile and especially wearable devices however they often rely on computationally intensive computer vision algorithms with extreme latency requirements to compensate for the lack of computing power offloading data processing to a distant machine is often desired computation offloading introduces new constraints in applications especially in terms of latency and bandwidth although there are a plethora of realtime multimedia transport protocols there is a need for support from network infrastructure as well input devices techniques include speech recognition systems that translate a users spoken words into computer instructions and gesture recognition systems that interpret a users body movements by visual detection or from sensors embedded in a peripheral device such as a wand stylus pointer glove or other body wear products which are trying to serve as a controller of ar headsets include wave by seebright inc and nimble by intugine technologies computer the computer analyzes the sensed visual and other data to synthesize and position augmentations computers are responsible for the graphics that go with augmented reality augmented reality uses a computergenerated image and it has an striking effect on the way the real world is shown with the improvement of technology and computers augmented reality is going to have a drastic change on our perspective of the real world computers are improving at a very fast rate which means that we are figuring out new ways to improve other technology the more that computers progress augmented reality will become more flexible and more common in our society computers are the core of augmented reality software and algorithms a key measure of ar systems is how realistically they integrate augmentations with the real world the software must derive real world coordinates independent from the camera from camera images that process is called image registration and uses different methods of computer vision mostly related to video tracking many computer vision methods of augmented reality are inherited from visual odometry usually those methods consist of two parts the first stage is to detect interest points fiducial markers or optical flow in the camera images this step can use feature detection methods like corner detection blob detection edge detection or thresholding and other image processing methods augmented reality markup language arml is a data standard developed within the open geospatial consortium ogc which consists of xml grammar to describe the location and appearance of virtual objects in the scene as well as ecmascript bindings to allow dynamic access to properties of virtual objects to enable rapid development of augmented reality applications some software development kits sdks have emerged development the implementation of augmented reality in consumer products requires considering the design of the applications and the related constraints of the technology platform since ar system rely heavily on the immersion of the user and the interaction between the user and the system design can facilitate the adoption of virtuality for most augmented reality systems a similar design guideline can be followed the following lists some considerations for designing augmented reality applications environmentalcontext design context design focuses on the endusers physical surrounding spatial space and accessibility that may play a role when using the ar system designers should be aware of the possible physical scenarios the enduser may be in such as public in which the users uses their whole body to interact with the software personal in which the user uses a smartphone in a public space intimate in which the user is sitting with a desktop and is not really in movement private in which the user has on a wearable by evaluating each physical scenario potential safety hazard can be avoided and changes can be made to greater improve the endusers immersion ux designers will have to define user journeys for the relevant physical scenarios and define how the interface will react to each especially in ar systems it is vital to also consider the spatial space and the surrounding elements that change the effectiveness of the ar technology environmental elements such as lighting and sound can prevent the sensor of ar devices from detecting necessary data and ruin the immersion of the enduser another aspect of context design involves the design of the systems functionality and its ability to accommodate for user preferences while accessibility tools are common in basic application design some consideration should be made when designing timelimited prompts to prevent unintentional operations audio cues and overall engagement time it is important to note that in some situations the applications functionality may hinder the users ability for example applications that is used for driving should reduce the amount of user interaction and user audio cues instead interaction design interaction design in augmented reality technology centers on the users engagement with the end product to improve the overall user experience and enjoyment the purpose of interaction design is to avoid alienating or confusing the user by organising the information presented since user interaction relies on the users input designers must make system controls easier to understand and accessible a common technique to improve usability for augmented reality applications is by discovering the frequently accessed areas in the devices touch display and design the application to match those areas of control in interaction design it is important for developers to utilize augmented reality technology that complement the systems function or purpose the most exciting factor of augmented reality technology is the ability to utilize the introduction of 3d space this means that a user can potentially access multiple copies of 2d interfaces within a single ar application ar applications are collaborative a user can also connect to anothers device and view or manipulate virtual objects in the other persons context visual design in general visual design is the appearance of the developing application that engages the user to improve the graphic interface elements and user interaction developers may use visual cues to inform user what elements of ui are designed to interact with and how to interact with them since navigating in ar application may appear difficult and seem frustrating visual cues design can make interactions seem more natural in some augmented reality applications that uses a 2d device as an interactive surface the 2d control environment does not translate well in 3d space making users hesitant to explore their surroundings to solve this issue designers should apply visual cues to assist and encourage users to explore their surroundings it is important to note the two main objects in ar when developing vr applications 3d volumetric objects that are manipulatable and realistically interact with light and shadow and animated media imagery such as images and videos which are mostly traditional 2d media rendered in a new context for augmented reality possible applications augmented reality has been explored for many applications from gaming and entertainment to medicine education and business example application areas described below include archaeology architecture commerce and education some of the earliest cited examples include augmented reality used to support surgery by providing virtual overlays to guide medical practitioners to ar content for astronomy and welding literature an example of an ar code containing a qr code the first description of ar as it is known today was in virtual light the 1994 novel by william gibson in 2011 ar was blended with poetry by ni ka from sekai camera in tokyo japan the prose of these ar poems come from paul celan die niemandsrose expressing the aftermath of the 2011 thoku earthquake and tsunami5 archaeology ar has been used to aid archaeological research by augmenting archaeological features onto the modern landscape ar allows archaeologists to formulate possible site configurations from extant structures ar has been recently adopted also in the underwater archaeology field to efficiently support and facilitate the manipulation of archaeological artefacts architecture ar can aid in visualizing building projects computergenerated images of a structure can be superimposed into a real life local view of a property before the physical building is constructed there this was demonstrated publicly by trimble navigation in 2004 ar can also be employed within an architects workspace rendering animated 3d visualizations of their 2d drawings architecture sightseeing can be enhanced with ar applications allowing users viewing a buildings exterior to virtually see through its walls viewing its interior objects and layout with the continual improvements to gps accuracy businesses are able to use augmented reality to visualize georeferenced models of construction sites underground structures cables and pipes using mobile devices examples include the daqri smart helmet an androidpowered hard hat used to create augmented reality for the industrial worker including visual instructions realtime alerts and 3d mapping following the christchurch earthquake the university of canterbury released cityviewar not only did this provide planners with tools to reference the previous cityscape but it also served as a reminder to the magnitude of the devastation caused as entire buildings had been demolished visual art ar applied in the visual arts allows objects or places to trigger artistic multidimensional experiences and interpretations of reality augmented reality can aid in the progression of visual art in museums by allowing museum visitors to view artwork in galleries in a multidimensional way through their phone screens the museum of modern art in new york has created an exhibit in their art museum showcasing augmented reality features that viewers can see using an app on their smartphone this allows individuals to see hidden aspects and information about the paintings and to be able to have an interactive technological experience with the artwork as well ar technology aided the development of eye tracking technology commerce the aricon can be used as a marker on print as well as on online media it signals the viewer that digital content is behind it the content can be viewed with a smartphone or tablet ar is used to integrate print and video marketing printed marketing material can be designed with certain trigger images that when scanned by an arenabled device using image recognition activate a video version of the promotional material a major difference between augmented reality and straightforward image recognition is that one can overlay multiple media at the same time in the view screen such as social media share buttons the inpage video even audio and 3d objects traditional printonly publications are using augmented reality to connect many different types of media ar can enhance product previews such as allowing a customer to view whats inside a products packaging without opening it by 2010 virtual dressing rooms had been developed for ecommerce augment sdk offers brands and retailers the capability to personalize their customers shopping experience by embedding ar product visualization into their ecommerce platforms in 2012 a mint used ar techniques to market a commemorative coin for aruba the coin itself was used as an ar trigger and when held in front of an arenabled device it revealed additional objects and layers of information that were not visible without the device in 2013 loreal paris used crowdoptic technology to create an augmented reality experience at the seventh annual luminato festival in toronto canada in 2014 loreal brought the ar experience to a personal level with their makeup genius app it allowed users to try out makeup and beauty styles via a mobile device in 2015 the bulgarian startup igreet developed its own ar technology and used it to make the first premade live greeting card a traditional paper card was augmented with digital content which was revealed by using the igreet app education in educational settings ar has been used to complement a standard curriculum text graphics video and audio may be superimposed into a students realtime environment textbooks flashcards and other educational reading material may contain embedded markers or triggers that when scanned by an ar device produced supplementary information to the student rendered in a multimedia format as ar evolved students can participate interactively and interact with knowledge more authentically instead of remaining passive recipients students can become active learners able to interact with their learning environment computergenerated simulations of historical events allow students to explore and learning details of each significant area of the event site in higher education construct3d a studierstube system allows students to learn mechanical engineering concepts math or geometry remote collaboration primary school children learn easily from interactive experiences astronomical constellations and the movements of objects in the solar system were oriented in 3d and overlaid in the direction the device was held and expanded with supplemental video information paperbased science book illustrations could seem to come alive as video without requiring the child to navigate to webbased materials in 2013 a project was launched on kickstarter to teach about electronics with an educational toy that allowed children to scan their circuit with an ipad and see the electric current flowing around ar would also be a way for parents and teachers to achieve their goals for modern education which might include providing a more individualized and flexible learning making closer connections between what is taught at school and the real world and helping students to become more engaged in their own learning a recent research compared the functionalities of augmented reality tools with potential for education emergency managementsearch and rescue augmented reality systems are used in public safety situations from super storms to suspects at large as early as 2009 two articles from emergency management magazine discussed the power of this technology for emergency management the first was augmented realityemerging technology for emergency management by gerald baron the article also describes this applications utility and current use as a training tool with rural firefighters part of a national farm medicine center research project on agricultural safety throughout wisconsin and minnesota another early example was a search aircraft looking for a lost hiker in rugged mountain terrain augmented reality systems provided aerial camera operators with a geographic awareness of forest road names and locations blended with the camera video the camera operator was better able to search for the hiker knowing the geographic context of the camera image once located the operator could more efficiently direct rescuers to the hikers location because the geographic position and reference landmarks were clearly labeled social interaction ar can be used to facilitate social interaction an augmented reality social network framework called talk2me enables people to disseminate information and view others advertised information in an augmented reality way the timely and dynamic information sharing and viewing functionalities of talk2me help initiate conversations and make friends for users with people in physical proximity augmented reality also gives users the ability to practice different forms of social interactions with other people in a safe riskfree environment hannes kauffman associate professor for virtual reality at tu vienna says in collaborative augmented reality multiple users may access a shared space populated by virtual objects while remaining grounded in the real world this technique is particularly powerful for educational purposes when users are collocated and can use natural means of communication speech gestures etc but can also be mixed successfully with immersive vr or remote collaboration hanneshannes cites a specific use for this technology education video games see also list of augmented reality software games merchlars mobile game get on target uses a trigger image as fiducial marker the gaming industry embraced ar technology a number of games were developed for prepared indoor environments such as ar air hockey titans of space collaborative combat against virtual enemies and arenhanced pool table games augmented reality allowed video game players to experience digital game play in a real world environment companies and platforms like niantic and proxy42 emerged as major augmented reality gaming creators disney has partnered with lenovo to create the augmented reality game star wars jedi challenges that works with a lenovo mirage ar headset a tracking sensor and a lightsaber controller scheduled to launch in december 2017 industrial design industrial augmented reality ar allows industrial designers to experience a products design and operation before completion volkswagen has used ar for comparing calculated and actual crash test imagery medical since 2005 a device called a nearinfrared vein finder that films subcutaneous veins processes and projects the image of the veins onto the skin has been used to locate veins ar provides surgeons with patient monitoring data in the style of a fighter pilots headsup display and allows patient imaging records including functional videos to be accessed and overlaid examples include a virtual xray view based on prior tomography or on realtime images from ultrasound and confocal microscopy probes ar has been used for cockroach phobia treatment patients wearing augmented reality glasses can be reminded to take medications with the creation of google glass and microsoft hololens has helped pushed augmented reality into medical education spatial immersion and interaction augmented reality applications running on handheld devices utilized as virtual reality headsets can also digitalize human presence in space and provide a computer generated model of them in a virtual space where they can interact and perform various actions such capabilities are demonstrated by project anywhere developed by a postgraduate student at eth zurich which was dubbed as an outofbody experience flight training building on decades of perceptualmotor research in experimental psychology researchers at the aviation research laboratory of the university of illinois at urbanachampaign used augmented reality in the form of a flight path in the sky to teach flight students how to land a flight simulator an adaptive augmented schedule in which students were shown the augmentation only when they departed from the flight path proved to be a more effective training intervention than a constant schedule military augmented reality system for soldier arc4usa an interesting early application of ar occurred when rockwell international created video map overlays of satellite and orbital debris tracks to aid in space observations at air force maui optical system in their 1993 paper debris correlation using the rockwell worldview system the authors describe the use of map overlays applied to video from space surveillance telescopes the map overlays indicated the trajectories of various objects in geographic coordinates this allowed telescope operators to identify satellites and also to identify and catalog potentially dangerous space debris starting in 2003 the us army integrated the smartcam3d augmented reality system into the shadow unmanned aerial system to aid sensor operators using telescopic cameras to locate people or points of interest the system combined both fixed geographic information including street names points of interest airports and railroads with live video from the camera system the system offered a picture in picture mode that allows the system to show a synthetic view of the area surrounding the cameras field of view this helps solve a problem in which the field of view is so narrow that it excludes important context as if looking through a soda straw the system displays realtime friendfoeneutral location markers blended with live video providing the operator with improved situational awareness researchers at usaf research lab calhoun draper et al found an approximately twofold increase in the speed at which uav sensor operators found points of interest using this technology this ability to maintain geographic awareness quantitatively enhances mission efficiency the system is in use on the us army rq7 shadow and the mq1c gray eagle unmanned aerial systems in combat ar can serve as a networked communication system that renders useful battlefield data onto a soldiers goggles in real time from the soldiers viewpoint people and various objects can be marked with special indicators to warn of potential dangers virtual maps and 360 view camera imaging can also be rendered to aid a soldiers navigation and battlefield perspective and this can be transmitted to military leaders at a remote command center navigation see also automotive navigation system landform video map overlay marking runways road and buildings during 1999 helicopter flight test the nasa x38 was flown using a hybrid synthetic vision system that overlaid map data on video to provide enhanced navigation for the spacecraft during flight tests from 1998 to 2002 it used the landform software and was useful for times of limited visibility including an instance when the video camera window frosted over leaving astronauts to rely on the map overlays ar can augment the effectiveness of navigation devices information can be displayed on an automobiles windshield indicating destination directions and meter weather terrain road conditions and traffic information as well as alerts to potential hazards in their path workplace augmented reality may have a good impact on work collaboration as people may be inclined to interact more actively with their learning environment it may also encourage tacit knowledge renewal which makes firms more competitive ar was used to facilitate collaboration among distributed team members via conferences with local and virtual participants ar tasks included brainstorming and discussion meetings utilizing common visualization via touch screen tables interactive digital whiteboards shared design spaces and distributed control rooms in industrial environments augmented reality is proving to have a substantial impact with more and more use cases emerging across all aspect of the product lifecycle starting from product design and new product introduction npi to manufacturing to service and maintenance to material handling and distribution for example labels were displayed on parts of a system to clarify operating instructions for a mechanic performing maintenance on a system as ar technology has evolved and second and third generation ar devices come to market the impact of ar in enterprise continues to flourish in one harvard business review article by magid abraham of upskill and marco annunziata of ge the authors discuss how ar devices are now being used to boost workers productivity on an array of tasks the first time theyre used even without prior training they go on to contend that these technologies increase productivity by making workers more skilled and efficient and thus have the potential to yield both more economic growth and better jobs to support their points they reference a sidebyside timelapse comparison of a ge technician wiring a wind turbines control box using the companys current process and then doing the same task while guided by lineofsight instructions overlaid on the job by an ar headset the device improved the workers performance by 34 on first use the new wave of professionals the millennial workforce demands more efficient knowledge sharing solutions and easier access to rapidly growing knowledge bases augmented reality offers a solution to that broadcast and live events weather visualizations were the first application of augmented reality to television it has now become common in weathercasting to display full motion video of images captured in realtime from multiple cameras and other imaging devices coupled with 3d graphics symbols and mapped to a common virtual geospace model these animated visualizations constitute the first true application of ar to tv ar has become common in sports telecasting sports and entertainment venues are provided with seethrough and overlay augmentation through tracked camera feeds for enhanced viewing by the audience examples include the yellow first down line seen in television broadcasts of american football games showing the line the offensive team must cross to receive a first down ar is also used in association with football and other sporting events to show commercial advertisements overlaid onto the view of the playing area sections of rugby fields and cricket pitches also display sponsored images swimming telecasts often add a line across the lanes to indicate the position of the current record holder as a race proceeds to allow viewers to compare the current race to the best performance other examples include hockey puck tracking and annotations of racing car performance and snooker ball trajectories augmented reality for next generation tv allows viewers to interact with the programs they were watching they can place objects into an existing program and interact with them such as moving them around objects include avatars of real persons in real time who are also watching the same program ar has been used to enhance concert and theater performances for example artists allow listeners to augment their listening experience by adding their performance to that of other bandsgroups of users tourism and sightseeing travelers may use ar to access realtime informational displays regarding a location its features and comments or content provided by previous visitors advanced ar applications include simulations of historical events places and objects rendered into the landscape ar applications linked to geographic locations present location information by audio announcing features of interest at a particular site as they become visible to the user companies can use ar to attract tourists to particular areas that they may not be familiar with by name tourists will be able to experience beautiful landscapes in first person with the use of ar devices companies like phocuswright plan to use such technology in order to expose the lesser known but beautiful areas of the planet and in turn increase tourism other companies such as matoke tours have already developed an application where the user can see 360 degrees from several different places in uganda matoke tours and phocuswright have the ability to display their apps on virtual reality headsets like the samsung vr and oculus rift translation ar systems such as word lens can interpret the foreign text on signs and menus and in a users augmented view redisplay the text in the users language spoken words of a foreign language can be translated and displayed in a users view as printed subtitles music it has been suggested that augmented reality may be used in new methods of music production mixing control and visualization a tool for 3d music creation in clubs that in addition to regular sound mixing features allows the dj to play dozens of sound samples placed anywhere in 3d space has been conceptualized leeds college of music teams have developed an ar app that can be used with audient desks and allow students to use their smartphone or tablet to put layers of information or interactivity on top of an audient mixing desk armony is a software package that makes use of augmented reality to help people to learn an instrument in a proofofconcept project ian sterling interaction design student at california college of the arts and software engineer swaroop pal demonstrated a hololens app whose primary purpose is to provide a 3d spatial ui for crossplatform devices the android music player app and arduinocontrolled fan and light and also allow interaction using gaze and gesture control ar mixer is an app that allows one to select and mix between songs by manipulating objects such as changing the orientation of a bottle or can in a video uriel yehezkel demonstrates using the leap motion controller and geco midi to control ableton live with hand gestures and states that by this method he was able to control more than 10 parameters simultaneously with both hands and take full control over the construction of the song emotion and energy a novel musical instrument that allows novices to play electronic musical compositions interactively remixing and modulating their elements by manipulating simple physical objects has been proposed a system using explicit gestures and implicit dance moves to control the visual augmentations of a live music performance that enable more dynamic and spontaneous performances andin combination with indirect augmented realityleading to a more intense interaction between artist and audience has been suggested research by members of the cristal at the university of lille makes use of augmented reality in order to enrich musical performance the controllar project allows musicians to augment their midi control surfaces with the remixed graphical user interfaces of music software retail augmented reality is becoming more frequently used for online advertising retailers offer the ability to upload a picture on their website and try on various clothes which is overlaid on the picture even further companies such as bodymetrics install dressing booths in department stores that offer fullbody scanning these booths render a 3d model of the user allowing the consumers to view different outfits on themselves without the need of physically changing clothes ar technology is also used by furniture retailers such as ikea houzz and wayfair snapchat snapchat users have access to augmented reality in the companys instant messaging app through use of camera filters in september 2017 snapchat updated its app to include a camera filter that allowed users to render an animated cartoon version of themselves called bitmoji these animated avatars would be projected in the real world through the camera and can be photographed or video recorded the dangers of ar reality modifications there is a danger that ar will make individuals overconfident and put their life at risk because of it pokemon go with a couple of deaths and many injuries is the perfect example of it death by pokmon go by a pair of researchers from purdue universitys krannert school of management says the game caused a disproportionate increase in vehicular crashes and associated vehicular damage personal injuries and fatalities in the vicinity of locations called pokstops where users can play the game while driving the paper extrapolated what that might mean nationwide and concluded the increase in crashes attributable to the introduction of pokmon go is 145632 with an associated increase in the number of injuries of 29370 and an associated increase in the number of fatalities of 256 over the period of july 6 2016 through november 30 2016 the authors valued those crashes and fatalities at between 2bn and 73 billion for the same period furthermore more than one in three surveyed advanced internet users would like to edit out disturbing elements around them such as garbage or graffiti they would like to even modify their surroundings by erasing street signs billboard ads and uninteresting shopping windows so it seems that ar is a threat to companies as it is an opportunity although this could be a nightmare to numerous brands that do not manage to capture consumer imaginations it also creates the risk that the wearers of augmented reality glasses may become unaware of surrounding dangers consumers want to use augmented reality glasses to change their surroundings into something that reflects their own personal opinions around two in five want to change the way their surroundings look and even how people appear to them next to the possible privacy issues that are described below overload and overreliance issues is the biggest danger of ar for the development of new ar related products this implies that the userinterface should follow certain guidelines as not to overload the user with information while also preventing the user to overly rely on the ar system such that important cues from the environment are missed once the key is not taken into account people might not need the real word anymore privacy concerns the concept of modern augmented reality depends on the ability of the device to record and analyze the environment in real time because of this there are potential legal concerns over privacy while the first amendment to the united states constitution allows for such recording in the name of public interest the constant recording of an ar device makes it difficult to do so without also recording outside of the public domain legal complications would be found in areas where a right to a certain amount of privacy is expected or where copyrighted media are displayed in terms of individual privacy there exists the ease of access to information that one should not readily possess about a given person this is accomplished through facial recognition technology assuming that ar automatically passes information about persons that the user sees there could be anything seen from social media criminal record and marital status privacycompliant image capture solutions can be deployed to temper the impact of constant filming on individual privacy notable researchers ivan sutherland invented the first vr headmounted display at harvard university steve mann formulated an earlier concept of mediated reality in the 1970s and 1980s using cameras processors and display systems to modify visual reality to help people see better dynamic range management building computerized welding helmets as well as augmediated reality vision systems for use in everyday life he is also an adviser to meta louis rosenberg developed one of the first known ar systems called virtual fixtures while working at the us air force armstrong labs in 1991 and published the first study of how an ar system can enhance human performance mike abernathy pioneered one of the first successful augmented video overlays also called hybrid syntheric vision using map data for space debris in 1993 steven feiner professor at columbia university is the author of a 1993 paper on an ar system prototype karma the knowledgebased augmented reality maintenance assistant along with blair macintyre and doree seligmann he is also an advisor to meta tracy mcsheery of phasespace developer in 2009 of wide field of view ar lenses as used in meta 2 and others s ravela b draper j lim and a hanson developed a markerfixtureless augmented reality system with computer vision in 1994 they augmented an engine block observed from a single video camera with annotations for repair they use modelbased pose estimation aspect graphs and visual feature tracking to dynamically register model with the observed video francisco delgado is a nasa engineer and project manager specializing in human interface research and development starting 1998 he conducted research into displays that combined video with synthetic vision systems called hybrid synthetic vision at the time that we recognize today as augmented reality systems for the control of aircraft and spacecraft in 1999 he and colleague mike abernathy flighttested the landform system aboard a us army helicopter delgado oversaw integration of the landform and smartcam3d systems into the x38 crew return vehicle bruce h thomas and wayne piekarski develop the tinmith system in 1998 they along with steve feiner with his mars system pioneer outdoor augmented reality mark billinghurst is director of the hit lab new zealand hit lab nz at the university of canterbury in new zealand and a notable ar researcher he has produced over 250 technical publications and presented demonstrations and courses at a wide variety of conferences reinhold behringer performed important early work 1998 in image registration for augmented reality and prototype wearable testbeds for augmented reality he also coorganized the first ieee international symposium on augmented reality in 1998 iwar98 and coedited one of the first books on augmented reality felix g hamzalup larry davis and jannick rolland developed the 3d arc display with optical seethrough headwarned display for ar visualization in 2002 dieter schmalstieg and daniel wagner developed a marker tracking systems for mobile phones and pdas in 2009 history 1901 l frank baum an author first mentions the idea of an electronic displayspectacles that overlays data onto real life in this case people it is named a character marker 195762 morton heilig a cinematographer creates and patents a simulator called sensorama with visuals sound vibration and smell 1968 ivan sutherland invents the headmounted display and positions it as a window into a virtual world 1975 myron krueger creates videoplace to allow users to interact with virtual objects 1980 the research by gavan lintern of the university of illinois is the first published work to show the value of a heads up display for teaching realworld flight skills 1980 steve mann creates the first wearable computer a computer vision system with text and graphical overlays on a photographically mediated scene see eyetap see heads up display 1981 dan reitan geospatially maps multiple weather radar images and spacebased and studio cameras to earth maps and abstract symbols for television weather broadcasts bringing a precursor concept to augmented reality mixed realgraphical images to tv 1984 in the film the terminator the terminator uses a headsup display in several parts of the film in one part he accesses a diagram of the gear system of the truck he gets into towards the end of the film 1987 douglas george and robert morris create a working prototype of an astronomical telescopebased headsup display system a precursor concept to augmented reality which superimposed in the telescope eyepiece over the actual sky images multiintensity star and celestial body images and other relevant information 1990 the term augmented reality is attributed to thomas p caudell a former boeing researcher 1992 louis rosenberg developed one of the first functioning ar systems called virtual fixtures at the united states air force research laboratoryarmstrong that demonstrated benefit to human perception 1993 steven feiner blair macintyre and doree seligmann present an early paper on an ar system prototype karma at the graphics interface conference 1993 mike abernathy et al report the first use of augmented reality in identifying space debris using rockwell worldview by overlaying satellite geographic trajectories on live telescope video 1993 a widely cited version of the paper above is published in communications of the acm special issue on computer augmented environments edited by pierre wellner wendy mackay and rich gold 1993 loral wdl with sponsorship from stricom performed the first demonstration combining live arequipped vehicles and manned simulators unpublished paper j barrilleaux experiences and observations in applying augmented reality to live training 1999 1994 julie martin creates first augmented reality theater production dancing in cyberspace funded by the australia council for the arts features dancers and acrobats manipulating bodysized virtual object in real time projected into the same physical space and performance plane the acrobats appeared immersed within the virtual object and environments the installation used silicon graphics computers and polhemus sensing system 1995 s ravela et al at university of massachusetts introduce a visionbased system using monocular cameras to track objects engine blocks across views for augmented reality 1998 spatial augmented reality introduced at university of north carolina at chapel hill by ramesh raskar welch henry fuchs 1999 frank delgado mike abernathy et al report successful flight test of landform software video map overlay from a helicopter at army yuma proving ground overlaying video with runways taxiways roads and road names 1999 the us naval research laboratory engages on a decadelong research program called the battlefield augmented reality system bars to prototype some of the early wearable systems for dismounted soldier operating in urban environment for situation awareness and training 1999 hirokazu kato created artoolkit at hitlab where ar later was further developed by other hitlab scientists demonstrating it at siggraph 2000 bruce h thomas develops arquake the first outdoor mobile ar game demonstrating it in the international symposium on wearable computers 2001 nasa x38 flown using landform software video map overlays at dryden flight research center 2004 outdoor helmetmounted ar system demonstrated by trimble navigation and the human interface technology laboratory 2008 wikitude ar travel guide launches on 20 oct 2008 with the g1 android phone 2009 artoolkit was ported to adobe flash flartoolkit by saqoosha bringing augmented reality to the web browser 2012 launch of lyteshot an interactive ar gaming platform that utilizes smart glasses for game data 2013 meta announces the meta 1 developer kit 2013 google announces an open beta test of its google glass augmented reality glasses the glasses reach the internet through bluetooth which connects to the wireless service on a users cellphone the glasses respond when a user speaks touches the frame or moves the head 2014 mahei creates the first generation of augmented reality enhanced educational toys 2015 microsoft announces windows holographic and the hololens augmented reality headset the headset utilizes various sensors and a processing unit to blend high definition holograms with the real world 2016 niantic released pokmon go for ios and android in july 2016 the game quickly became one of the most popular smartphone applications and in turn spikes the popularity of augmented reality games 2017 magic leap announces the use of digital lightfield technology embedded into the magic leap one headset the creators edition headset includes the glasses and a computing pack worn on your belt alternate reality game artag augmented browsing augmented realitybased testing augmented web automotive headup display bionic contact lens brain in a vat computermediated reality cyborg eyetap headmounted display holography lifelike experience list of augmented reality software magic leap mixed reality optical headmounted display simulated reality smartglasses transreality gaming video mapping viractualism virtual reality visuohaptic mixed reality wearable computing wikimedia commons has media related to augmented reality authority control gnd 47158025 lccn sh2006020069 ndl 01211643 