Posted by Davina Jackson on 7 Nov 2013

Christchurch really is devastated. After the collapse of many of its key buildings during the earthquakes of 2010-11, most of the city centre now is vacant land growing weeds, with coffee being served from shipping containers and caravans.

CityView AR, an augmented reality app showing panoramas of key Christchurch streets as they were before 2011. HITLab NZ/Uni Canterbury.

CityView AR, an augmented reality app showing key Christchurch buildings as they were before 2011. HITLab NZ/Uni Canterbury.

Fortunately, the earthquakes delivered a rare opportunity for Christchurch wanderers to experience ‘augmented reality’ (AR).

On a garden campus 15 minutes from the devastated zones, the University of Canterbury has a squad of AR experts pushing next frontiers of photorealism, video games, mobile telephony, networked devices, sensor geography, user interfaces, citizen science … in short, ubiquitous environmental computing.

Led by Professor Mark Billinghurst, HITLab NZ’s postgraduate researchers are blending ‘virtual’ (computer-simulated) environments with still or video images showing ‘real-time’ reality and/or historic scenes.

HIT is the acronym for Human Interface Technology. HITLab NZ’s experts are plotting how to use digital technologies to confuse people getting around with mobile devices. They want their users to misunderstand where they are and what reality ‘really’ is.

Billinghurst and his colleagues – including Dr Gun Lee, a Fellow from Korea – manipulate visible and invisible computer tools to create informative games that can trick their users’ eyes, ears, noses, skins, points of touch and/or tastes.

HITLab NZ's Dr Gun Lee (left) and Professor Mark Billinghurst wearing Google Glasses in post-earthquake Christchurch.

HITLab NZ’s Dr Gun Lee (left) and Professor Mark Billinghurst wearing Google Glasses in post-earthquake Christchurch.

There’s a significant purpose to AR. It underlies the ‘serious games’ and ‘gamification’ movement that is becoming essential to help educate next generations of professionals across many disciplines, industries, sectors and activities around the world. A critical field is urban environmental planning and management – requiring computer simulations and algorithmic data analysis to support ‘predictive futures’ projects.

HITLab NZ attracts top researchers from Europe and Asia to concoct video games that support important challenges or inform and entertain about aspects of culture. Its customers usually are Christchurch government agencies and its sponsoring partners include some of the world’s top IT corporations.

After a 2012 stint in California, helping Google develop AR systems for its new Glass sensor-assisted spectacles, Billinghurst is back supervising HITLab NZ teams developing apps for Glass, Android and other Google Maps or Earth-enabled devices and programs (including Apple mobiles).

They also are creating videos for viewing through Oculus’ Rift virtual reality headsets – which stop users from seeing their ‘real’ surroundings, forcing them to focus entirely on the simulated environments on their screens.

Recent HITLab NZ projects include:

CityView AR (aka Earthquake AR). An Android mobile app for pedestrians wandering Christchurch’s city centre. Depending on where users are standing and where their devices are pointing, the app shows images of buildings which are now absent from (or in ruins at) the ‘real’ blocks of land being simultaneously viewed. Hundreds of city buildings were 3D-modelled (by architect Jason Mill) as they looked when the earthquakes happened – or earlier in history or how they might look when replaced in future. Pop-up screens provide background information on each building.

High Street Stories

Antarctic AR. A video game for Android mobiles that allows visitors to Christchurch’s Hagley Park to re-enact Robert Falcon Scott’s trek to the South Pole. Snow-white scenes from Antarctica, including a panorama of real photographs of the interior of Scott’s hut, have been geo-mapped to the park using GPS (global satellite positioning) sensors. The game can change its Antarctic scenery according to where users are standing in the real (green) park.


Experience an Antarctic AR (augmented reality) trek while wandering in Christchurch’s Hagley Park. HITLab NZ/Uni Canterbury.

CCDU AR. Using this app on a phone or tablet, users can see 3D hologram-like images of major buildings proposed for Christchurch’s reconstruction. These ‘arise’ from their designated plots of land on a coloured map printed on paper. The app uses computer vision technology to recognise the map through the device’s camera lens. The building images change as the lens scans different areas of the map.

Images of proposed buildings arise from a map of Christchurch, using HITLab NZ's CCDU AR app on a mobile device.

Images of proposed buildings arise from a map of Christchurch, using HITLab NZ’s CCDU AR app on a mobile device.

GeoBoids. GeoBoids are animated creatures (like avatars) which fly around the game screen when users walk into a real area (field of play) shown on Google Maps. The scenery on screen is an artists’ version of the real environment around each player. The GeoBoids are ‘creatures from another dimension that have been pushed into our space due to a rip in the time-space continuum’. They must be ‘captured’ by clicking the cursor so they can be ‘returned home’. The serious point of the game is to encourage vigorous exercise among children and game addicts who normally play while seated.


Game addicts don’t know they’re exercising with the GPS-enabled GeoBoids app. HITLab NZ/Uni Canterbury.

Most of these apps and games have been built with HITLab NZ’s Mobile AR Framework, a platform of computer tools, including libraries of digital graphic objects, that developers can use to create other mobile AR experiences.

Davina Jackson