Notice anything weird about the image below? Normally when you point your cellphone camera on an object, the cellphone screen shows you exactly what is in front of it. However in the image below, rather than see just a black square on the phone screen, an elephant seems to have magically appeared on the square! Are you curious how this is achieved? Well, let me tell you about Augmented Reality – a technology which adds to the real world – i.e. augments reality.
There is no elephant in the real world – then how did it appear on the phone screen?
There are three pieces to making this magic happen – computer graphics, object recognition and your smartphone. Let us first tackle these one by one and then see how they are put together.
Computer graphics is the technology behind countless Disney movies like Toy Story. In the image below, computer graphics is used to generate the character Woody, who runs on a flat wooden floor. However Woody, created using computer graphics alone, lives only in your computer but not in the real world around you.
Computer graphics generated an image of Woody running on the floor.
Object Recognition is the technology which allows a smartphone that has a camera, to look at the real world and identify objects. For example look at the rather simple rectangular pattern in the image below. It is possible for the phone to recognize that pattern.
A simple pattern for the phone to recognize
In fact, the image below shows an iPad looking at precisely the same pattern laid on a table. The ipad indicates that it recognized the pattern, by placing a bright green border around it. Also, the green border seems to lie on the table, just over the pattern – i.e. our iPad has found a flat surface in the real world!
The phone recognized the pattern and drew a green box around it.
So how does all this result in Augmented Reality? Well, let us look at the pieces we have right now. Graphics gives the phone the capability, to make virtual characters appear on virtual flat surfaces, in a virtual world, . Object recognition gives the phone the capability to recognize a real pattern on a real surface, in the real world. If you put those two technologies together, it should be possible to make a virtual character appear on a real surface, that the phone sees in the real world.
That is precisely what was happening in the first picture – the phone camera looked down and saw a flat black square pattern. It recognized the pattern and hence the flat surface. Before the phone showed that image on the screen, it created an elephant and placed it on the pattern. Viola – you now see an elephant where there is none! Not magic but some very clever engineering!
Nikhil's Musings 