How Google’s India maps are built

In a rare setback to Apple, its Wednesday update to its iPhone and iPad operating system replacing Google Maps reportedly makes the new maps less detailed, weird-looking.

Google Maps here is an entirely user-driven project, aided by some sophisticated code

How did Google Inc., a start-up that had majored in the art and science of Internet search, create maps that have become the go-to guide for commuters world over? Last month, The Atlantic offered a behind-the-grids view of how the technology major went about building the maps that have come to become one of the most expansive and accurate sources of geo-information on the Web.

This first of its kind report on what Google dubs as ‘Project Ground Truth’, its very secret mapping programme, revealed that base layers for these maps source data from the U.S. Census Bureau, the Geological Survey, and of course, from the famous – infamously discontinued after its launch in India – Google Street View vans that have clocked over 5 million miles till date. All this is, of course, crunched by Google’s sophisticated algorithms and clever, rigorous code.

Evidently, the process of creating among the most accurate maps in the world is complex, and fascinating. But in India, where an increasing number of smartphone users are logging on to maps, the ‘ground truth’ is significantly different: all that you see on the grid on Google Maps in India has been created by users. An entirely crowd-sourced product, our maps are created by us.

Altogether different from how the search giant went about mapping in these developed countries, in India, Google is only now exploring partnerships with some government establishments to map rural India better. That its Street View vans are now resting in the basement of Google’s office in Bangalore after the police stymied the project citing security concerns, is also perhaps one of the reasons why Google may be sticking to this user-only approach.

But Lalitesh Katragadda, Google India, Country Head (Products), one of the co-creators of Google Map Makers, insists that the reason they choose to focus on users is because emerging markets like India are in “hyper-growth”. “This means, the maps here change every day. If I leave Bangalore for a year, I come back to see an altogether different city, unlike say Silicon Valley where not much changes. There is constant churn of points of interest, road geometry, one-ways and building profiles. That kind of data can’t be given fresh to users without getting users on board,” he says. This, he says, is a problem that hasn’t been solved.

Brushing aside questions on what competitors are doing, and competition, he says everybody is fighting fierce over the first billion users (in developed economies), but the real challenge is to take this to the next 4.5 billion users and “getting that right is more important”. Since its launch in 2005, Google claims it has covered 1.5 billion more people across the world, and managed to map over 180 developing countries that had practically nothing in the way of high quality maps before. These numbers, the company clarifies is based on its own estimations based on populations of areas that have access to high quality maps. For instance, the population of a slum settlement in Bangalore doesn't get counted as its user base for the purpose of this estimation. "When we sat down to write code in 2004-05, the entire developing world was dark. Today, we've managed to pushed the needle from around 1 billion in the developed world to 2.5 billion (worldwide)." This, he adds, would not have been possible if it weren't for a platform like Google Map Maker that provided users with a simple, lively and interactive platform to map their streets, their neighbourhoods or their cities. It solved a problem that an individual company couldn't, by simply tapping into the wisdom and initiative of the crowd.

A "socio-technological" task

Map Maker, the “socio-technological phenomenon” that Mr. Katragadda and a small team at Google started in 2006 when they geo-tagged Ulsoor lake here, is a tool that allows users everywhere to edit (or create from a scratch) the maps they use. Maps, whether user-created or otherwise, are scientifically some of the most challenging data sets ever dealt with; far from simple text or images, they involve what in computer science parlance is called a graph comprising connected edges or nodes. And most of all, the demand for accuracy is so high that maps less than 97 or 98 per cent accurate are simply irrelevant. So how does Google ensure that these maps, created by random but enthusiastic users, meet such high standards?

How it's done

Creating maps out of crowd-sourced data presents technological and systems challenges that are huge. To begin with, how does a user make changes, and how does the machine learn to recognise these gestures, trust the data that is being fed, and update the maps in real time?

The high-resolution satellite imagery on the Map Maker platform is where you start. Once the user feeds in a landmark, or say names a road, the Map Maker encodes the science of computational geometry to automatically synthesise the map based on individual gestures from users. This begins with a user visually gesturing their knowledge, i.e. sketch on top of satellite imagery. Map Maker then takes these small inputs from a whole bunch of users and synthesises that into a connected graph that is geo-spatially overlaid on top of the world, Mr. Katragadda explains.

Another key challenge, he explains, is making all this happen in real time. This, he says, is what sets Google’s platform apart from other community-driven Open Source mapping projects such as Open Street Maps. Besides code written by “very smart engineers” to solve what is obviously a hard system challenge, it also leverages the Google Cloud.

This gives Google the ability to manipulate the entire map of the world in real time in its data centres. So, every change flows to all the data centres that are serving maps.

But what about accuracy, undoubtedly, the most important prerequisite of a map? The “secret sauce” is what Mr. Katragadda calls the “trust system”, which computer engineers will call a supervised learning system. This, he explains, is a machine learning system that notes all that people are doing — who is editing, who is moderating and what which moderator is saying.

This means that when you come in as a new user, with no mapping history, the first few edits go straight into moderation. This moderation is done mostly by the community, but also a small team of engineers that provides ‘support and guidance’ to the mapping community. It is, essentially, PageRank (Google's patented link analysis algorithm, for internet search) applied to people for the task of mapping. This “trust system” coupled with the community approach lies at the heart of the entire Google Map experience in India. “Over time, the machine learns based on hundreds of signals — where can you be trusted to map, what you map, which regions and what kind of mapping you do. It creates a profile that informs when you come in to map, the system knows that you are more trustworthy,” Mr. Katragadda explains, adding that over 80 per cent edits in India are approved immediately in real time.

When asked if this isn’t similar to the kind of system they have at the other community-run Open Source project, Wikipedia, he says that it’s radically different because there, the moderators are hand-picked, and the machine doesn’t continually learn and change your trust parameters based on your history. “That’s static, while this is a truly dynamic system.”

Not the 'California Indians'

While these huge technical challenges have more or less been solved, Mr. Katragadda feels that the path ahead is dotted with “socio-technological” hurdles. In India, for instance, he explains, the first 100 million users are taken care of. He calls them the “California Indians”, who behave and have needs similar to his friends back in California. But for the next billion and more, the needs are “extraordinarily different”: their ability to afford devices is different, their language and content needs are local and education levels are different. This, he says, translates into them needing a visual Web. The challenge for us is to understand what they need and build services that are different from what Google, and literally the entire Internet has built for them till now, he says. That is, how to get the mainstream of the world online, make the Internet useful for them and remove the huge information asymmetry that we now live with.

And the solution to this, Mr. Katragadda recognises, goes deeper than offering turn-by-turn navigations for motorcyclists in local languages. “It’s a lot more, and a lot more different.”