Neither A Pat On The Back Nor A Slap On The Wrist: India’s Open Govt Data Builds On Collaboration

Last Friday, a stakeholders’ workshop for the government data ecosystem was hosted in New Delhi by the Open Data Technology Division of National Informatics Centre (NIC), Ministry of Electronics and Information Technology (MeitY). 

The initiative was held with an aim to understand the experiences, concerns and expectations of Open Government Data (OGD) users and data providers. While the government has made undeniable strides in creating an ecosystem to collect, process, and provide information under its data strategy towards a Viksit Bharat, Deputy Director General, NIC Alka Misra recognised the existence of challenges and opportunities within the ecosystem. “We wouldn’t be here if all we wanted was a pat on the back” she said during the open house discussion. 

Creating An Ecosystem

About a decade ago, troves of data generated by government departments were being underutilised for policy making. There was a dire need to provide standardised data in easy to access forms through a common access point to ensure wider accessibility.

The National Data Sharing and Accessibility Policy (NDSAP) notified in 2012 paved the way by mandating proactive disclosure and enhanced discoverability of open government data resources.

It is in this context that the data.gov.in platform was designed to enable Central Government entities to implement the NDSAP. Independent portals of the OGD platform are also being created at the State, Union Territory, and Urban Local Body level.

In theory, the public now has the raw material to achieve the novel goals of enabling informed public participation in governance, improving delivery of government services, and promoting accountability and transparency. Government data freed from the trappings of dusty files now has a greater possibility to be harnessed for better policy-making, governance, and economic growth. 

Today, 12 years after this portal to information was opened, the platform hosts over 6 lakh datasets from government departments across 37 sectors.

Through three panel discussions, the recent workshop brought together government officials, chief data officers, researchers, policymakers, journalists, and representatives from civil society to view the portal through different lenses in an endeavour to improve data governance, quality, and usability.

Stakeholders Reflect And Recommend

In the beginning, NDSAP had not laid clear protocols for information gathering, processing, and sharing. Only a few datasets were up to the mark.

With differences in definitions and methodologies, departments could collect different datasets under the same terminology, or even worse, the same information under different terminology.

“A simple question of how many districts were in India could be answered differently, depending on if you asked the Reserve Bank of India or referred to the census,” said John Samuel Raja Duraipandy, co-founder of How India Lives, a data-analytics firm specialising in utilising publicly available data. 

“In 2015, when How India Lives was founded, the primary problem was that data was housed in silos. The key has always been to make one data source talk to another data source” he said.

Ten years into the endeavour with Chief Data Officers spearheading the implementation of the policy in their respective ministries and departments, the scorecard on interoperability fairs better. 

Regarding the quality of datasets, data is still often uploaded in the form of scanned PDFs or images on government websites making data extraction difficult. “About 65 per cent of our budget still goes into making government data machine-readable,” said V.R. Raman, Executive Director at the Centre for Budget and Governance Accountability.

Any data event is bound to mention the analogy of data being the new oil—unrefined, difficult to extract, and valuable—whether participants agree or debate a new metaphor is another story. “Data is the fuel for development and accountability,” said Raman. “The problem is when someone captures it but sits on it.” Ensuring this fuel isn’t left untapped in difficult-to-use formats remains a critical challenge. 

Use Cases: Current and Potential

Access to information is often linked to the realisation of rights and an improved quality of life. How does one fix something that isn’t measured? 

Open government data can be used to see how well government programs are working, find areas that need more attention for policy changes, and create community projects that share useful information.

Open Street Maps is one such use-case, presented at the workshop, wherein government data such as the location of health facilities was imported into a map to aid disaster management in Kerala. Another example is Open Contracting India by CivicDataLab, which aims at making public procurement processes in India more efficient, accessible and participatory.

Taking it down to the grassroots, Bolbhav is an AI-based WhatsApp chatbot that enables Indian farmers to access timely, consistent, and credible data before they make decisions such as what price to sell their produce. Over 10 million queries were answered for over 250,000 farmers by the bot, explained Achint Sanghi co-founder of Gramhal Foundation, the tech non-profit behind the app.

In these use cases, the data existed but it took time, effort, money, and the creation of a whole new product to generate public value from it. “India has a last mile problem,” says Rukmini S, founder of Data For India, a public platform to expand the understanding of India through data.

Long term knowledge and public understanding isn’t coming from navigating the vast amounts of data by oneself. Such platforms are on the way to solve the problem. “The last mile problem is limited by imagination and ambition,” she adds.

There are ways that the government can help create such repositories of value. 

Gaps and Takeaways

The first pillar of using government data is knowing where the data is. Not everyone has the capability or patience to open a 100 tabs or incessantly call ministries. While Rukmini’s book Whole Number and Half Truths gives an insight into what data exists and what it can and cannot tell us about India, “The book shouldn’t have needed to exist” she says.

Among the takeaways that emerged from the discussions was the complete, continued, and regular publication of open government data resources. Some datasets are still incomplete or go missing at times, much to the chagrin of data users.

Although the ecosystem has evolved, stakeholders agreed that a lack of metadata thwarted efforts to understand what each column in a dataset exactly meant. 

Tim Berners-Lee’s 5-star open data scheme starts at inaccessible PDFs and ends at Linked Open Data wherein data publishers connect their data with data from different sources, providing context.

This 5-star rating is something that remains to be implemented by the platform and can drastically increase access and usability of the OGD. Stakeholders recommended introducing a common metadata catalogue for various open government data resources.

It was also noted that only a few states such as Tamil Nadu and Punjab have come out with a state-level policy akin to the NDSAP. There is also scope to embrace new opportunities of data utilisation through Artifical Intelligence (AI) readable and AI-understandable data resources to stay up to date with latest technology innovations.

Lastly, India does not have data anonymisation standards yet and with shifting power structures from private to public data, non-personal data is not covered in the much-awaited Digital Personal Data Protection Bill. This brings to fore the possible threat of community profiling even through non-personal open government data.

“It begs for the introduction of grievance redressal portals” says Shashank Mohan, from the Centre for Communication Governance, National Law University, Delhi. 

While this stakeholder workshop was a welcome step, it remains to be seen how the government plans to incorporate these suggestions.