The Instagram pages of the BJP and the Congress already show hints of AI in use. A vast majority of the parties' social media teams still rely on traditional graphics but there is a smattering of deep fakes too on their handles.

Over the last 18 months, deep fakes and AI-generated content have become indistinguishable from the real thing. The government, policymakers and Big Tech are still trying to catch up on how it can be managed.

Who's Behind These Carefully Crafted AI Campaigns

One of the first instances of deep fakes was in 2020 when a deep fake of BJP MP and Bhojpuri actor Manoj Tiwari's voice was cloned into English and Haryanvi, languages he wasn't fluent in.

"The response and feedback were pretty amazing," Sagar Vishnoi, a political consultant who was behind the deep fake, told The Secretariat. "Such is the power of AI."

While there were concerns raised then on why this could be troubling, Vishnoi believes what resonated with the constituents was the personalisation of a political leader speaking in the language of his target audience. 

Now, voice cloning has gone up a notch. The Secretariat spoke to The Indian Deepfaker (TID), a company that has moved from advertising and films to working for politicians and political consultancies.

TID made headlines in July 2023 after it created a deep fake video of Bollywood actors Kangana Ranaut (and now the BJP candidate from Mandi) and Hrithik Roshan and morphed their faces onto Margot Robbie and Ryan Gosling's characters from the film Barbie.

By October-November 2023, six months before the elections, TID was inundated with requests from political parties to avail their services.

TID co-founder Divyendra Singh Jadoun and his colleagues are now using AI to replace traditional forms of campaigning. Their two-way AI conversation is an advanced robocall tool which uses a political leader's voice and likeness. The leader's speech isn't pre-recorded and uses AI to formulate a response to the caller in real-time.

It's like talking to ChatGPT, the difference being that the voice of your political leader would be responding to your question prompts.

"Earlier, political leaders used automated calls, but it wasn't immersive or personalised. It felt like a call from a call centre," said Jadoun.

Personalised To A Fault

But these AI calls are personalised, which means the leader will get your name right, enquire about your age and where you live, and deliver a formulated message to suit your demographic. 

The calls would be recorded and transcribed. By adding semantic search, politicians will be able to extrapolate issues concerning a particular demographic, explained Jadoun.

"Through AI conversational calls, we can have a survey of 10 lakh people in just one day, which would take months earlier and, of course, a lot of money," added Vishnoi.

Most people can identify and end an automated call within two or three seconds. Jadoun's betting that with these highly personalised calls which tell the callers right away that it's an AI-supported call, his digitally created AI politicians won't get the same treatment. 

TID charges Rs 2 per call per minute for domestic clients, and are expecting to rake in between 10 lakh to 1 crore calls right before the elections get underway on April 19.

Citing non-disclosure agreements, Jadoun didn't reveal his political clients, some of whom are Independents, but said he gets most requests from tier-1 cities like Delhi, and north-eastern and southern states.

Imagine if Independent candidates have such a hefty AI election budget, how much are mainstream parties like the BJP and the Congress pouring into this technology? Vishnoi estimated the AI budget for all parties together this election cycle would be between Rs 40 crore and Rs 50 crore, which accounts for 1-2 per cent of the campaign budget.

"Parties down South, like AIADMK and DMK, are spending around one to two crore (rupees) respectively," added Vishnoi. The AI spending is expected to rise for elections as parties are likely to create a separate wing for AI in digital campaigns in the future.

Perils Of Using Advanced AI In Electioneering

While TID and their counterparts are banking on AI to enhance the reach of politicians during elections, there are malicious entities that have muddied the waters.

TID also got requests to create deep fakes of leaders belonging to opposition parties, but Jadoun said they turned down these requests due to ethical reasons. He repeats twice that TID isn't in the business of creating deep fakes for political clients. But many others are.

Recently, India Today carried out a sting operation on a Noida-based digital consultancy whose founder offered to create deep fakes to malign opposition candidates.

"We have received a lot of unethical requests to create a clone of the opposition party leader and make him say something that he has never said to harm his image," said Jadoun. "It seems like it's a common practice for them (political and PR consultancies)."

Even US President Joe Biden fell prey to voice cloning when a fake robocall did the rounds ahead of the New Hampshire primary, discouraging constituents from casting their votes. The voice resembled Biden's and employed phrases used by him, but it wasn't real and was a product of AI wizardry. 

"Fortunately in India, we haven't come across any such big use case which has divided the social fabric of a particular constituency," said Vishnoi. 

Voice cloning isn't novel. Anyone who can access Hugging Face, a repository of open-source machine learning models, can download a voice-cloning model and go to town with it.

In a recent blog, OpenAI, the creator of ChatGPT, acknowledged the existence of its in-house voice cloning technology, which can create an 'emotive' and 'realistic' sounding replica from a 15-second extract of someone's voice and a textual prompt.

Another AI company, ElevenLabs, can clone a person's voice in English in three seconds. For Telugu or Tamil, it takes much longer. 

OpenAI has made ChatGPT and DALL-E public, but isn't sure if they will open up the Voice Engine model as a use-for-all due to potential misuse.

Speaking on the dangers, Jadoun said, "This technology is already out there and it's very good. Tech is neutral, it depends on the person how he or she is using it. If it's in the wrong hands, it can be very, very  destructive."

To create a deep fake earlier, he said, it would take a lot of computational power and data training but things have gotten easier. "Just with a single image, anyone can create a deep fake in three minutes. Earlier, it used to take us around 20 days," said Jadoun.

India has witnessed a fair share of deep fakes and AI-generated content to sway voters. The DMK resurrected the likeness of M Karunanidhi from beyond the grave. Although TID did not create that video, Jadoun estimated AI companies charge Rs 50,000 or so to create one such deep fake video.

Building Guardrails Around AI

As India and about 50 other countries head to elections this year, it's becoming imperative that constituents be able to identify disinformation from the real thing. With regulations around AI uniformly slow all around the world, the fate of elections hangs in the balance.

"Any change in policy is a time-taking process. We are expecting a change in June or July maybe," said Vishnoi. "What we're seeing is a transition period, and the speed with which it (AI) is changing, very soon you'll see an Indian leader speaking somewhere in Tamil Nadu and you'll be hearing it in Punjabi in real-time."

In fact, Prime Minister Narendra Modi used Bhashini, an AI-powered tool developed by the Indian government, to translate his speech to a non-Hindi-speaking audience in real time at two events in December 2023. Modi's speech was translated to Tamil language and reached the people after a lag of a couple of seconds.

Even the Election Commission of India will be deploying AI at polling booths in Kolkata, West Bengal, to detect any anomalies or malpractices, reported TOI. 

On March 15, the Ministry of Electronics and Information Technology (MeitY) issued an advisory to intermediaries and AI platforms to ensure the sanctity of the electoral process is not compromised by deep fake technology.

MeitY's advisory asked entities to label and embed AI-generated content with metadata to identify that it has been created or modified using computer techniques. Companies such as Adobe, Microsoft and Google have incorporated metadata and watermarks in the content people generate using AI. 

But often these watermarks may not be visible to the naked eye. How then does one identify a deep fake or a cloned voice?

"Awareness is very important. Don't believe everything you see," said Jadoun. "It's very difficult to detect high-quality deep fakes… if that content is taking our emotions to the next level, then we shouldn't share it and do a quick fact-check. Your instinct is the best deep fake detection algorithm."