Should Governments Use Algorithms in Public Service Delivery?

Never Without Policy Frameworks Governing the Use of Artificial Intelligence

Newsreel Asia Insight #111
Jan. 24, 2024

The integration of Artificial Intelligence (AI) in public service delivery promises efficiency and transparency, but the absence of policy frameworks governing its use can lead to grave injustices, particularly for the most vulnerable sections of society. In Telangana, an algorithm recently denied thousands of poor people access to food subsidies.

Telangana is using an algorithmic system, called Samagra Vedika, to profile over 30 million residents to determine their eligibility for welfare benefits. This system, initially designed for police use, was extended to welfare schemes. However, due to algorithmic errors, many legitimate beneficiaries were wrongfully denied their entitlements, according to a report published in Al Jazeera, citing the example of Bismillah Bee, a widow from Hyderabad.

Since the passing of her husband, a rickshaw driver, from mouth cancer two years ago, Bee has been earning her livelihood by peeling garlic for a local vendor. However, the algorithmic system erroneously identified her deceased husband, Syed Ali, as Syed Hyder Ali, who owned a car. Consequently, in 2021, the authorities, relying on the algorithm’s output, wrongly classified her husband as a car owner.

During the height of the COVID-19 pandemic, which coincided with the worsening of her husband’s cancer, Bee found herself in a relentless struggle with government offices. She endeavoured to prove that she did not own a car and was indeed living in poverty. Despite her efforts, the authorities placed their trust in the algorithm rather than her testimony.

AI systems, by their nature, are prone to errors, especially in contexts with inconsistent data. Moreover, the opaque nature of these systems makes it difficult for the affected individuals to seek redressal. The absence of transparency in AI algorithms and the data they use can lead to a trust deficit in public services.

Several countries have drafted policy frameworks governing the use of AI in public service delivery. Notably, governments in the U.S., the U.K., France, Japan and China have released their policy and strategy papers relating to AI.

The Indian government’s think tank, NITI Aayog, released a discussion paper in 2018, titled “National Strategy for Artificial Intelligence,” which outlines the potential of AI in India and suggests ways to harness its benefits while addressing the challenges. The document, however, serves more as a strategic discussion and guideline rather than a detailed policy framework specifically for AI in public service delivery.

To ensure the responsible use of AI, the government must adopt an approach that prioritises transparency, accountability and public engagement.

Firstly, it is essential for the government to mandate the disclosure of AI algorithms and the data used in public services. By doing so, it opens the door to independent audits, a crucial step in verifying the fairness and impartiality of these systems.

Alongside transparency, there must be a robust system of human oversight. This system would play a key role in reviewing and, where necessary, overriding decisions made by AI. The objective here is to ensure that AI serves as an aid to human judgment rather than a replacement. By doing so, the government can maintain a human touch in governance, ensuring that technology complements rather than dominates the decision-making process.

Furthermore, the establishment of clear and accessible channels for individuals to challenge AI-driven decisions is critical. Such mechanisms will provide a safety net for when things go wrong, and empowering citizens. Promptly addressing errors through these channels will also continually refine the AI systems in use.

Regular audits and impact assessments of AI systems are equally important. These evaluations are vital tools for assessing the effectiveness and fairness of AI applications. Through these assessments, the government can identify and address systemic issues, ensuring that AI tools are aligned with the public interest.

Finally, the government’s engagement with the public is paramount. By actively informing and educating citizens about how AI is utilised in public services, the government can demystify these technologies. This engagement will build a bridge between the government and its citizens, creating an environment where technology is understood, trusted and used responsibly.