India's top court angry after junior judge cites fake AI-generated orders

Source: Hacker News

Background

India’s Supreme Court has warned of legal consequences after a junior civil judge relied on AI‑generated judgments in a property dispute case from Andhra Pradesh. The top court described the incident as a matter of “institutional concern,” emphasizing that fake AI‑generated judgments threaten the integrity of the adjudicatory process.

Junior Judge’s Use of AI

• In August 2022, a junior civil judge in Vijayawada passed an order on a disputed property.
• The judge cited four past legal judgments, which were later discovered to be fabricated by an AI tool.
• Generative AI systems can “hallucinate,” presenting false information as fact and sometimes inventing sources.

High Court Response

The defendants challenged the order in the Andhra Pradesh High Court, pointing out the fabricated citations. The High Court:

1. Recognized the judge’s error as made in “good faith.”
2. Held that if the correct legal principles were applied, the presence of non‑existent citations could not alone overturn the decision.
3. Sought a report from the judge, who explained it was her first use of an AI tool and she believed the citations were genuine.
4. Emphasized the need for “the exercise of actual intelligence over artificial intelligence.”

Supreme Court Intervention

The Supreme Court, less forgiving, stayed the lower court’s order and labeled the use of AI in judgments not merely an “error in decision making” but “misconduct.” Key points from the Supreme Court’s order:

• The case raises “considerable institutional concern” because of the process of adjudication, not just the merits.
• Notices were issued to the Attorney General, Solicitor General, and the Bar Council of India for further examination.
• The Court reiterated its stance from a recent case where it criticized lawyers for using AI‑generated petitions, calling the practice “absolutely uncalled for.”

International Context

• United States (Oct 2024): Two federal judges were called out for AI‑generated errors in rulings. Bloomberg Law
• England and Wales (June 2025): The High Court warned lawyers against using AI‑generated case material after several fictitious citations appeared. The New York Times

Regulatory Developments in India

• In 2023, the Supreme Court released a white paper on AI in the Indian judiciary, outlining best practices and guidelines for judges, lawyers, and court staff.
• The paper stresses human oversight and the necessity of maintaining robust institutional safeguards when integrating AI tools.

For further reading, see the Supreme Court’s concerns about AI‑generated fake citations reported by LiveLaw: https://www.livelaw.in/top-stories/supreme-court-raises-concerns-over-lawyers-using-ai-generated-fake-citations-523441