The Ministry of Electronics and Information Technology (MeitY) has issued a notice to social media platform X (formerly Twitter) over the circulation of obscene and sexually explicit content generated through its AI chatbot, Grok. Officials raised concerns that the AI tool was being misused to create offensive images and videos, particularly involving women and children, calling it a serious violation of Indian laws and digital safety standards.
MeitY has asked X to take action within 72 hours. The company must remove all unlawful content generated or shared via Grok, review its AI system to prevent further misuse, and act against users posting offensive material, including suspending or banning accounts. X is also required to submit a report to the ministry detailing the measures it has taken.
The government warned that non-compliance could result in legal action, including penalties under the Information Technology Act, the Indecent Representation of Women (Prohibition) Act, and the Protection of Children from Sexual Offences Act. X could also lose certain legal protections under Section 79 of the IT Act if it fails to address the issue.
The notice comes after complaints from lawmakers and users about how easily Grok can be prompted to generate sexualized or offensive images, including content involving minors. Officials emphasized that platforms offering AI services are responsible for preventing misuse and protecting digital safety.
The notice was sent to X’s Chief Compliance Officer in India, highlighting that AI platforms must follow Indian laws and ensure their tools are not used to create harmful content. MeitY stressed that AI systems should be closely monitored, and platforms remain accountable for the spread of obscene or illegal material.
This strict discipinary action is one way to regulate AI and safeguard the dignity and safety of vulnerable groups online, signaling that platforms cannot overlook the potential harms of generative AI.
