OpenAI has announced a broad crackdown on malicious networks abusing its artificial intelligence tools, banning accounts allegedly linked to Chinese law enforcement, international romance scams, and covert political influence operations. The disclosures came in a detailed report outlining how its flagship chatbot, ChatGPT, has been misused in sophisticated cybercrime and disinformation campaigns.
According to OpenAI, a small group of accounts believed to have originated in China used its AI models to gather information about U.S. individuals, online forums, and federal building locations. The same network reportedly generated English-language emails targeting state-level U.S. officials and policy analysts in business and finance, inviting them to participate in paid consultations as part of a broader social engineering effort. The accounts also sought advice on face-swapping software, signaling potential efforts to enhance deception tactics.
In one of the most politically sensitive revelations, OpenAI said it banned an account connected to an individual associated with Chinese law enforcement who allegedly orchestrated a covert influence operation targeting Sanae Takaichi. The campaign reportedly involved a smear effort against Japan’s first female prime minister, underscoring the growing use of generative AI in geopolitical information warfare.
The company also detailed a large-scale romance scam in Southeast Asia. A cluster of ChatGPT accounts was used to operate a fraudulent dating scheme targeting Indonesian men, with OpenAI estimating that hundreds of victims may have been defrauded each month. The scam allegedly relied on AI-generated promotional materials and advertisements for a fake dating service, luring users onto the platform and pressuring them to complete a series of costly tasks requiring substantial payments.
In addition, OpenAI reported that several banned accounts used its models to impersonate legitimate institutions, posing as law firms, real attorneys, and U.S. law enforcement officials. These operations targeted individuals who were already victims of fraud, exploiting their vulnerability by offering fake legal assistance or recovery services.
