ChatGPT launched in November 2022. By July 2023, criminals had built their own versions, stripped of safety rails and designed for fraud. Malicious AI tool mentions surged 200% on dark web forums in 2024, and scam software now sells for as little as $20.
The result isn't just new tools. It's an entire economy. Marketplaces like Huione Guarantee have processed over $70 billion in crypto transactions, selling everything from voice cloning software to fake identity documents. This is scam-as-a-service at industrial scale.
The Scale of the Problem
The Scam Tool Landscape
ChatGPT launched in November 2022. By July 2023, criminals had built their own versions, stripped of safety rails and designed for fraud.
The result: an ecosystem of malicious AI tools that make sophisticated scams accessible to anyone.
The Dark Web AI Arsenal
FraudGPT & WormGPT: Where It Started
WormGPT
On June 28, 2023, a user named "Last" announced WormGPT on Hackforums. Within days, it spread to ExploitIn, a Russian-speaking hacker forum. [6]
WormGPT is built on GPT-J and trained specifically on malware-related data. No safety filters. No content restrictions. Its specialties:
- Writing grammatically perfect phishing emails
- Generating polymorphic malware that mutates to evade detection
- Crafting BEC (business email compromise) messages
- Creating malicious code in multiple programming languages
The creator shut WormGPT down on August 9, 2023, citing unwanted attention. [6] But variants built on xAI's Grok and Mistral's Mixtral have emerged, selling for around €60. [1]
FraudGPT
FraudGPT appeared on July 22, 2023, advertised on dark web marketplaces and Telegram. The seller, using the alias "CanadianKingpin," marketed it as a "bot without limitations, rules, [and] boundaries." [4]
Pricing:
- $200 per month
- $1,700 per year
What it does:
- Writes undetectable malware
- Creates convincing phishing pages
- Finds vulnerable websites
- Generates scam letters and emails
- Teaches hacking techniques
Security researcher John Bambenek believes the same group operates both WormGPT and FraudGPT: "FraudGPT focused on short-duration, high-volume attacks such as phishing, while WormGPT focused on longer-term attacks with malware and ransomware." [4]
The Growing Ecosystem
FraudGPT and WormGPT spawned an industry. Current variants include:
- DarkBard: Based on Google's Bard AI [6]
- WolfGPT: Python-based, claims "complete confidentiality" [6]
- DarkGPT: Queries breached credentials on demand [7]
- Evil-GPT: General-purpose malicious assistant [7]
- XXXGPT: Focused on explicit content generation for sextortion [7]
- GhostGPT: 2025 variant advertising "no logs, instant responses" [8]
The pattern is consistent: strip safety rails from legitimate AI, train on malicious data, sell subscriptions.
Huione Guarantee: The Scammer Supermarket
Individual tools are one thing. Huione Guarantee is an entire marketplace, a one-stop shop for fraud infrastructure.
The Scale
- $70 billion in crypto transactions processed since 2021 [5]
- $375.9 million to scam technology vendors in 2024 alone [2]
- $98 billion processed by the wider Huione Group [9]
In May 2025, the U.S. Treasury designated Huione as a money-laundering operation. [9]
What's For Sale
Huione isn't just a payment processor. Vendors sell everything scammers need:
- AI face-changing software for live video calls ($200 lifetime subscription) [2]
- WormGPT licenses ($200 lifetime subscription spotted on Huione) [10]
- Money laundering services
- Social media account packages
- Stolen personal data
- Fake identity documents
In 2024, AI service providers were among Huione's most successful vendors, with revenue growing 1,900% year-over-year. [2]
The Trafficking Connection
Huione also sells tools for keeping captive workers in line, electric batons, ankle bracelets. Many pig butchering operations use trafficking victims forced to run scams. [10]
In November 2025, Myanmar military forces arrested nearly 1,600 foreign nationals during raids on scam compounds along the Thai border. [2]
The Competitor: Xinbi Guarantee
Huione isn't alone. Elliptic researchers uncovered Xinbi Guarantee, an $8 billion marketplace incorporated in Colorado, serving pig butchering scammers and North Korean hackers. [11]
Both Huione and Xinbi operated primarily through Telegram until the platform shut down thousands of their channels in late 2025. [11]
Voice Cloning Tools
Clone someone's voice with 3 seconds of audio. Use it to impersonate family members, executives, or anyone else.
Consumer Tools Weaponized
A March 2025 Consumer Reports investigation tested voice cloning from ElevenLabs, Descript, Lovo, PlayHT, Resemble AI, and Speechify. Finding: most lack meaningful safeguards. [12]
- ElevenLabs: $5 per voice clone, no technical consent verification [12]
- Speechify, PlayAI, Lovo: No technical mechanism to ensure consent, just a checkbox [12]
Berkeley researchers created voice clones of 220 real people using ElevenLabs. When volunteers listened to real versus fake voices, only 60% correctly identified the fakes: barely better than a coin flip. [12]
The Damage
- Global losses from deepfake-enabled fraud: $200 million in Q1 2025 alone [13]
- UK energy firm lost €220,000 after an employee received a deepfake call from their "CEO" [14]
- Engineering firm Arup lost $25 million to a deepfake video call [15]
Real Case
July 2025: Sharon Brightwell of Dover, Florida received a call from her "daughter" crying about a car accident. She sent $15,000 in cash before realizing she'd listened to an AI-generated imitation of her daughter's voice. [14]
Deepfake Video Tools
Real-time deepfakes let scammers impersonate anyone during live video calls. No expensive equipment required.
The Toolbox
- DeepFaceLive: Real-time face swapping during video calls [16]
- DeepFaceLab: High-quality impersonations making scammers look like CEOs [16]
- FaceSwap: Free, easy to use, pre-built models, active forums [16]
- Magicam, Amigo AI: Alter face, voice, gender, and race during live calls [16]
- Avatarify: Used in deepfake job interviews to pose as candidates with stolen identities [16]
These aren't underground tools. Most are marketed to content creators. Scammers just use them differently.
Identity Verification Bypass
One in 20 identity verification failures now involves deepfakes. [17] Fraudsters use face-swap tools to:
- Bypass "liveness" checks in banking KYC systems
- Create synthetic identity documents with AI-generated faces
- Conduct fake job interviews to infiltrate companies
With GPT-4o, creating fake IDs is "as easy as typing a prompt, no Photoshop required." [17]
The Brad Pitt Scam
A French woman lost nearly $1 million to a scammer posing as Brad Pitt, using AI-powered deepfake videos to maintain the deception. [17]
AI Phishing Kits
Phishing kits that took hours to build now take minutes. Some are subscription-based, others free with "premium" add-ons.
What's Changed
Intel 471 and Proofpoint researchers found AI-powered phishing kits sold openly on Telegram, featuring ChatGPT integrations and LinkedIn scraping. [18]
- Generative AI writes phishing emails in 5 minutes; humans take 16 hours for comparable quality [18]
- Kits include easy interfaces for training models on specific industries or companies [18]
- MFA bypass (Adversary-in-the-Middle) kits are increasingly bundled in [18]
AI vs. Human Red Teams
In March 2025, Hoxhunt's AI agents reached what they called the "Skynet Moment", for the first time in two years of testing, AI-created phishing campaigns outperformed elite human red teams against millions of global users. [19]
In November 2024, AI was 10% less effective than humans. By March 2025, AI was 24% more effective. [19]
The Numbers
- 78% of people open AI-generated phishing emails [20]
- 21% click on malicious links [20]
- 83% of phishing emails now advantage AI-generated content [20]
- 466% increase in phishing reports in Q1 2025 [21]
AI Fraud Agents
"In 2025, we saw the first appearance of AI fraud agents, autonomous systems that combine generative content, scripting, and behavioral mimicry to execute full verification attempts end-to-end." [22]
These aren't scripts requiring human operators. They're autonomous systems that can conduct entire fraud attempts independently.
Polymorphic Malware Generators
ChatGPT can create malware that mutates with every execution, evading signature-based detection.
The BlackMamba Proof-of-Concept
HYAS researchers built BlackMamba, a keylogger that uses an LLM to rewrite its own code at runtime. Every time it runs, it regenerates its malicious components. [23]
Result: Zero alerts or detections from industry-leading EDR solutions across multiple tests. [23]
How Detection Is Bypassed
CyberArk researchers demonstrated that ChatGPT's content filters can be bypassed through "prompt engineering." Worse, the ChatGPT API had weaker filters than the web version, "unclear why," but it made malware creation easier. [24]
The resulting polymorphic malware:
- Doesn't exhibit malicious behavior on disk
- Often shows no suspicious logic in memory
- Bypasses AMSI (Anti-Malware Scanning Interface)
- Evades signature-based detection entirely [24]
The Implication
Traditional antivirus is built around signatures, known patterns of malicious code. AI-generated polymorphic malware has no stable signature. Each instance is unique.
The technical barrier to creating evasive malware has collapsed.
Why This Matters
The Democratization of Fraud
The old model: sophisticated hackers with technical skills running complex operations.
The new model: anyone with $20 and a Telegram account buying scam-as-a-service.
Chainalysis head of fraud Elad Fouks: "GenAI enables the generation of realistic fake content, including websites and listings, making these attacks more convincing and harder to detect." [2]
The barriers are gone. The marketplaces are thriving. The tools keep improving.
Law Enforcement Response
In December 2024, the FBI issued an official alert warning that "criminals exploit generative AI to commit fraud on a larger scale, which increases the believability of their schemes." [25]
Enforcement actions to date:
- May 2025: U.S. Treasury designates Huione as money-laundering operation [9]
- October 2025: FinCEN targets Huione [9]
- November 2025: OFAC launches strike force targeting $10 billion scam industry [26]
- Late 2025: Telegram shuts thousands of Huione and Xinbi channels [11]
But the whack-a-mole problem persists. WormGPT's creator shut down, but variants emerged. Telegram closed channels, but new ones opened. The infrastructure adapts faster than enforcement can respond.
What You Can Actually Do
For Individuals
- Establish family safe words for emergency calls: if they can't say it, it's not them
- Verify through separate channels: call back on known numbers, not the one they called from
- Be suspicious of urgency: every scam creates pressure to act immediately
- Assume unsolicited contact is hostile: if they reached out first, verify everything
For Businesses
- Multi-person authorization for wire transfers over any threshold
- Callback verification using known numbers for any payment changes
- Never authorize payments based solely on video calls: deepfakes beat visual confirmation
- Employee training on deepfake/AI threats: the old tells (grammar, formatting) don't work anymore
For Security Teams
- Traditional signature-based detection is insufficient against polymorphic AI malware
- XDR solutions with behavioral analysis offer better protection than signature matching
- Assume phishing emails will be grammatically perfect, train employees on contextual red flags instead
The Bottom Line
Scam tools are cheap, abundant, and improving constantly. The marketplace ecosystem provides everything from AI assistants to money laundering to face-changing software.
The old indicators of fraud, grammatical errors, obvious fakes, technical barriers, are gone. What remains is process: verification protocols, callback procedures, family safe words.
AI broke the trust signals we relied on. The only defense is assuming nothing and verifying everything.
References
- SiliconANGLE - Malicious AI tool mentions surge 200% across dark web channels in 2024
- CNBC - Crypto scams likely hit a new record in 2024, driven by 'pig butchering' and AI
- Mastercard - New cybersecurity survey 2025: AI, scam fears and fraud risks
- Infosecurity Magazine - Dark Web Markets Offer New FraudGPT AI Tool
- CoinDesk - Crypto Scam Revenue From 'Pig Butchering,' AI Schemes Likely Grew in 2024
- Infosecurity Europe - The Dark Side of Generative AI: Five Malicious LLMs Found on the Dark Web
- Daily Security Review - FraudGPT, WormGPT, and Dark AI Models Fuel Surge in Cybercrime
- Big Think - Dark AI is fueling cybercrime, and accelerating the cybersecurity arms race
- Elliptic - Huione: the company behind the largest ever illicit online marketplace has launched a stablecoin
- Elliptic - Are pig butchering scammers using AI?
- Elliptic - Xinbi: The $8 Billion Colorado-Incorporated Marketplace for Pig-Butchering Scammers
- Consumer Reports - Voice Cloning Apps Make It Easy for Criminals to Steal Your Voice
- DeepStrike - Deepfake Statistics 2025: The Data Behind the AI Fraud Wave
- American Bar Association - The Rise of the AI-Cloned Voice Scam
- Fortune - Companies are increasingly falling victim to AI impersonation scams
- Spottable - Deepfake Tools Fraudsters Are Using in 2025-26
- Veriff - Real-time deepfake fraud in 2025: AI-driven scams
- Security Boulevard - AI-Powered Phishing Kits: The New Frontier in Social Engineering
- Hoxhunt - AI-Powered Phishing Outperforms Elite Cybercriminals in 2025
- CybelAngel - The Rise of AI-Powered Phishing 2025
- Sift - How AI Is Fueling Online Fraud in 2025
- KnowBe4 - Report: Sophisticated Fraud Attacks Are on the Rise
- HYAS - BlackMamba: Using AI to Generate Polymorphic Malware
- CyberArk - Chatting Our Way Into Creating a Polymorphic Malware
- FBI IC3 - Criminals Use Generative AI to Facilitate Financial Fraud
- Chainalysis - U.S. Launches New Strike Force to Combat $10 Billion Scam Industry