The New Era of Misinformation in the Digital Age
Imagine receiving a video call from your company’s senior executive asking you to approve an urgent payment. The face looks familiar, the voice sounds correct, and the person even speaks in the same professional style you expect. Or imagine receiving a phone call from a family member who sounds frightened and says they urgently need money.
A few years ago, verifying a person by hearing their voice or seeing their face on video might have felt like strong evidence. Generative AI is changing that assumption.
Modern AI can generate convincing emails, conversations, images, voices, videos, identities, and social media profiles. Criminals do not necessarily need to invent entirely new types of fraud. Instead, AI can make existing scams such as phishing, business email compromise, investment fraud, romance scams, impersonation, and identity fraud easier to scale and potentially more convincing. The UK National Cyber Security Centre similarly assesses that AI is primarily enhancing existing attacker techniques rather than creating completely new attack categories.
The financial scale of the wider problem is already significant. The FBI Internet Crime Complaint Center received more than one million complaints in 2025, with reported losses reaching approximately $20.9 billion. Within that dataset, 22,364 complaints were marked with an AI-related descriptor and represented more than $893 million in adjusted reported losses. Importantly, the FBI defines this descriptor as meaning the complaint contained a reference to AI; it does not mean AI was proven to have caused every dollar of those losses.
The result is a new challenge for digital trust. Misinformation is no longer only about convincing people that something false is true. In financial fraud, false information can be used to convince someone to transfer money, reveal credentials, approve a transaction, open an account, or trust an identity that does not actually exist.
What Is AI-Powered Online Fraud?
AI-powered fraud refers to scams in which artificial intelligence is used somewhere in the fraud process. The AI may generate phishing emails, create fake identities, clone voices, manipulate video, automate conversations, produce fraudulent documents, or personalize messages for particular victims.
The important distinction is that AI is usually an enabler rather than the entire attack. A scammer may still rely on traditional social engineering, stolen credentials, malicious websites, messaging applications, cryptocurrency transfers, or compromised accounts. AI simply makes parts of that operation faster or more realistic.
For example, a traditional phishing campaign might involve manually writing one generic email and sending it to thousands of people. With generative AI, attackers can potentially generate many versions of the same message, adapt the language and tone for different targets, and produce convincing business-style communication at scale. The NCSC has warned that language models can help criminals create convincing phishing and social-engineering messages, including content in languages in which the attacker may not personally be proficient.
That makes the security problem less about identifying badly written scam messages and more about verifying whether the person, organisation, request, and transaction behind a message are legitimate.
How Misinformation Becomes Financial Fraud
Misinformation becomes particularly dangerous when it creates enough trust to trigger an action. Consider a fake investment advertisement. AI might generate a convincing video showing a well-known executive apparently discussing an investment opportunity. A synthetic website may contain professionally written explanations, fake testimonials, AI-generated customer profiles, and chatbot-based support. The victim may then speak with what appears to be a real investment adviser before being instructed to transfer money.
Every individual piece of information is designed to reinforce the others. This is different from the old idea of a scam as a single suspicious email. Modern fraud can become a complete artificial information environment in which the advertisement, person, conversation, documentation, social proof, and customer support experience all appear consistent.
FBI data illustrates how this is already appearing in financial crime reporting. In 2025, investment fraud complaints carrying a reported AI nexus represented more than $632 million in losses. The FBI describes criminals using AI-generated videos and voices of celebrities, executives, and other trusted figures to support fraudulent investment opportunities.
This combination of synthetic media and financial manipulation is one reason misinformation and cybersecurity are increasingly overlapping.
AI-Generated Phishing and Smishing
Phishing remains one of the most common ways criminals attempt to steal credentials or direct victims toward fraudulent websites. AI does not fundamentally change how phishing works, but it can improve the quality and scalability of the content.
Attackers can generate professional emails that imitate banks, suppliers, delivery companies, employers, government departments, or internal corporate teams. Messages can also be rewritten automatically for different industries, languages, job roles, or situations.
The result is that older warning signs such as poor grammar or unnatural wording become less useful.
The FBI recorded 191,561 phishing or spoofing complaints in 2025, making it the largest crime category in its annual report by complaint count. That number does not mean those attacks were AI-generated, but it demonstrates the size of the existing attack channel into which generative AI can be introduced.
For businesses, the lesson is important: phishing awareness cannot depend only on teaching employees to look for spelling mistakes. Verification of links, sender domains, authentication requests, payment instructions, and unusual business processes is becoming more important.
Voice Cloning and Emergency Scams
Voice cloning creates another difficult problem because humans naturally associate familiar voices with familiar people.
A criminal may impersonate a family member who claims to have been arrested, injured, stranded abroad, or involved in an emergency. In a corporate environment, the same principle can be used to imitate a manager asking an employee to disclose information or approve a transaction.
The FTC specifically warns about scams in which criminals clone the voice of a family member and ask for urgent financial assistance. Its recommendation is simple but important: contact the supposed caller through a phone number that you already know belongs to them rather than trusting the incoming communication.
The FBI reported more than $5 million in 2025 losses from distress scams within the AI-related section of its annual report and noted that voice cloning can be used to imitate family members or close friends in emergency situations.
The security principle here is straightforward: voice should no longer be treated as authentication.
Deepfake Executives and Business Email Compromise
Business Email Compromise, commonly called BEC, traditionally involves an attacker impersonating or compromising a senior employee, supplier, lawyer, or business partner and convincing someone to transfer money.
AI can strengthen this attack by extending impersonation beyond email.
An employee might first receive a realistic message from a supposed executive and later receive a voice call or video interaction appearing to confirm the instruction. The channels reinforce each other, making the request feel more legitimate.
The FBI reported that businesses lost more than $30 million during 2025 to BEC scams identified as involving AI in the complaints submitted to IC3. It specifically notes that AI chat generators can create official-sounding messages while voice cloning can support fraudulent requests for wire payments.
This changes the meaning of transaction verification. A video call with a senior employee should not automatically override payment controls, just as an email should not. High-value transfers should remain subject to established authorization processes, independent verification, separation of duties, and transaction limits.
AI-Generated Investment Scams
Investment fraud is particularly suitable for AI-enabled misinformation because successful schemes depend heavily on perceived authority and credibility.
Fraudsters can create fake financial experts, artificial customer testimonials, manipulated screenshots, professionally written market reports, celebrity endorsements, investment communities, and synthetic video content. Generative systems can also support large numbers of personalized conversations with potential victims.
The FBI reported that investment fraud generated the largest loss category in its 2025 Internet Crime Report. It also specifically described the use of AI-generated celebrity, CEO, and trusted-person videos or voices in investment scams.
This means a professional-looking investment website or convincing video should never be considered evidence that an opportunity is legitimate. Verification needs to happen through independent regulatory, financial, and organisational channels.
Romance Scams and Synthetic Personalities
Romance and confidence scams have traditionally required considerable human effort because scammers need to maintain communication and slowly build trust. Generative AI changes that cost structure.
A scammer can use AI-generated photographs, conversation scripts, translation tools, and personalized responses to maintain an artificial identity. A single criminal operation may therefore be able to conduct more conversations while maintaining relatively convincing communication.
The FBI reported more than $19 million in 2025 losses associated with confidence or romance scams carrying a likely AI nexus. It specifically noted the use of AI-generated profiles and conversation scripts to make interactions more believable.
INTERPOL has also observed AI being used to generate online photos and profiles for romance scams, sextortion, and other social-engineering operations.
The danger is not simply that an image may be fake. The entire personality behind the image may be synthetic.
Synthetic Identities and Identity Verification
AI-powered fraud is also challenging systems that were designed to prove identity. Banks, fintech companies, cryptocurrency platforms, insurers, and other regulated businesses frequently ask customers to provide identity documents, photographs, or video during onboarding. Some services also perform facial matching or liveness checks.
Generative AI gives criminals additional ways to attack these controls. FinCEN has reported increased suspicious activity involving deepfake media and specifically warns that criminals have used generative AI to create or alter identity documents, photographs, and videos in attempts to bypass customer identification and verification processes. FinCEN also notes that publicly available generative tools have reduced the cost, time, and resources required to create high-quality synthetic content.
This creates an important architectural lesson for financial services: identity verification should not depend on a single image, document, or biometric check.
Device intelligence, historical behaviour, account activity, transaction characteristics, document verification, liveness controls, and ongoing risk monitoring may all need to contribute to the final risk decision.
AI-Enabled Employment and Recruitment Fraud
Employment processes are another emerging target. A fake employer can use AI to generate job advertisements, recruiter profiles, interview messages, company documentation, or employment contracts. At the same time, criminals may also use synthetic identities when applying for legitimate remote jobs.
The FBI’s 2025 report describes complaints involving voice spoofing or potential voice deepfakes during online interviews and notes that some employment-related attacks appear intended to gain access to private computer networks. It recorded almost $13 million in losses from AI-involved employment-type scams during the year.
For organisations, recruitment security therefore becomes part of cybersecurity. Identity verification, device provisioning, access approval, background checks, and least-privilege access should not be weakened simply because an applicant successfully completes a video interview.
Why AI Scams Are Harder to Detect
One major reason AI-enabled scams are challenging is that humans tend to verify information through consistency. If the email looks correct, the voice sounds correct, the video appears authentic, and the person knows internal information, confidence rises quickly.
AI can help attackers reproduce several of these signals at once. The NCSC notes that generative AI and deepfake technologies allow text, images, voice, and video to be created or modified with relatively low effort and increasing realism. It also warns that synthetic-content detection tools themselves can be ineffective or unreliable, meaning organisations should not depend on a single AI detector as their primary defence.
That is an important limitation. The answer to AI fraud is not simply another AI model labelled “deepfake detector.” Detection technology can play a role, but authentication procedures and business controls remain necessary when the consequences involve money, privileged access, sensitive information, or identity.
How an AI-Powered Scam Typically Works
Although individual attacks vary, many follow a recognisable process. First, the attacker gathers information. Public websites, social media profiles, company pages, previous data breaches, and other publicly available sources may provide names, job roles, relationships, writing patterns, photographs, or audio material.
AI can then help transform this information into convincing synthetic content. The attacker might generate messages, fake photographs, documents, audio, or video appropriate to the target.
Next comes trust-building. Instead of immediately demanding money, a fraudster may maintain a conversation, provide supporting documentation, move communication to another platform, or introduce additional fake identities that appear to confirm the original story.
Only when sufficient trust has been established does the financial action usually appear: transferring money, changing bank details, providing an authentication code, purchasing cryptocurrency, sharing credentials, or approving an internal payment.
Seen this way, the deepfake itself is not necessarily the attack. It is one component inside a broader social-engineering workflow.
What Individuals Can Do
The most useful defence is to separate communication from verification. If someone contacts you claiming to be a family member, bank employee, government official, colleague, or manager and requests money or sensitive information, verify the request using a communication channel that you select independently. For example, call a family member through a number already stored in your contacts or access your bank through its official application rather than using a link supplied in a message. This reflects FTC guidance for dealing with suspected voice-cloning scams.
Urgency should also increase suspicion rather than reduce verification. Scammers frequently depend on making victims feel that there is not enough time to check the situation properly. Most importantly, seeing or hearing someone should no longer be considered sufficient proof of identity for a high-risk financial action.
What Businesses Should Change
Businesses need to treat AI impersonation as a process-security problem rather than only a media-detection problem.
High-value transactions should require approval workflows that cannot be bypassed because an executive appears on a video call. Changes to supplier bank details should receive independent confirmation through existing contact information. Sensitive account recovery should use stronger authentication than voice recognition alone. Privileged system access should require phishing-resistant authentication where practical, and unusual transactions should trigger additional verification.
Financial institutions need additional layers because criminals may directly attack digital identity systems. FinCEN’s warning about synthetic documents and deepfake media demonstrates why identity verification should combine several signals rather than relying on a single submitted photograph or video.
Organisations should also assume that convincing phishing will continue to become cheaper to produce. The NCSC assesses that AI will make elements of cyber operations more effective and efficient and expects AI to increase the volume and impact of existing attacker techniques through 2027. Security training therefore needs to evolve from “spot the fake” toward “verify the action.”
Can AI Also Fight AI-Powered Fraud?
Yes, and it is already an important part of the defence.
Machine-learning systems can analyse transaction behaviour, device characteristics, login locations, identity documents, communication patterns, account relationships, and other risk signals. Deepfake-detection systems can examine media for inconsistencies, while behavioural models can identify unusual transaction patterns.
However, these systems should be treated as part of a layered architecture rather than perfect fraud detectors. NIST identifies several technical approaches to synthetic-content risk, including content authentication, provenance, watermarking, synthetic-content detection, metadata recording, testing, and auditing.
Content provenance is particularly interesting. Standards such as C2PA are designed to record information about the origin and history of digital media so that systems can provide stronger evidence about how content was created or modified.
But provenance has limitations too. Not every image, video, or audio file will contain trustworthy provenance information, existing media ecosystems need time to adopt these standards, and criminals may simply distribute content through channels that remove or do not support the metadata. The NCSC therefore describes Content Credentials as promising but not a complete solution by itself.
The Bigger Problem: Erosion of Digital Trust
The most damaging long-term effect of AI-generated misinformation may not be a particular deepfake.
It may be uncertain. If any image can potentially be generated, any voice can potentially be cloned, and any message can potentially be automated, people may gradually stop trusting digital evidence altogether.
The NCSC has specifically highlighted this broader integrity problem, noting that generative AI makes it increasingly difficult to distinguish authentic online content from manipulated material.
Financial systems therefore need to move toward a different trust model. Trust should increasingly come from verified identity, authenticated channels, transaction controls, provenance, independent confirmation, and behavioural evidence rather than simply from whether a message looks convincing. That is a much more durable defence because it remains useful even as synthetic media improves.
Future Outlook
AI-powered fraud will probably become more automated, personalized, and multimodal. A single scam may combine generated text, cloned voices, manipulated video, synthetic documents, automated chat systems, and fake identities rather than relying on one technique.
The NCSC assesses that criminal use of AI is highly likely to increase as these technologies become more widely adopted and that AI-enabled tools will expand capabilities across a broader range of threat actors. At the same time, it does not expect completely automated advanced cyberattacks to replace skilled human attackers in the near term; human-machine collaboration remains the more realistic model.
That distinction matters. Criminals are not becoming irrelevant because AI can run scams automatically. Instead, AI allows the same criminal operation to potentially research more targets, generate more convincing material, communicate at greater scale, and adapt attacks more quickly.
Defenders will therefore need similar automation, but with stronger governance, identity assurance, transaction controls, anomaly detection, content provenance, and human escalation for high-risk decisions.
Conclusion
AI has not invented financial fraud. It has changed the economics and credibility of it. Phishing, impersonation, romance scams, investment fraud, identity fraud, and business email compromise already existed. Generative AI can make those attacks easier to personalize, automate, translate, and support with convincing synthetic media.
Official reporting is beginning to show the scale of that transition. The FBI recorded 22,364 complaints containing AI-related information and more than $893 million in associated adjusted losses during 2025, while the FTC reported $3.5 billion in losses to impersonation scams overall during the same year. These datasets measure different things and should not be added together, but both highlight how financially significant digital impersonation and fraud have become.
The most important security change is therefore conceptual. We can no longer ask only, “Does this email, voice, image, or video look real?” We also need to ask, “Has this person, request, account, and transaction been independently verified?”
In the age of generative AI, something that looks real can increasingly be manufactured. Strong digital trust will depend less on appearance and more on verifiable evidence, secure processes, and multiple independent layers of authentication.
References
- Federal Bureau of Investigation, Internet Crime Complaint Center (2026), 2025 IC3 Annual Report.
- Federal Trade Commission (2026), FTC Data Show People Reported Losing $3.5 Billion to Imposter Scams in 2025.
- Federal Trade Commission (2024), Fighting Back Against Harmful Voice Cloning.
- Financial Crimes Enforcement Network (2024), Alert on Fraud Schemes Involving Deepfake Media Targeting Financial Institutions.
- National Cyber Security Centre (2025), Impact of AI on Cyber Threat from Now to 2027.
- National Cyber Security Centre (2025), Preserving Integrity in the Age of Generative AI.
- National Institute of Standards and Technology (2024), Reducing Risks Posed by Synthetic Content: An Overview of Technical Approaches to Digital Content Transparency.
- INTERPOL (2025), reporting on the growing use of AI-generated profiles, deepfakes, and synthetic media within social-engineering and scam operations.
