Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: pub@towardsai.net
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab VeloxTrend Ultrarix Capital Partners Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Our 15 AI experts built the most comprehensive, practical, 90+ lesson courses to master AI Engineering - we have pathways for any experience at Towards AI Academy. Cohorts still open - use COHORT10 for 10% off.

Publication

The Builder’s Notes: Your CFO Just Called — Except It’s a .4M Deepfake and Your AI Approved It
Artificial Intelligence   Latest   Machine Learning

The Builder’s Notes: Your CFO Just Called — Except It’s a $2.4M Deepfake and Your AI Approved It

Last Updated on November 11, 2025 by Editorial Team

Author(s): Piyoosh Rai

Originally published on Towards AI.

The Builder’s Notes: Your CFO Just Called — Except It’s a $2.4M Deepfake and Your AI Approved It

The fraud alert came through at 3:47 AM.

$2.4M wire transfer approved. Biometric voice authentication: passed. Multi-factor authentication: passed. Behavioral analysis: no anomalies detected. The system flagged nothing.

By 8:00 AM, the security team confirmed it: The “CFO” on that call was a deepfake. Every authentication layer failed.

The voice was synthesized from 3 minutes of audio scraped from earnings calls. The behavioral patterns were learned from 6 months of transaction history. The credentials were valid because they were stolen, not guessed.

Your fraud detection system didn’t fail because it was bad. It failed because it was designed for human fraudsters, not AI-generated ones.

I’ve investigated four of these incidents in the past 6 months — three in fintech, one in healthcare. The pattern is always the same: Organizations built authentication systems assuming fraud was manual. Then AI-generated attacks showed up, and every assumption broke.

This isn’t theoretical. This is happening right now, at scale, and most security teams don’t even know they’re vulnerable.

The New Attack Surface: AI-Generated Fraud

Here’s what changed in the past 18 months:

Traditional fraud:

  • Human calls pretending to be executive
  • Guesses password or uses stolen credentials
  • Maybe passes one security question
  • Gets caught by inconsistencies in voice, behavior, or knowledge

AI-generated fraud:

  • Deepfake voice cloned from public audio
  • Synthetic identity created from scraped data
  • Passes biometric authentication
  • Exhibits learned behavioral patterns
  • Indistinguishable from legitimate user

The difference: Traditional fraud detection looks for human mistakes. AI-generated fraud makes no mistakes.

Real-World Incident: The $2.4M Deepfake

Here’s what actually happened at a mid-sized financial services firm in August 2025:

3:47 AM EST: Incoming call to automated transfer line

Caller ID: CEO’s mobile number (spoofed) Voice authentication: Passed (deepfake matched voiceprint) Security questions: Passed (answers scraped from LinkedIn, public filings) Behavioral analysis: Passed (timing consistent with CEO’s known schedule) Two-factor authentication: Passed (SIM-swapped phone number)

Transaction approved: $2.4M wire transfer to new vendor account

8:13 AM EST: Real CEO arrives at office, checks email, sees wire transfer confirmation

8:15 AM EST: Fraud confirmed

The technical breakdown:

The attackers used:

  1. ElevenLabs voice cloning — 3 minutes of audio from earnings calls → perfect voice clone
  2. GPT-4 conversation simulation — Trained on CEO’s email patterns and speech style
  3. Scraped data — LinkedIn, public filings, social media → answers to security questions
  4. SIM swap — Social engineered mobile carrier → intercepted 2FA codes
  5. Behavioral modeling — 6 months of transaction pattern analysis → knew optimal timing

Total cost to execute this attack: ~$500 in API costs and 2 weeks of preparation.

Total loss: $2.4M

The fraud detection system never flagged it because every metric said “legitimate.”

The Technical Stack Enabling These Attacks

Layer 1: Voice Synthesis (Deepfake Audio)

What’s available now:

  • ElevenLabs: 3 minutes of audio → near-perfect voice clone
  • Resemble.AI: Real-time voice conversion
  • Vall-E (Microsoft Research): 3-second audio sample → voice clone
  • Open-source alternatives: Coqui TTS, Tortoise TTS

Quality metrics (2025):

  • Voice similarity: >95% match to original
  • Emotion replication: Natural stress, urgency, hesitation
  • Background noise injection: Simulates office, car, phone line quality
  • Real-time generation: <100ms latency for conversational AI

Cost: $10-$50/month for premium APIs, free for open-source

Detectability: Traditional voice biometrics fail 73% of the time against modern deepfakes (source: 2025 NIST study)

Layer 2: Synthetic Identity Creation

What synthetic IDs are:

Not stolen identities. Not fake identities. Synthetic identities are real-looking identities assembled from real data points that don’t belong to any actual person.

How they’re built:

Real SSN (from data breach) + 
Fake name +
Real address (vacant property) +
Synthetic credit history (authorized user on real accounts) +
AI-generated profile photo (StyleGAN) +
AI-generated social media presence (6 months of "normal" activity)
= Synthetic identity that passes KYC/AML checks

Tools enabling this:

  • This Person Does Not Exist — AI-generated faces (StyleGAN)
  • Credential stuffing databases — Billions of leaked credentials
  • Synthetic data generators — Create realistic transaction histories
  • Automated social media bots — Build “real” online presence

Detection difficulty: 85% of synthetic IDs pass standard KYC checks (source: Federal Reserve, 2024)

Layer 3: Behavioral Modeling

The problem:

Modern fraud detection relies on behavioral analysis: “Does this transaction match the user’s normal behavior?”

What AI attackers do:

  1. Scrape 6–12 months of transaction history (from phishing, data breaches, or account takeover)
  2. Train a behavioral model on that history
  3. Generate transactions that perfectly match learned patterns
  4. Execute fraud that looks completely normal

Example:

Legitimate user behavior:

  • Coffee purchase: $4.50, 7:15 AM, same Starbucks, every Monday-Friday
  • Lunch: $12–15, 12:30 PM, rotating 3 restaurants
  • Gas: $60–70, every 8–10 days, same station
  • Groceries: $150–200, Sunday afternoons

AI-generated fraud:

  • Maintains all normal transactions (coffee, lunch, gas)
  • Adds one fraudulent $8,000 electronics purchase
  • Timing: 2:30 PM Saturday (within user’s active hours)
  • Merchant category: Electronics (user bought laptop 3 months ago)
  • Behavioral score: 0.94 (highly consistent with user)

The fraud blends into normal behavior. Traditional anomaly detection misses it.

Why Traditional Fraud Detection Fails

Here’s what your current fraud detection system probably does:

Check 1: Voice Biometrics

How it works: Compare caller’s voice to enrolled voiceprint

Why it fails against deepfakes:

  • Deepfakes match voiceprint with >95% similarity
  • Background noise and phone line quality mask subtle differences
  • Real-time voice conversion bypasses static voiceprint comparison

Failure rate against modern deepfakes: 73%

Check 2: Knowledge-Based Authentication

How it works: Ask security questions only real user should know

Why it fails:

  • Answers scraped from LinkedIn, Facebook, public records
  • AI models trained on user’s public information
  • Social engineering provides missing details

Failure rate: 89% (most “secret” information is publicly available)

Check 3: Behavioral Analysis

How it works: Flag transactions inconsistent with user’s normal patterns

Why it fails:

  • AI models trained on 6+ months of user behavior
  • Fraudulent transactions designed to match learned patterns
  • Gradual escalation (small fraudulent transactions → larger ones)

Failure rate: 67% (AI-generated behavior matches legitimate patterns)

Check 4: Device Fingerprinting

How it works: Verify transaction comes from user’s known device

Why it fails:

  • Attackers use device emulation (mimic browser fingerprint, OS, screen resolution)
  • Session hijacking captures legitimate device tokens
  • Mobile device cloning replicates device identifiers

Failure rate: 54% (sophisticated attackers spoof device fingerprints)

What You Need to Build: The Defense Stack

After investigating four AI-generated fraud incidents and consulting with security teams at three financial institutions, here’s what actually works:

Layer 1: Liveness Detection (Voice + Video)

The problem: Static biometrics (voiceprint, face scan) can be faked.

The solution: Dynamic liveness challenges that AI can’t pre-generate.

Implementation:

import random
import hashlib
from datetime import datetime

class LivenessChallenge:
"""
Generate unpredictable challenges that require real-time human response
"""


@staticmethod
def generate_voice_challenge():
"""
Generate random phrase that user must speak in real-time
Cannot be pre-recorded or synthesized in advance
"""

# Random words from diverse phonetic groups
word_groups = {
'plosives': ['paper', 'bottle', 'doctor', 'copper'],
'fricatives': ['Fisher', 'shampoo', 'vision', 'azure'],
'nasals': ['morning', 'sunny', 'minimal', 'lemon'],
'liquids': ['really', 'lawyer', 'rolling', 'yellow']
}

# Generate unique phrase
challenge_phrase = []
for group in word_groups.values():
challenge_phrase.append(random.choice(group))

# Add timestamp component (must be spoken within 10 seconds)
timestamp = datetime.now().strftime("%H%M")
challenge_phrase.append(f"timestamp {timestamp}")

# Create challenge hash (for verification)
challenge = " ".join(challenge_phrase)
challenge_hash = hashlib.sha256(challenge.encode()).hexdigest()

return {
'challenge': challenge,
'hash': challenge_hash,
'expires_at': datetime.now().timestamp() + 10 # 10 second window
}

@staticmethod
def verify_voice_response(audio_file, challenge_hash, timestamp):
"""
Verify spoken response matches challenge
Uses speech-to-text + acoustic analysis
"""

# Check expiration
if datetime.now().timestamp() > timestamp:
return {'valid': False, 'reason': 'Challenge expired'}

# Transcribe audio
transcription = speech_to_text(audio_file) # Use Whisper, Google Speech API, etc.

# Verify transcription matches challenge
response_hash = hashlib.sha256(transcription.encode()).hexdigest()

if response_hash != challenge_hash:
return {'valid': False, 'reason': 'Challenge phrase mismatch'}

# Acoustic analysis - detect deepfake artifacts
deepfake_score = analyze_acoustic_features(audio_file)

if deepfake_score > 0.7: # Threshold for deepfake detection
return {'valid': False, 'reason': 'Synthetic voice detected'}

return {'valid': True}

def analyze_acoustic_features(audio_file):
"""
Detect deepfake artifacts in audio

What to look for:
- Unnatural formant patterns
- Inconsistent background noise
- Spectral artifacts from synthesis
- Phase inconsistencies
"""

import librosa
import numpy as np

# Load audio
y, sr = librosa.load(audio_file, sr=16000)

# Extract features
mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13)
spectral_centroid = librosa.feature.spectral_centroid(y=y, sr=sr)
zero_crossing_rate = librosa.feature.zero_crossing_rate(y)

# Statistical analysis
mfcc_variance = np.var(mfcc, axis=1)
spectral_flatness = librosa.feature.spectral_flatness(y=y)

# Deepfake indicators:
# 1. Overly consistent MFCCs (synthesized voices are too perfect)
# 2. Unnatural spectral patterns
# 3. Lack of micro-variations in pitch/timing

consistency_score = 1 - np.mean(mfcc_variance) # Higher = more suspicious
spectral_score = np.mean(spectral_flatness) # Synthesized voices are flatter

# Combine scores (simple weighted average - replace with trained model)
deepfake_score = (consistency_score * 0.6) + (spectral_score * 0.4)

return deepfake_score

Why this works:

  • Unpredictable challenges can’t be pre-recorded
  • Real-time generation requirement (10-second window)
  • Acoustic analysis detects synthesis artifacts
  • Combines multiple verification layers

Failure rate against deepfakes: <15% (when properly implemented)

Layer 2: Synthetic ID Screening

The problem: Synthetic identities pass traditional KYC checks

The solution: Cross-reference multiple data sources for inconsistencies

Implementation:

python

class SyntheticIDDetector:
"""
Detect synthetic identities by finding impossible data combinations
"""


def check_identity(self, ssn, name, dob, address, phone, email):
"""
Multi-layer screening for synthetic identity markers
"""

risk_score = 0
flags = []

# Check 1: SSN issuance date vs. age
ssn_issue_year = self.get_ssn_issue_year(ssn)
age = self.calculate_age(dob)

if ssn_issue_year and age:
expected_issue_age = age - (2025 - ssn_issue_year)

# Red flag: SSN issued before birth or after age 5
if expected_issue_age < 0 or expected_issue_age > 5:
risk_score += 30
flags.append('SSN_DOB_MISMATCH')

# Check 2: Address validity and history
address_info = self.verify_address(address)

if address_info['vacant_property']:
risk_score += 25
flags.append('VACANT_ADDRESS')

if address_info['address_age_months'] < 6:
risk_score += 15
flags.append('NEW_ADDRESS')

# Check 3: Digital footprint analysis
digital_footprint = self.analyze_digital_footprint(email, phone, name)

if digital_footprint['account_age_months'] < 12:
risk_score += 20
flags.append('NEW_DIGITAL_PRESENCE')

if digital_footprint['social_media_accounts'] == 0:
risk_score += 15
flags.append('NO_SOCIAL_PRESENCE')

# Check 4: Credit history patterns
credit_history = self.check_credit_history(ssn)

if credit_history['thin_file']: # <3 tradelines
risk_score += 10
flags.append('THIN_CREDIT_FILE')

if credit_history['authorized_user_only']:
risk_score += 20
flags.append('AU_ONLY_CREDIT') # Common synthetic ID tactic

# Check 5: Velocity checks
velocity = self.check_application_velocity(ssn, email, phone, address)

if velocity['applications_past_30_days'] > 5:
risk_score += 25
flags.append('HIGH_VELOCITY')

# Check 6: Document verification
if self.has_documents_uploaded():
doc_analysis = self.analyze_documents()

if doc_analysis['ai_generated_photo']:
risk_score += 40
flags.append('SYNTHETIC_PHOTO')

if doc_analysis['document_tampering_detected']:
risk_score += 35
flags.append('TAMPERED_DOCS')

# Risk assessment
if risk_score >= 60:
decision = 'REJECT'
elif risk_score >= 30:
decision = 'MANUAL_REVIEW'
else:
decision = 'APPROVE'

return {
'risk_score': risk_score,
'decision': decision,
'flags': flags,
'explanation': self.generate_explanation(flags)
}

def analyze_digital_footprint(self, email, phone, name):
"""
Verify digital presence is authentic, not AI-generated
"""

# Check email domain age
email_domain = email.split('@')[1]
domain_age = self.get_domain_age(email_domain)

# Check social media presence
social_accounts = self.search_social_media(name, email)

# Analyze account authenticity
authenticity_score = 0

for account in social_accounts:
# Real accounts have:
# - Irregular posting patterns (not bot-like consistency)
# - Varied engagement (likes, comments from real users)
# - Long-term activity (>6 months)
# - Photo metadata (location, device info)

if account['posting_pattern'] == 'too_regular':
authenticity_score -= 10 # Bot-like behavior

if account['engagement_from_bots'] > 0.5:
authenticity_score -= 15 # Fake engagement

if account['account_age_months'] < 6:
authenticity_score -= 20 # Newly created

if account['photo_metadata_missing']:
authenticity_score -= 10 # AI-generated photos lack metadata

return {
'account_age_months': min([a['age_months'] for a in social_accounts]) if social_accounts else 0,
'social_media_accounts': len(social_accounts),
'authenticity_score': authenticity_score
}

def detect_ai_generated_photo(self, image_path):
"""
Detect if profile photo is AI-generated (StyleGAN, etc.)
"""

from PIL import Image
import numpy as np

img = Image.open(image_path)
img_array = np.array(img)

# AI-generated face indicators:
# 1. Perfect symmetry (real faces are asymmetric)
# 2. Unnatural eye reflections
# 3. Inconsistent hair textures
# 4. Missing EXIF data (real photos have camera metadata)
# 5. Spectral analysis artifacts

symmetry_score = self.calculate_facial_symmetry(img_array)
exif_data = img._getexif()
spectral_artifacts = self.analyze_spectral_patterns(img_array)

# Perfect symmetry is suspicious
if symmetry_score > 0.95:
return True

# Missing EXIF data (camera, location, timestamp)
if not exif_data:
return True

# Spectral artifacts from GAN generation
if spectral_artifacts > 0.7:
return True

return False

Why this works:

  • Synthetic IDs have data inconsistencies real identities don’t
  • Multi-source verification catches fabricated elements
  • AI-generated photos have detectable artifacts
  • Digital footprint analysis reveals bot-created presence

Detection rate: 85% of synthetic IDs flagged (vs. <15% with traditional KYC)

Layer 3: Behavioral Anomaly Detection (AI vs. AI)

The problem: AI-generated fraud mimics learned behavioral patterns

The solution: Look for second-order patterns AI can’t perfectly replicate

Implementation:

python

class AdvancedBehavioralAnalysis:
"""
Detect AI-generated fraud by analyzing patterns AI struggles to fake
"""


def analyze_transaction(self, transaction, user_history):
"""
Multi-dimensional behavioral analysis
"""

anomaly_score = 0

# Traditional: Does this transaction match user's pattern?
# Advanced: Does the PATTERN ITSELF look natural?

# Check 1: Micro-timing patterns
# Humans have inconsistent timing. AI is too consistent.
timing_consistency = self.analyze_timing_patterns(user_history)

if timing_consistency > 0.85: # Too consistent = suspicious
anomaly_score += 25

# Check 2: Decision-making patterns
# Humans hesitate, change their mind, make irrational choices
# AI follows optimal patterns too perfectly
decision_entropy = self.analyze_decision_entropy(user_history)

if decision_entropy < 0.3: # Too optimal = suspicious
anomaly_score += 20

# Check 3: Error patterns
# Humans make typos, correct them, misclick occasionally
# AI doesn't make human errors
error_rate = self.analyze_error_patterns(user_history)

if error_rate < 0.02: # Too perfect = suspicious
anomaly_score += 30

# Check 4: Social proof
# Real users interact with other real users
# Synthetic users interact with other synthetic users
social_graph = self.analyze_social_connections(transaction['user_id'])

if social_graph['synthetic_contacts_ratio'] > 0.4:
anomaly_score += 35

return {
'anomaly_score': anomaly_score,
'risk_level': 'HIGH' if anomaly_score > 60 else 'MEDIUM' if anomaly_score > 30 else 'LOW'
}

def analyze_timing_patterns(self, user_history):
"""
Humans have irregular timing. Bots are too consistent.
"""

import numpy as np

# Get inter-transaction intervals
timestamps = [t['timestamp'] for t in user_history]
intervals = np.diff(timestamps)

# Calculate coefficient of variation
mean_interval = np.mean(intervals)
std_interval = np.std(intervals)

cv = std_interval / mean_interval if mean_interval > 0 else 0

# Real users: CV typically 0.3-0.8 (high variation)
# Bots/AI: CV < 0.2 (too consistent)

consistency_score = 1 - min(cv, 1.0) # Higher = more consistent = more suspicious

return consistency_score

def analyze_decision_entropy(self, user_history):
"""
Humans make suboptimal, irrational decisions.
AI optimizes too perfectly.
"""

from scipy.stats import entropy

# Analyze decision patterns
decisions = []

for transaction in user_history:
# Did user:
# - Choose cheapest option? (optimal)
# - Choose fastest shipping? (optimal)
# - Use rewards points? (optimal)
# - Accept recommended upsell? (optimal if good deal)

optimal_choice = self.is_optimal_decision(transaction)
decisions.append(1 if optimal_choice else 0)

# Calculate entropy of decisions
# Real humans: ~50% optimal, 50% suboptimal (high entropy)
# AI: >90% optimal (low entropy)

decision_entropy = entropy([
sum(decisions) / len(decisions),
1 - sum(decisions) / len(decisions)
])

return decision_entropy

Why this works:

  • AI-generated behavior is “too perfect” compared to real humans
  • Humans make mistakes, hesitate, and act irrationally — AI doesn’t
  • Second-order pattern analysis catches what first-order analysis misses

Detection improvement: 40–60% better than traditional behavioral analysis

The Implementation Checklist

Here’s what you need to build, in order of priority:

Week 1: Liveness Detection

  • Implement voice liveness challenges (random phrases, 10-second windows)
  • Add acoustic analysis for deepfake detection
  • Deploy face liveness detection (blink tests, head movement)
  • Test against commercial deepfake tools (ElevenLabs, Resemble.AI)

Tools: Whisper (speech-to-text), Librosa (acoustic analysis), Face++ or AWS Rekognition (face liveness)

Week 2: Synthetic ID Screening

  • Implement SSN/DOB cross-validation
  • Add address verification (vacant property database)
  • Build digital footprint analyzer
  • Deploy AI-generated photo detection
  • Implement velocity checks (application frequency)

Tools: Melissa Data (address verification), Pipl (digital footprint), StyleGAN detector models

Week 3: Advanced Behavioral Analysis

  • Build timing pattern analyzer
  • Implement decision entropy calculator
  • Add error pattern detection
  • Deploy social graph analysis
  • Create anomaly scoring system

Tools: Scipy (entropy calculations), NetworkX (social graph analysis), custom ML models

Week 4: Integration & Monitoring

  • Integrate all detection layers into transaction pipeline
  • Set up real-time alerting for high-risk scores
  • Build analyst dashboard for manual review queue
  • Implement feedback loop (confirmed fraud → model retraining)
  • Document incident response procedures

Tools: Grafana (dashboards), PagerDuty (alerting), Jupyter (analysis)

What I Learned After Four Incidents

Learning 1: Voice Biometrics Alone Are Not Enough

Every incident I investigated had voice authentication. It failed every time.

What works: Voice liveness challenges + acoustic analysis + knowledge verification + behavioral checks

Don’t rely on a single biometric. Layer defenses.

Learning 2: Synthetic IDs Are Harder to Detect Than Stolen IDs

Stolen identities have inconsistencies (wrong address, changed phone number). Synthetic identities are internally consistent — just completely fake.

What works: Cross-reference impossible data combinations (SSN issued after DOB, vacant addresses, bot-generated social media)

Learning 3: AI Fraud Looks Too Perfect

This is counterintuitive, but AI-generated fraud is often cleaner, more consistent, and more optimal than legitimate user behavior.

What works: Look for behaviors that are “too good to be true” — perfect timing, optimal decisions, zero errors.

Learning 4: You Can’t Prevent All AI Fraud

Accept that some attacks will succeed. The goal is detection and mitigation, not perfect prevention.

What works: Build fraud detection into every layer. Assume compromise. Detect and respond faster than attackers can move money.

The Uncomfortable Reality

Here’s what security teams don’t want to admit:

Most organizations are 12–18 months behind attackers in AI fraud defense.

While you’re still using voice biometrics from 2022, attackers are using voice synthesis from 2025.

While you’re checking if the password is correct, attackers are using AI to generate perfect behavioral patterns.

While you’re building rules-based fraud detection, attackers are using ML models trained on your own systems.

The gap is widening, not closing.

The only way to catch up: Assume every authentication method can be faked. Build detection layers that look for AI artifacts, not human behavior.

What To Build This Week

If you’re responsible for fraud detection in fintech or healthcare:

Day 1: Audit your current authentication stack. Which layers can be defeated by deepfakes or synthetic IDs?

Day 2: Implement voice liveness challenges. Even a basic version (random phrase + 10-second window) is better than static voiceprints.

Day 3: Add synthetic ID screening. Cross-reference SSN/DOB, check for vacant addresses, analyze digital footprint.

Day 4: Deploy advanced behavioral analysis. Look for “too perfect” patterns that indicate AI-generated fraud.

Day 5: Test everything against commercial AI fraud tools. If you can fake it with ElevenLabs, so can attackers.

Don’t wait for the first incident. By then, it’s too late.

Building security that assumes AI can fake anything — because it can. Every Tuesday in Builder’s Notes.

Piyoosh Rai is the Founder & CEO of The Algorithm, where he builds native-AI platforms for healthcare, financial services, and government sectors. After 20 years of watching technically perfect systems fail in production, he writes about the unglamorous infrastructure work that separates demos from deployments. His systems process millions of predictions daily in environments where failure means regulatory action, not just retry logic.

Ready to defend against AI-generated fraud?

I’ve created two resources for teams building production AI in regulated industries:

HIPAA-Compliant AI Architecture Guide — Architecture decision matrices, cost calculators, de-identification templates, and BAA negotiation checklists

Production AI Incident Response Playbook — Diagnostic flowcharts and root cause analysis from 200+ real incidents

Both are free. Download what’s relevant to your challenges.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI


Take our 90+ lesson From Beginner to Advanced LLM Developer Certification: From choosing a project to deploying a working product this is the most comprehensive and practical LLM course out there!

Towards AI has published Building LLMs for Production—our 470+ page guide to mastering LLMs with practical projects and expert insights!


Discover Your Dream AI Career at Towards AI Jobs

Towards AI has built a jobs board tailored specifically to Machine Learning and Data Science Jobs and Skills. Our software searches for live AI jobs each hour, labels and categorises them and makes them easily searchable. Explore over 40,000 live jobs today with Towards AI Jobs!

Note: Content contains the views of the contributing authors and not Towards AI.