Generative AI has entered cybersecurity with full force, and like every powerful technology, it comes with its pros and cons. On one side, attackers are already experimenting with AI to generate malware, craft phishing campaigns, and create deepfakes that erode trust. On the other hand, defenders are beginning to use AI to scale penetration testing, accelerate application security, and reduce the pain of compliance. The stakes are high. A recent ForeScout Vedere Labs 2025 report showed zero-day exploits have risen 46% year over year — a clear signal that attackers are accelerating. At the same time, Gartner predicts that by 2028, 70% of enterprises will adopt AI for security operations. ForeScout Vedere Labs 2025 report Gartner predicts The reality sits in between: AI is already changing penetration testing, application security, and compliance — but it’s not a replacement for human expertise. Instead, it’s a force multiplier, reshaping how quickly and effectively security teams can discover weaknesses, meet regulatory obligations, and prepare for adversaries that are also harnessing AI. The Dual-Use Nature of Generative AI Generative AI in cybersecurity is best understood as a dual-use technology — it amplifies both attack and defense capabilities. GenAI for Attackers AI lowers barriers by generating sophisticated phishing emails, fake personas, malicious code, and even automated exploit chains. Tools like CAI (Cognitive Autonomous Intelligence) demonstrate how autonomous agents can be tasked with scanning, exploiting, and pivoting through systems — blurring the line between proof-of-concept research and adversary capability. BlackMamba (an AI-generated polymorphic keylogger) and WormGPT (marketed on underground forums as “ChatGPT for cybercrime”) have already shown what’s possible. CAI (Cognitive Autonomous Intelligence) BlackMamba WormGPT GenAI for Defenders AI provides scale, speed, and intelligence. Beyond SOC copilots, AI is being embedded directly into the software development lifecycle (SDLC) via AI security code reviewers and AI-powered vulnerability scanners. GitHub Copilot (with secure coding checks), CodiumAI, and Snyk Code AI catch issues earlier, reducing downstream remediation costs. Microsoft’s Security Copilot helps analysts triage alerts and accelerate investigations. GitHub Copilot CodiumAI Snyk Code AI Security Copilot This duality is why many experts warn of an “AI arms race” between security teams and cybercriminals — where speed, automation, and adaptability may decide outcomes. Offensive Security & Penetration Testing Penetration testing has traditionally been time-intensive, relying on skilled specialists to probe for vulnerabilities in networks, applications, and infrastructure. AI is shifting the tempo. Large language models and autonomous agents can now: Large language models and autonomous agents can now: Generate custom exploits and payloads on demand.
Mimic phishing and social engineering campaigns at scale.
Run fuzzing routines to simulate zero-day vulnerabilities before attackers do. Generate custom exploits and payloads on demand. Mimic phishing and social engineering campaigns at scale. Run fuzzing routines to simulate zero-day vulnerabilities before attackers do. A striking proof point is XBOW, the autonomous AI pentester that recently climbed to #1 on HackerOne’s U.S. leaderboard. In controlled benchmarks, XBOW solved 88 out of 104 challenges in just 28 minutes — a task that took a seasoned human tester over 40 hours. In live programs, it has already submitted over a thousand vulnerability reports, including a zero-day in Palo Alto’s GlobalProtect VPN. XBOW #1 Palo Alto’s GlobalProtect VPN Other examples include: Other examples include: AutoSploit, an early attempt at AI-assisted exploitation pairing Shodan with Metasploit.
Bug bounty hunters using LLMs as copilots for reconnaissance and payload generation.
MITRE ATLAS, a framework mapping how adversaries might use AI in cyberattacks. AutoSploit, an early attempt at AI-assisted exploitation pairing Shodan with Metasploit. AutoSploit Bug bounty hunters using LLMs as copilots for reconnaissance and payload generation. LLMs as copilots MITRE ATLAS, a framework mapping how adversaries might use AI in cyberattacks. MITRE ATLAS Yet despite its speed and precision, tools like XBOW still require human oversight. Automated results must be validated, prioritized, and — critically — mapped to regulatory and business risk. Without that layer, organizations risk drowning in noise or overlooking vulnerabilities that matter most for compliance and trust. This is the shape of penetration testing to come: faster, AI-augmented discovery coupled with expert judgment to make results meaningful for businesses under pressure from regulators and partners. How Can Generative AI Be Used in Application Security Application security (AppSec) is another area seeing rapid AI adoption. The software supply chain has grown too vast and complex for purely manual testing, and generative AI is stepping in as a copilot. Key applications include: Key applications include: Code analysis and secure SDLC copilots: GitHub Copilot and CodiumAI spot insecure patterns before code reaches production.
AI-powered security scanners: Snyk Code AI and ShiftLeft Scan continuously crawl apps and APIs, flagging vulnerabilities in real time.
Auto-patching suggestions: GitHub now generates AI-driven pull requests suggesting secure fixes.
Testing LLM-based apps: The rise of AI-powered chatbots introduces new risks. Prompt injection attacks are already in the wild. OWASP responded with the first Top 10 for LLM Applications in 2023.
API fuzzing and zero-day simulations: Tools like Peach Fuzzer and AI-driven agents autonomously generate malformed inputs at scale. Code analysis and secure SDLC copilots: GitHub Copilot and CodiumAI spot insecure patterns before code reaches production. GitHub Copilot CodiumAI AI-powered security scanners: Snyk Code AI and ShiftLeft Scan continuously crawl apps and APIs, flagging vulnerabilities in real time. Snyk Code AI ShiftLeft Scan Auto-patching suggestions: GitHub now generates AI-driven pull requests suggesting secure fixes. Testing LLM-based apps: The rise of AI-powered chatbots introduces new risks. Prompt injection attacks are already in the wild. OWASP responded with the first Top 10 for LLM Applications in 2023. Prompt injection attacks API fuzzing and zero-day simulations: Tools like Peach Fuzzer and AI-driven agents autonomously generate malformed inputs at scale. Peach Fuzzer The promise is efficiency — but the challenge is trust. An AI-generated patch may fix one issue while creating another. That’s why AI is best deployed as an accelerator in AppSec, with humans validating its findings and ensuring fixes align with compliance frameworks like ISO 27001, HIPAA, or FDA MDR/IVDR for medical software. ISO 27001 HIPAA FDA MDR/IVDR How Can Generative AI Be Used in Compliance & Governance Beyond pentesting and AppSec, AI is finding a role in the often overlooked world of compliance. For companies in healthtech, biotech, or fintech, compliance can make or break growth — and AI is beginning to reduce the heavy lift. Emerging applications include: Emerging applications include: Automating evidence collection for ISO 27001, SOC 2, HIPAA, and GDPR.
Mapping vulnerabilities to controls: Linking pentest findings directly to FDA SPDF or ISO clauses.
Generating audit-ready reports: Platforms like Vendict, Scrut, and Thoropass use AI to translate security posture into regulator-friendly documentation. Automating evidence collection for ISO 27001, SOC 2, HIPAA, and GDPR. ISO 27001 SOC 2 Mapping vulnerabilities to controls: Linking pentest findings directly to FDA SPDF or ISO clauses. FDA SPDF Generating audit-ready reports: Platforms like Vendict, Scrut, and Thoropass use AI to translate security posture into regulator-friendly documentation. Vendict Scrut Thoropass This is particularly powerful in genomics or diagnostics, where startups face heavy regulatory burden and need to show both security and compliance maturity to win partnerships or funding. Industry Examples The use of AI in cybersecurity isn’t hypothetical — it’s playing out across industries today: IBM, NVIDIA, Accenture: AI copilots for SOC operations and threat detection.
Vendict, Scrut, Thoropass: Embedding AI in GRC workflows.
Governments and defense sectors: DARPA’s AI Cyber Challenge (AIxCC) uses AI for red-teaming resilience.
Adversaries: North Korean APT groups and organized fraud rings are already using AI for smishing, phishing, and deepfake scams.
Case study: In 2019, a UK energy firm lost $240,000 after a CEO voice deepfake tricked staff into wiring money. IBM, NVIDIA, Accenture: AI copilots for SOC operations and threat detection. IBM NVIDIA Accenture Vendict, Scrut, Thoropass: Embedding AI in GRC workflows. Vendict Scrut Thoropass Governments and defense sectors: DARPA’s AI Cyber Challenge (AIxCC) uses AI for red-teaming resilience. DARPA’s AI Cyber Challenge (AIxCC) Adversaries: North Korean APT groups and organized fraud rings are already using AI for smishing, phishing, and deepfake scams. North Korean APT groups Case study: In 2019, a UK energy firm lost $240,000 after a CEO voice deepfake tricked staff into wiring money. UK energy firm lost $240,000 Emerging Risks of Generative AI in Cybersecurity With opportunity comes risk. AI introduces new attack vectors and amplifies existing ones: AI-powered phishing and social engineering: Deepfake audio scams are growing in sophistication.
Prompt injection and model manipulation: OWASP’s LLM Top 10 highlights prompt injection as the #1 risk.
Bias and privacy: Training models on sensitive datasets risks compliance violations under GDPR.
Over-reliance: Treating AI outputs as gospel risks blind spots and false positives.
Hallucinations: Studies show AI copilots fabricate vulnerabilities or fixes.
Dependency risk: SaaS outages or API shifts in AI platforms can disrupt pipelines. AI-powered phishing and social engineering: Deepfake audio scams are growing in sophistication. Deepfake audio scams Prompt injection and model manipulation: OWASP’s LLM Top 10 highlights prompt injection as the #1 risk. OWASP’s LLM Top 10 #1 Bias and privacy: Training models on sensitive datasets risks compliance violations under GDPR. GDPR Over-reliance: Treating AI outputs as gospel risks blind spots and false positives. Hallucinations: Studies show AI copilots fabricate vulnerabilities or fixes. Studies show Dependency risk: SaaS outages or API shifts in AI platforms can disrupt pipelines. Best Practice Strategy for Secure AI Adoption To adopt AI in pentesting, AppSec, or compliance responsibly, organizations should: Keep humans in the loop: Validate AI findings before action.
Govern “shadow AI”: Prevent unsanctioned AI tool use (e.g., Samsung’s data leak into ChatGPT).
Run continuous simulations: Microsoft’s AI Red Team tests copilots for adversarial risks.
Integrate into secure SDLC: Deploy AI reviewers and scanners directly in dev pipelines.
Apply governance frameworks: NIST AI Risk Management Framework and ENISA’s AI Security Guidelines help ensure ethical and safe AI use. Keep humans in the loop: Validate AI findings before action. Govern “shadow AI”: Prevent unsanctioned AI tool use (e.g., Samsung’s data leak into ChatGPT). Samsung’s data leak into ChatGPT Run continuous simulations: Microsoft’s AI Red Team tests copilots for adversarial risks. Microsoft’s AI Red Team Integrate into secure SDLC: Deploy AI reviewers and scanners directly in dev pipelines. Apply governance frameworks: NIST AI Risk Management Framework and ENISA’s AI Security Guidelines help ensure ethical and safe AI use. NIST AI Risk Management Framework ENISA’s AI Security Guidelines Conclusion & Outlook So, how can generative AI be used in cybersecurity? It won’t replace penetration testers, application security engineers, or compliance leads. But it will accelerate their work, expand their coverage, and reshape how vulnerabilities are found and reported. The winners won’t be those who adopt AI blindly, nor those who ignore it. They’ll be the organizations that harness AI as a trusted copilot — combining speed with human judgment, technical depth with regulatory alignment, and automation with accountability. By 2030, AI-driven pentesting and compliance automation may become table stakes. The deciding factor will not be whether companies use AI, but how responsibly, strategically, and securely they use it — especially in regulated sectors where compliance and trust are non-negotiable. Further Reading & References ForeScout Vedere Labs H1 2025 Threat Review


Gartner – The Future of AI in Cybersecurity


CAI – Cognitive Autonomous Intelligence


BlackMamba AI Keylogger


WormGPT Underground Tool


GitHub Copilot


CodiumAI


Snyk Code AI


Microsoft Security Copilot


XBOW Autonomous Pentester


Palo Alto GlobalProtect VPN Vulnerability


AutoSploit


AI in Bug Bounties – PortSwigger


MITRE ATLAS


OWASP Top 10 for LLM Apps


ISO 27001 Standard


HIPAA Security Rule


FDA Medical Device Regulation


FDA SPDF Guidance


Vendict


Scrut


Thoropass


IBM Security AI


NVIDIA AI for Security


Accenture Security


DARPA AIxCC


North Korean APT Attacks – Mandiant


WSJ – Deepfake CEO Fraud Case


FT – Deepfake Audio Scams


GDPR Text


Samsung ChatGPT Data Leak – The Register


Microsoft – AI Red Teaming


NIST AI Risk Management Framework


ENISA AI Security Guidelines ForeScout Vedere Labs H1 2025 Threat Review ForeScout Vedere Labs H1 2025 Threat Review ForeScout Vedere Labs H1 2025 Threat Review Gartner – The Future of AI in Cybersecurity Gartner – The Future of AI in Cybersecurity Gartner – The Future of AI in Cybersecurity CAI – Cognitive Autonomous Intelligence CAI – Cognitive Autonomous Intelligence CAI – Cognitive Autonomous Intelligence BlackMamba AI Keylogger BlackMamba AI Keylogger BlackMamba AI Keylogger WormGPT Underground Tool WormGPT Underground Tool WormGPT Underground Tool GitHub Copilot GitHub Copilot GitHub Copilot CodiumAI CodiumAI CodiumAI Snyk Code AI Snyk Code AI Snyk Code AI Microsoft Security Copilot Microsoft Security Copilot Microsoft Security Copilot XBOW Autonomous Pentester XBOW Autonomous Pentester XBOW Autonomous Pentester Palo Alto GlobalProtect VPN Vulnerability Palo Alto GlobalProtect VPN Vulnerability Palo Alto GlobalProtect VPN Vulnerability AutoSploit AutoSploit AutoSploit AI in Bug Bounties – PortSwigger AI in Bug Bounties – PortSwigger AI in Bug Bounties – PortSwigger MITRE ATLAS MITRE ATLAS MITRE ATLAS OWASP Top 10 for LLM Apps OWASP Top 10 for LLM Apps OWASP Top 10 for LLM Apps ISO 27001 Standard ISO 27001 Standard ISO 27001 Standard HIPAA Security Rule HIPAA Security Rule HIPAA Security Rule FDA Medical Device Regulation FDA Medical Device Regulation FDA Medical Device Regulation FDA SPDF Guidance FDA SPDF Guidance FDA SPDF Guidance Vendict Vendict Vendict Scrut Scrut Scrut Thoropass Thoropass Thoropass IBM Security AI IBM Security AI IBM Security AI NVIDIA AI for Security NVIDIA AI for Security NVIDIA AI for Security Accenture Security Accenture Security Accenture Security DARPA AIxCC DARPA AIxCC DARPA AIxCC North Korean APT Attacks – Mandiant North Korean APT Attacks – Mandiant North Korean APT Attacks – Mandiant WSJ – Deepfake CEO Fraud Case WSJ – Deepfake CEO Fraud Case WSJ – Deepfake CEO Fraud Case FT – Deepfake Audio Scams FT – Deepfake Audio Scams FT – Deepfake Audio Scams GDPR Text GDPR Text GDPR Text Samsung ChatGPT Data Leak – The Register Samsung ChatGPT Data Leak – The Register Samsung ChatGPT Data Leak – The Register Microsoft – AI Red Teaming Microsoft – AI Red Teaming Microsoft – AI Red Teaming NIST AI Risk Management Framework NIST AI Risk Management Framework NIST AI Risk Management Framework ENISA AI Security Guidelines ENISA AI Security Guidelines ENISA AI Security Guidelines

Aptos

Keep

Mandiant

NVIDIA

Snyk

Microsoft

GitHub

What Happens When Hackers Get Your DNA Data? Ask 23andMe

Visit Sekurno.com

Too Long; Didn't Read

How Generative AI Can Be Used in Cybersecurity

How Generative AI Can Be Used in Cybersecurity

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Channel Your Inner Hacker By Breaking Into a System With Nothing But a Name

Enable Additional Logs in Azure Pipeline Execution With This Guide

Why Your Global All Green Dashboards Lie: A Guide to Multidimensional Anomaly Detection

Business perspective in the Future of AI development by 2025

Ethical Implications of AI in Cybersecurity

Getting to Know the XDR Solution and Its Uses

Channel Your Inner Hacker By Breaking Into a System With Nothing But a Name

Enable Additional Logs in Azure Pipeline Execution With This Guide

Why Your Global All Green Dashboards Lie: A Guide to Multidimensional Anomaly Detection

Business perspective in the Future of AI development by 2025

Ethical Implications of AI in Cybersecurity

Getting to Know the XDR Solution and Its Uses

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps