Artificial General Intelligence is Upon Us, and it’s Time to Prepare

Advanced AI capability creates a practical security problem long before any consensus on timelines, scams become cheaper, faster, and more convincing.

Preparation should focus on durable controls that survive uncertainty, especially account recovery protection, verification habits, and limits on automated trust.

Risk controls to put in place

• Secure your primary email, Apple ID, Google account, and password manager first. These usually control account resets.
• Turn on 2FA or passkeys wherever possible, and remove old recovery methods you do not control.
• Assume impersonation gets easier: verify money requests and sensitive changes through a second channel you control.
• Reduce your public attack surface: review what is public, prune connections, and tighten privacy settings. Use this digital footprint reduction guide to prioritize high-leverage changes.
• Set up monitoring: login alerts, payment alerts, and recovery alerts on your critical accounts.
• If you are already dealing with account takeover, switch from preparation to recovery: Been hacked? Take these steps immediately.

Rule of thumb: Verify identity and intent, not writing quality. AI makes emails, texts, and voice messages look and sound "normal".

What AGI means

Artificial general intelligence (AGI) usually means an AI system that can perform a wide range of tasks at or above human level, not just one narrow job. Whether or not AGI arrives soon, one part is already true: AI capabilities are improving and spreading fast.

Preparation is not about predicting timelines. It is about building a security posture that still works when scams are more realistic, targeting is cheaper, and attackers can scale their efforts.

What will change first

The first changes are not sci-fi robots. They are familiar crimes that become more effective:

• Phishing and social engineering that reads like a real coworker, friend, or support agent.
• Support scams that imitate real companies and real workflows.
• Account takeover chains where one compromised account is used to compromise the next.
• Business email compromise where invoices, bank details, or vendor details are changed at the last minute.

What does not change is the foundation of defense: strong authentication, controlled recovery paths, and verification habits that do not rely on “it looked real”.

How AI changes the threat model

Phishing and impersonation get easier

Attackers can generate messages that sound natural, match tone, and adapt to your replies. That makes the usual advice about spotting bad spelling less reliable.

Start here: The rising threat of AI-powered phishing and social engineering.

Deepfakes raise the baseline of doubt

As synthetic audio and video improves, you will see more attempts to impersonate trusted people. This is especially dangerous when the scam tries to create urgency, secrecy, and authority.

If you want a practical overview of what deepfakes are and why they matter: What are deepfakes and why are they dangerous?

Practical rule for deepfake-era communication: do not treat a voice or video call as authorization for high-risk actions. Treat it as a prompt to verify through a second channel.

• If someone asks for money, credentials, or urgent changes on a call, hang up and call back using a number you already have saved.
• If someone claims to be support, open the official app or website and start support from there. Do not follow links sent in chat.
• If a request is both urgent and secret, treat that as a red flag and slow down.

The goal is not to become a deepfake detective. The goal is to make scams fail even when the content looks perfect.

Targeted attacks shift from rare to cheap

As tools improve, “targeted” is no longer reserved for celebrities. Small businesses, contractors, and individuals become viable targets because tailoring gets cheaper.

Core idea: The defense is less about being famous and more about being difficult to exploit.

Practical preparation

1) Secure the accounts that control everything

If you only do one thing: Protect your primary email. If an attacker controls your inbox, they can often reset the rest.

If an attacker gets your primary email, they can often reset everything else. Prioritize:

1. Primary email account
2. Apple ID or Google account
3. Password manager
4. Financial accounts
5. Social accounts (because they are used to scam your friends)

Two basics matter more than almost anything else

• Stop password reuse and predictable patterns: Common mistakes creating passwords.
• Turn on strong sign-in protections: Two-factor authentication (2FA) and its many names.

Practical rule: if a recovery method is not fully under your control (old email you forgot, old phone number, shared inbox), remove it. Attackers love recovery paths.

2) Add verification for high-risk requests

When scams are more convincing, the winning move is process. Verification should be boring, consistent, and hard to bypass.

• If someone asks for money, gift cards, crypto, or urgent changes, confirm through a second channel you control.
• Do not accept a phone number, chat link, or email address provided inside the suspicious message. Use a known number or official site.
• If the request demands secrecy or urgency, treat that as a signal to slow down.

Make verification easy on yourself by pre-committing to a small set of rules. In practice, that means you decide in advance what you will do when you feel pressured.

• Pause phrase: “I need 10 minutes to verify this.” If the requester pushes back, treat it as a red flag.
• Known-channel rule: verify requests using a known phone number, known email address, or a previously saved contact. Do not use contact details provided in the message.
• Two-person rule for payments: if a request moves money, you always involve a second person.

Deepfake-resistant default: treat voice and video as helpful context, not proof. Proof comes from an independent channel you already trust.

3) Reduce your public attack surface

You do not need to delete your online life to improve safety. Focus on high-leverage changes:

• Make follower lists, friend lists, and old posts less public where possible.
• Do not accept new connections you cannot verify.
• Avoid posting high-resolution photos of IDs, tickets, receipts, and travel documents.
• Be careful with public answers to common security questions (birthplace, pet names, family details).

4) Set up monitoring so you learn fast

Turn on alerts so you learn fast

• New sign-ins and new devices on email and cloud accounts
• Password changes and recovery method changes
• Large purchases, new payees, and wire transfers on financial accounts

These alerts do not prevent attacks, but they shrink the time between compromise and response.

5) Protect your phone number and recovery paths

Your phone number is still used for account recovery in many services. That makes it a common target, not because attackers can hack the phone system directly, but because they can exploit weak recovery settings or social engineer carriers.

• Use an account PIN with your mobile carrier and keep it unique.
• Prefer app-based or hardware-based 2FA over SMS where the platform supports it.
• Audit your recovery email and recovery phone settings on critical accounts, and remove anything you do not control.

6) Keep devices and browsers boring

Many compromises still start with a simple step: installing the wrong app, accepting the wrong browser prompt, or postponing updates. A boring device is a safe device.

• Update operating systems and browsers promptly, especially on the devices you use for email and banking.
• Install apps only from official stores and be skeptical of free VPNs, cleaner apps, and keyboard apps.
• Use a password manager and keep your browser extensions minimal.

7) Teach the people around you

Attackers often start with the easiest person in the network, not the most important one. A short conversation can prevent a lot of damage.

• Explain the new baseline: messages can look perfect, and callers can sound convincing.
• Agree on a verification habit for money requests and urgent account changes.
• Make it easy to ask for help without embarrassment. Shame is a common reason people do not report early.

Practical preparation

Key idea: Assume a convincing message will get through. Your controls should fail closed: no single inbox or person should be able to move money or change access alone.

AI-assisted scams affect businesses first because business workflows move money and data. The best return on effort is training and process.

Train for modern phishing, not 2010 phishing

Training should focus on behaviors (verify, slow down, escalate), not just spotting typos. Start here: Why every business should train employees to spot phishing emails.

Put “call-back” controls on payments and sensitive changes

• No invoice, payout, or bank detail change should happen based on a single email thread.
• Require a call-back to a known number and a second approver for payment changes.
• Document the workflow so it is easy for employees to do the safe thing under pressure.

Minimum technical controls

• Require MFA everywhere, especially email and admin consoles. Prefer phishing-resistant methods where possible.
• Disable legacy sign-in methods that bypass modern controls if your environment supports it.
• Implement SPF, DKIM, and DMARC on your domains to reduce spoofing and improve email authenticity.
• Centralize identity (SSO where practical) and remove unused accounts quickly.
• Log and alert on mailbox rules, forwarding changes, and new OAuth app grants.

Harden your helpdesk and payroll workflows

• Do not reset MFA or change bank details based on inbound email alone.
• Use a call-back to a known number and a second approver for payroll changes.
• Keep an internal directory of verified vendor contact details and verify changes against that directory.

Protect executive and finance workflows

Many high-impact scams target the people who can move money or approve access. Make the safe path the default path.

• Separate duties for payments: one person requests, another approves, and both verify through a known channel.
• Lock down executive email accounts with the strongest MFA available and dedicated admin controls.
• Be cautious about public org charts and public staff directories that make targeting easier.

Run short drills and make reporting easy

• Do a 15-minute tabletop exercise quarterly: fake invoice change, fake CEO call, fake support ticket.
• Define a single reporting channel for suspected phishing and impersonation.
• Reward early reporting. The best incidents are the ones caught early.

Prepare an incident playbook for impersonation

If a scam is in motion, use a standard response

• Preserve evidence (emails, headers, chat transcripts, call recordings if lawful in your region).
• Notify impacted vendors or customers using established channels.
• Reset credentials, rotate API keys, and review mail rules and forwarding settings.

If you think a scam is in progress

Speed matters, but so does precision. The goal is to cut off attacker access, stop payments, and reduce the chance of follow-on compromise.

• Assume the attacker can read anything in the compromised inbox. Move sensitive coordination to another channel.
• Freeze payment changes and notify your bank immediately if any transfer might have happened.
• Reset passwords and sessions for the affected account, then review mailbox rules and connected apps.
• Notify the internal team in a way that does not rely on the compromised channel.

After-action hardening

After the immediate fire is out, do a quick persistence check. Many incidents recur because one control is left behind.

• Re-check recovery methods, forwarding rules, and OAuth grants after 24 to 48 hours.
• Rotate long-lived credentials (API keys, shared mailbox passwords) and remove unused integrations.
• Check whether any new inbox rules or forwarding settings were created in other mailboxes.
• Verify vendor bank details through a known channel, even if the change seems reversed.
• Brief the team on what happened and the verification step that would have stopped it.

For small teams, writing a short policy that defines which actions always require call-back verification is often the highest leverage improvement you can make. Keep it simple and enforce it consistently.

Also consider tracking attempted scams, not just successful ones. If employees report suspicious messages early without fear of blame, you get the signal before money moves. Over time, you can adjust training and controls based on real attempts your organization actually sees.

Finally, assume some attacks will land. Decide in advance who can freeze payments, who contacts the bank, and how you notify customers or vendors. Store key phone numbers outside your email (for example, in a runbook), so coordination still works if an inbox is compromised.

Official guidance

If you want a high-quality overview of business email compromise and defensive controls, CISA provides a practical summary: Business Email Compromise and Defenses.

Featured image by Midjourney and Jonas Borchgrevink.