LLM Pentesting Checklist for Crypto and Fintech Apps
Quick answer. Treat the model as an untrusted user. Map every tool and data source it reaches, then test prompt injection, data leakage, tool abuse, output handling and cost against the OWASP Top 10 for LLM Applications 2025.
Video transcript
An AI assistant in your exchange or wallet reads anyone's text, then calls your APIs. Treat it as an untrusted user.
OWASP says it is unclear whether prompt injection can be fully prevented. So test what happens when the model is tricked.
First, map every entry point, data source and tool the model can call. Flag value-moving tools, and sandbox anything that signs.
Apply Simon Willison's lethal trifecta test. An agent combining private data, untrusted content and external communication is your first test case.
Test direct injection, indirect injection through tickets or token names, and memory attacks that fire later. Injected text may change what the model says, never what it is allowed to do.
Where tools move money, authorize every call in the backend, never on the model's judgment. Keep caps out of the prompt and require two factor or a passkey outside the chat.
Then cover leakage, output handling and cost, and map each finding to the OWASP LLM Top 10.
Make findings reproducible. Record the exact payload, rerun it several times, rate severity by what the tool reached, and retest after every change.
An assistant's tools call into web apps, APIs and infrastructure. Guardian pentests all three, with offchain services led by HackTricks co-founder Carlos Polop.
Read the full checklist, or talk to Guardian about pentesting the systems behind your chat.
An AI assistant in an exchange or wallet reads text from users, documents and the web. Then it calls your APIs.
That makes the model a new user of your systems, and anyone who can write text can try to steer it.
OWASP says "it is unclear if there are fool-proof methods of prevention for prompt injection." So the test is not whether the model can be tricked. It is what happens when it is.
This checklist covers chat assistants and agents that act. For MCP servers, use our MCP pentesting checklist.
Scope the Feature
Start with a map. Every test later in this list comes from it.
- Entry points: chat widget, support inbox, trading assistant, wallet copilot, internal ops bot.
- Data sources: account data, retrieved documents, tickets, emails, web pages and onchain data.
- Tools: every function the model can call, the API behind it and the identity it uses.
- Value-moving tools: withdrawals, transfers, trades, address book changes, limits and API keys.
- Supply chain: the model provider, plugins, retrieval stack and any third-party models.
- Test setup: accounts at every role, and a testnet or sandbox for anything that signs.
Then apply Simon Willison's "lethal trifecta" test. Does one agent combine private data, untrusted content and a way to communicate externally?
If it does, an injected instruction can read your data and send it out. That path is your first test case.
Prompt Injection
OWASP's LLM01 names two kinds. Direct injection is a user's own prompt changing the model's behavior.
Indirect injection arrives in content the model reads from external sources, "such as websites or files."
Direct
- Ask the assistant to ignore its rules, reveal its instructions or act as an admin.
- Try role play, encodings, other languages and attacks split across several turns.
Indirect
- Plant instructions in content the model reads: tickets, PDFs, emails and web pages.
- Use crypto carriers: token names, NFT metadata, transaction memos, ENS records and governance proposals.
Memory
- Plant an instruction in one session, then check whether it fires in a later one.
- Test across users and channels if the agent shares memory between them.
A 2025 study by researchers at Princeton and the Sentient Foundation showed why memory matters. A prompt injected through Discord altered an ElizaOS agent's stored context.
Later, when a user on X asked for a transfer, the agent sent the funds to the attacker's wallet on a testnet.
The authors found that prompt-injection defenses give only "limited protection when stored context is corrupted."
The pass condition: injected text may change what the model says, never what it is allowed to do.
Data Leakage
OWASP items LLM02, LLM07 and LLM08 cover what the model can reveal.
- Other users' data: ask for another account's balance, orders, KYC status or tickets.
- The system prompt: extract it, then look for API keys, internal URLs and role rules.
- Retrieval: query for documents your role should not see, such as runbooks or other tenants' files.
- Exfiltration channels: try markdown images and links that carry chat data in the URL.
- Logs and vendors: trace where transcripts go, and whether they hold secrets or personal data.
OWASP is blunt about prompts: the system prompt "should not be considered a secret, nor should it be used as a security control."
For retrieval, LLM08 recommends permission-aware vector stores and strict partitioning of data between groups of users.
Tools, Authorization and Agency
This is where an LLM feature can move money. OWASP's LLM06, Excessive Agency, has three root causes: too much functionality, permission or autonomy.
Authorization
- Call every tool's API directly, without the model, as a low-privilege user.
- Confirm tools run as the requesting user, not a shared service account.
- Ask the model to act on another user's account ID or address.
OWASP calls the fix complete mediation: authorize in the downstream system, never on the model's judgment.
The same object-level tests on the web layer are in our guide to web application pentesting for crypto and fintech.
Wallets and transactions
- Use injected text to change a destination address, amount, asset or chain.
- Confirm withdrawals, address book changes and new API keys need 2FA or a passkey outside the chat.
- Check that caps and allowlists live in the backend, not in the prompt.
- Compare the confirmation screen with what is actually signed or executed.
- Find where an agent's signing key lives, and what one stolen session can spend.
Output Handling and Cost
Output handling (LLM05)
OWASP's advice is zero trust: treat model output as untrusted input before it reaches anything else.
- Test XSS through markdown, HTML and links the model returns.
- Test injection where output builds SQL, shell commands, file paths or API calls.
- Validate tool calls against a strict schema in code before running them.
- Check that deposit addresses and fees come from the backend, never from generated text.
Rate limits and cost (LLM10)
OWASP names "Denial of Wallet": attackers drive up usage to exploit pay-per-use pricing.
- Check per-user rate limits and token quotas, including for logged-out users.
- Send oversized inputs and prompts that loop tool calls.
- Confirm spend alerts and a kill switch exist.
Reporting Against OWASP
Map each finding to the OWASP list, so engineers and auditors read it the same way.
| Test area | OWASP 2025 item | Example finding |
|---|---|---|
| Prompt injection | LLM01 Prompt Injection | A support ticket makes the bot change a user's email |
| Other users' data | LLM02 Sensitive Information Disclosure | The bot returns another user's order history |
| Output | LLM05 Improper Output Handling | A markdown image sends chat data to an attacker |
| Tools | LLM06 Excessive Agency | A withdrawal tool runs without 2FA |
| System prompt | LLM07 System Prompt Leakage | An API key sits in the prompt |
| Retrieval | LLM08 Vector and Embedding Weaknesses | A retail user retrieves an internal runbook |
| Cost | LLM10 Unbounded Consumption | No per-user token quota |
Then make each finding reproducible:
- Record the exact payload, the model version and the tool calls it produced.
- Rerun each payload several times and report how often it worked.
- Rate severity by what the tool reached: funds, keys or personal data.
- Retest after every model, prompt or tool change.
Agents can run many of these tests. What still needs a human is covered in AI pentesting explained.
Where Guardian Fits
An assistant's tools call into web apps, APIs and infrastructure. Guardian pentests those systems for fintech platforms.
- Web app pentest: authentication, authorization, value-moving flows and the client-side surface.
- API pentest: key scoping, per-object authorization, rate limits and endpoints that fire bots or onchain transactions.
- Infrastructure pentest: cloud IAM, secrets handling and the pipelines that ship your code.
- Browser extension pentest: key storage, internal messaging and signing flows.
Guardian's offchain services are led by Carlos Polop, who co-founded HackTricks.
For continuous coverage, Managed Security runs Helix, Guardian's AI security agent pipeline, across backend services, infrastructure and web apps.
Only human-verified findings reach you, each with a fix PR attached.
Frequently Asked Questions
How do you pentest an LLM feature in a fintech or crypto app?
Map every entry point, data source and tool the model touches. Then test prompt injection, data leakage, tool authorization, output handling and cost limits, and report each finding against the OWASP LLM Top 10 with a reproducible payload.
What is the OWASP Top 10 for LLM Applications 2025?
It is a list of ten risk categories for applications built on large language models, from the OWASP GenAI Security Project. It runs from LLM01 Prompt Injection to LLM10 Unbounded Consumption, and includes Excessive Agency and System Prompt Leakage.
Can prompt injection be fully prevented?
OWASP says it is unclear whether fool-proof prevention exists. So design for failure: authorize every tool call in the backend, cap amounts, and require 2FA or a human approval outside the chat for anything that moves funds.
How is LLM pentesting different from MCP pentesting?
An MCP pentest targets a server that exposes tools over the Model Context Protocol. An LLM pentest covers the whole feature: prompts, retrieved data, tools, output rendering and cost. A feature built on MCP servers needs both.
Sources
- OWASP GenAI Security Project: Top 10 for LLM Applications 2025
- OWASP: LLM01:2025 Prompt Injection
- OWASP: LLM02:2025 Sensitive Information Disclosure
- OWASP: LLM05:2025 Improper Output Handling
- OWASP: LLM06:2025 Excessive Agency
- OWASP: LLM07:2025 System Prompt Leakage
- OWASP: LLM08:2025 Vector and Embedding Weaknesses
- OWASP: LLM10:2025 Unbounded Consumption
- Simon Willison: The lethal trifecta for AI agents (June 16, 2025)
- arXiv: Patlan et al., Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agents (v3, July 9, 2025)
- Guardian: Web application penetration testing
- Guardian: API penetration testing
- Guardian: Fintechs
Pentest the systems behind the chat
Guardian pentests the web apps, APIs and infrastructure that authenticate your users and move their funds and data.