AI-Powered Security Code Reviews That Actually Work: A Threat-Model-First Methodology

📢 I have some exciting news: I’ve recently started a YouTube channel for “AppSec Untangled”, where I’ll be sharing some of my content in video format. Check out the video version of this story here: https://youtu.be/OC2cTxCGQIM

Security code review is one of the most important activities in an AppSec engineer’s toolkit. But it is also one of the trickiest to do well, because, unlike dynamic testing, where you’re poking at a running application, here you are working directly with the code itself. And the naive approaches (e.g. reading the code line by line, or just throwing it at a static scanner) are going to leave a lot on the table.

So in today’s story, we are going to discuss a methodology for performing security code reviews, and then we are going to see how to use AI to follow that same methodology and get better, more consistent results. And we’re going to do a live demo on a real open-source repository and an actual open pull request.

But before we get there, a general observation about AI that is worth keeping in mind throughout this story:

AI is a great tool when you know what needs to be done but don’t have enough time for it. However, if you don’t have a methodology, asking AI to just “do the thing” is going to be hit or miss at best.

This is the key idea behind everything we are going to discuss today.

A Mental Model for Security Issues

Before starting any security code review, it helps to have a clear mental model of what kind of security issues you are even looking for. One mental model I find very useful is categorizing security issues into two main buckets.

Category 1: Business Logic Vulnerabilities (Things That Should Be There)

This category is about security controls that should always be present in your code. If they are missing, that is a vulnerability. For a typical web application backend, these include:

Authentication: Every authenticated route should verify that the request is coming from a valid authenticated user, whether that’s through a JWT token, a session cookie, or whatever mechanism your application uses.

Authorization: Once you know who the user is, you need to verify what they are allowed to do. This usually has multiple layers:

• Tenant isolation — in a multi-tenant SaaS, one customer should never be able to access another customer’s data.
• Role-based access control — users have roles and permissions, and actions should be gated by these.
• Resource-level authorization — a user’s relationship to a specific resource (e.g., are they a member of this project? do they have edit access?) should be checked before they can act on it.

CSRF Protection — For mutating operations, there should be a mechanism to verify that the request was intentionally initiated by the user, not triggered by a malicious third-party page that is exploiting the user’s active session cookies.

The key thing to note about this category is that static scanners are mostly blind to these vulnerabilities. Because authentication and authorization are deeply tied to the business logic of your specific application, tools simply don’t know enough context to identify when they’re missing. This is why code review is so important — it is often the only reliable way to find broken authentication, broken authorization, and missing CSRF protection.

Category 2: Source-Sink Vulnerabilities (Things That Shouldn’t Be There)

This category follows a specific pattern: user-controlled input (the source) reaches a dangerous function (the sink) without sufficient validation in between. If both conditions are true, you have a vulnerability.

The type of vulnerability depends on the sink:

• SQL query construction → SQL Injection
• HTML output / page generation → XSS
• OS command execution → Command Injection
• Outbound HTTP requests with user-supplied URLs → SSRF
• HTTP redirect with user-supplied target → Open Redirect
• NoSQL query construction → NoSQL Injection
• Unsafe deserialization (e.g., Pickle) → Remote Code Execution

Unlike the first category, static scanners can help here because there are well-defined dangerous functions to look for. But they still need to be combined with code review, especially to verify whether user input actually reaches those sinks and whether validation is in place.

The Methodology

With these two categories in mind, the methodology for security code review becomes:

1. Understand the scope and architecture: What is the application doing? What technologies does it use? What is the data model?
2. Build a threat model: Based on what you now know, which threats from the two categories above are actually relevant to this codebase?
3. Translate the threat model into security guidelines — For each relevant threat, what does secure code look like in this specific application? This becomes your security wiki.
4. Create a review plan — For the code change you are reviewing (e.g., a PR), which of the relevant threats apply?
5. Review the code — For each relevant threat, verify whether the mitigation is implemented or not. This is what you would do in a manual code review. And this is exactly what we are going to teach AI to do.

Demo: AI-Powered Security Code Review on a Real Codebase

For this demo, I selected Taiga, an open-source project management application (think Jira, but open source). It is a good example because it is a realistic multi-tenant web application with a Django/Python backend, a PostgreSQL database, asynchronous task processing via Celery and RabbitMQ, caching, and a REST API. Plenty of interesting security angles.

I am going to use Claude Code to follow the methodology step by step.

NOTE: You can watch the full demo in the video on https://youtu.be/OC2cTxCGQIM

Step 1: Understand the Architecture

The first thing I did was simply ask:

“What is this repo for and give me a high-level architecture of the application.”

This immediately tells us things that are security-relevant: there is async task processing (potential serialization/deserialization), a database (potential injection), caching (potential cache poisoning), and it is a multi-tenant app (tenant isolation will be important).

Step 2: Build the Threat Model

Now we use the two-category mental model to generate a threat model for this codebase. I gave Claude Code a prompt that:

• Described the two categories (business logic and source-sink)
• Asked it to identify which threats from each category are relevant based on the architecture it just summarized This gives us a solid starting point. It is not necessarily complete, but it covers the most important threats and saves a significant amount of time.

A Closer Look at One Threat: Insecure Deserialization via Pickle

As an exmple, let’s focus on one finding that stood out: the use of Pickle as the serializer for Celery tasks. Let me explain why this matters.

Pickle is Python’s default serialization library. The problem is that it is designed to execute arbitrary code during deserialization — that is actually how it works by design. If an attacker can supply a specially crafted pickled payload, they get Remote Code Execution.

Here is a quick example of what that looks like:

import pickle, os

class Exploit(object):
    def __reduce__(self):
        return (os.system, ('id',))

# Serialize the malicious object
payload = pickle.dumps(Exploit())

# VULNERABLE: Deserializing this executes: os.system('id')
pickle.loads(payload)

So in this codebase, Pickle is being used to serialize Celery task payloads, which are sent to RabbitMQ and then deserialized by the workers.

Now, is this actually exploitable? I asked Claude Code to investigate:

“Is there user input that could reach the Pickle deserialization? Is input validation in place to prevent it?”

The good news: Django itself constructs the task objects before serializing them, so an attacker cannot directly inject a malicious pickled payload through the normal application flow.

The bad news: if RabbitMQ is directly accessible from the internet, an attacker can bypass the application entirely and push a malicious payload straight into the queue. And checking the configuration reveals that RabbitMQ is using default credentials (guest/guest). That is a finding worth noting.

The key mitigations: RabbitMQ should not be exposed to the public internet, and default credentials must be changed.

Step 3: Generate the Security Wiki

Now we translate the threat model into a security wiki — a living document that defines what secure code looks like in this specific application, with code examples.

The key addition here (and this is important for the next step) is a “when is this relevant?” section for each threat. For example:

• Authentication is relevant whenever a new endpoint is added.
• SQL injection is relevant whenever the PR introduces database queries that use user-supplied input.
• CSRF protection is relevant whenever a mutating operation is added.
• This document is useful in two ways: developers can read it to understand what they need to implement, and AI can use it as the basis for code reviews.

Step 4: Create a Security Review Skill

Rather than writing the skill from scratch, I asked Claude Code to generate it:

“Add a Claude skill to perform AppSec code reviews. The skill should: (1) read the PR, (2) read the security wiki, (3) assess which threats from the wiki are relevant to this PR, (4) for the relevant threats, verify whether the mitigation is implemented.”

The output is a five-step skill:

1. Read the PR
2. Read the security wiki
3. Assess relevance
4. Deep review of relevant threats
5. Summary table.

Step 5: Run the Review on a Real Pull Request

I picked this open PR (selected at random), which adds “embed user stories and tasks in project details.” Let me quickly describe what the PR does: it adds user stories and tasks to the response of a project details endpoint, querying them from the database based on some input parameters.

Now let’s run the skill:

The skill correctly identified that:

• Authentication — not relevant, we’re not changing how authentication works on existing routes
• Authorization — relevant, we’re adding new data to a response
• Tenant isolation — relevant, we’re querying resources from the database
• SQL injection — relevant, we’re adding new database queries
• Everything else — not relevant

Now the deep review:

Finding: Missing Resource-Level Authorization

The PR adds user stories and tasks to the project details response, but only checks whether the user has access to the project — not whether they have permissions on the user stories and tasks themselves. The skill flagged this as a potential authorization bypass and suggested adding user_has_permission(VIEW_USER_STORIES) and user_has_permission(VIEW_TASKS) checks.

Tenant Isolation — The existing queryset filtering in the codebase is sufficient, so this was marked as mitigated.

SQL Injection — The PR is using a slightly less safe ORM pattern even though user input isn’t directly injected into the query string. The skill flagged this as a best practice improvement — not an active vulnerability, but worth addressing.

Key Takeaways

To wrap up, here are the main things to take away from this story:

1. Use the two-category mental model. Business logic vulnerabilities (things that should be there) and source-sink vulnerabilities (things that shouldn’t be there) give you a systematic way to think about what you’re looking for before you write a single line of review.

2. Start with a threat model, not with the code. Understanding which threats are relevant to this specific codebase before reading the code makes your review focused and efficient.

3. The security wiki is the key to good AI reviews. It is the difference between AI making educated guesses and AI following a structured plan that is tailored to your codebase. Treat it as a living document — update it with pentest findings, bug bounty findings, and new CVEs.

4. Never fully rely on AI. Always verify its findings manually, especially when they seem like valid security issues. The goal is to use AI to cover ground faster, not to replace your judgment.

5. Tune the skill over time. Run the skill, compare it to your own manual review, and use the gaps to improve both the wiki and the skill. You can even automate this — a scheduled job that reads new security tickets and suggests additions to your security wiki is not a far-fetched idea.

If you found this useful, feel free to leave a comment or share it with your team. And if you have questions about the methodology or the demo, drop them in the comments below. Thanks for reading, and see you in the next one!