DEV Community

Cover image for How to Test MCP Servers Before They Break Your CI
Anton Gulin
Anton Gulin

Posted on

How to Test MCP Servers Before They Break Your CI

#ai #cicd #mcp #testing

Most teams install an MCP server and hope it works.

That is how you get 3 AM pages.

An MCP server is a bridge between AI agents and your tools. It can crash, leak data, or silently return garbage. If your AI agent relies on it, your whole pipeline breaks.

MCP means Model Context Protocol (standard tool link).

Do not only test startup. Test behavior and permissions too.

This post is the checklist I run on every MCP server before it touches production.

The three-layer test stack

Layer What it catches Tool
Discovery Missing tools, broken metadata MCP Inspector
Behavior Silent failures, wrong output pytest smoke tests
Security Over-permissions, data leaks Permission audit

Layer 1: Discovery with MCP Inspector

MCP Inspector is the official debugging tool. Start it with:

npx @anthropic-ai/mcp-inspector node dist/server.js
Enter fullscreen mode Exit fullscreen mode

Check three things:

  1. Does the server start without errors?
  2. Does it list the tools it promises?
  3. Does a sample request return the right shape?

Layer 2: Behavior with pytest

Here is a minimal smoke test. It checks that initialization returns valid JSON-RPC:

import subprocess, json

def test_mcp_server_responds():
    proc = subprocess.Popen(
        ["npx", "-y", "@anthropic-ai/mcp-server-filesystem", "/tmp"],
        stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True
    )
    proc.stdin.write(json.dumps({
        "jsonrpc":"2.0","id":1,
        "method":"initialize",
        "params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}
    }) + "\n")
    proc.stdin.flush()
    response = json.loads(proc.stdout.readline())
    assert response["id"] == 1
    proc.terminate()
Enter fullscreen mode Exit fullscreen mode

Layer 3: Security with a permission audit

Check three things:

  • Does it need file system access (disk read/write)? Which paths?
  • Does it make network calls (external requests)? To which hosts?
  • Does it run shell commands (terminal execution)? Under which user?

If the answers are "all files, any host, root user," block it.

Where to find servers worth testing

  1. Official MCP Registry — https://registry.modelcontextprotocol.io
  2. GitHub — Search modelcontextprotocol topics
  3. npm / pip — Search @anthropic-ai/mcp-server-*

Red flags: no commits in 6+ months, no tests, no README, permission requests that are too broad.

Verdict

Testing MCP servers is not optional. An untested server is a bug waiting to become an incident.

The three-layer stack catches common failure modes. MCP Inspector for manual checks. pytest for CI gates. Permission audit for last defense.

Anton Gulin is an AI QA Architect — the first person to claim this title on LinkedIn. He builds AI-powered test automation systems where AI agents and human engineers collaborate on quality. Former Apple SDET, now Lead Software Engineer in Test. anton.qa

Top comments (0)