MermaidTestingCI/CD

Testing Mermaid diagrams in CI/CD: catch breaking changes before merge

June 27, 20266 min readThe MermaidCreator team

Diagrams are code. They have syntax, they can break, and they should be tested alongside your source. A diagram with a typo in the syntax doesn't render at all — a deprecated arrow notation breaks on newer versions — a failing import in an included diagram doesn't get caught until a user hits it in production. Testing diagrams in CI/CD prevents these surprises.

Why diagram testing matters

Code changes often touch diagrams: you add a service, an architecture diagram is now out of date. You rename a database, the ER diagram has the old name. Most teams catch these through code review ("someone should update the diagram"), which is slow and unreliable. Automated testing catches them instantly.

Three layers of diagram testing:

Syntax validation — does the diagram parse at all?
Render testing — can it be rendered to PNG/SVG without errors?
Content validation — do labels, node counts, or links match expected values?

Layer 1: Syntax validation with mermaid-cli

The simplest test is "can Mermaid parse this?" Use mermaid-cli, the command-line renderer:

npm install --save-dev @mermaid-js/mermaid-cli

Add a script to validate every .md or .mmd file:

# scripts/validate-diagrams.sh
#!/bin/bash

find content docs -name "*.md" -o -name "*.mmd" | while read file; do
  if grep -q '```mermaid' "$file"; then
    echo "Validating $file..."
    # Extract mermaid blocks and test them
    grep -A 100 '```mermaid' "$file" | grep -B 100 '```' | mmdc -o /dev/null 2>&1
    if [ $? -ne 0 ]; then
      echo "❌ Syntax error in $file"
      exit 1
    fi
  fi
done

echo "✅ All diagrams validated"

Run this in your GitHub Actions workflow:

# .github/workflows/ci.yml
name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - run: npm ci
      - run: npm run lint
      - run: bash scripts/validate-diagrams.sh
      - run: npm test
      - run: npm run build

Now every PR gets a syntax check. Typos in diagram blocks fail the build.

Layer 2: Render testing with screenshots

Syntax is necessary but not sufficient. A diagram can parse but fail to render due to layout issues, missing fonts, or incompatibilities with a newer Mermaid version. Render testing catches these.

Use mermaid-cli to export to PNG, then store snapshots in Git (like a visual regression test):

# scripts/snapshot-diagrams.sh
#!/bin/bash

SNAPSHOT_DIR="tests/diagram-snapshots"
mkdir -p "$SNAPSHOT_DIR"

find content docs -name "*.md" | while read file; do
  if grep -q '```mermaid' "$file"; then
    # Extract mermaid block
    sed -n '/```mermaid/,/```/p' "$file" > temp.mmd
    
    # Render to PNG
    output=$(echo "$file" | sed 's/\//_/g; s/.md/.png/')
    mmdc -i temp.mmd -o "$SNAPSHOT_DIR/$output" -w 1024 -H 768
    
    if [ $? -ne 0 ]; then
      echo "❌ Failed to render $file"
      rm temp.mmd
      exit 1
    fi
  fi
done

rm temp.mmd
echo "✅ Snapshots created in $SNAPSHOT_DIR"

Add to CI and commit snapshots alongside diagram changes:

# .github/workflows/ci.yml
- run: bash scripts/snapshot-diagrams.sh
- name: Check for snapshot changes
  run: git diff --exit-code tests/diagram-snapshots/ || exit 0

This way, visual regressions show up in the PR diff — reviewers see exactly what changed.

Layer 3: Content validation with custom checks

For critical diagrams (architecture, data model), add semantic checks: "the database node must exist," "all services must connect," "labels must not exceed 50 characters."

// scripts/validate-diagram-content.js
const fs = require('fs');
const mermaid = require('mermaid');

const rules = {
  'architecture.md': {
    requiredNodes: ['API', 'Database', 'Cache'],
    maxNodes: 20,
  },
  'data-model.md': {
    requiredNodes: ['users', 'orders', 'products'],
    maxLabelLength: 50,
  },
};

async function validateDiagramContent(filePath, rule) {
  const content = fs.readFileSync(filePath, 'utf8');
  const diagram = content.match(/```mermaid\n([\s\S]*?)\n```/)[1];

  // Parse the diagram
  let db;
  try {
    db = await mermaid.mermaidAPI.parse(diagram);
  } catch (e) {
    console.error(`❌ Parse error in ${filePath}: ${e.message}`);
    return false;
  }

  // Check required nodes
  if (rule.requiredNodes) {
    for (const node of rule.requiredNodes) {
      if (!db.nodes.some(n => n.label.includes(node))) {
        console.error(`❌ Missing required node '${node}' in ${filePath}`);
        return false;
      }
    }
  }

  // Check max nodes
  if (rule.maxNodes && db.nodes.length > rule.maxNodes) {
    console.error(
      `❌ Too many nodes in ${filePath}: ${db.nodes.length} > ${rule.maxNodes}`
    );
    return false;
  }

  // Check label length
  if (rule.maxLabelLength) {
    for (const node of db.nodes) {
      if (node.label?.length > rule.maxLabelLength) {
        console.error(`❌ Label too long in ${filePath}: "${node.label}"`);
        return false;
      }
    }
  }

  return true;
}

async function run() {
  for (const [file, rule] of Object.entries(rules)) {
    const valid = await validateDiagramContent(file, rule);
    if (!valid) process.exit(1);
  }
  console.log('✅ All diagram content validated');
}

run().catch(e => {
  console.error(e);
  process.exit(1);
});

Run this in CI:

- run: node scripts/validate-diagram-content.js

Integration with GitHub Actions

Put it all together in a single workflow:

# .github/workflows/test-diagrams.yml
name: Test Diagrams

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - run: npm install -g @mermaid-js/mermaid-cli
      
      - name: Validate syntax
        run: bash scripts/validate-diagrams.sh
      
      - name: Render and snapshot
        run: bash scripts/snapshot-diagrams.sh
      
      - name: Validate content
        run: node scripts/validate-diagram-content.js
      
      - name: Upload snapshots
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: diagram-snapshots
          path: tests/diagram-snapshots/

On failure, the PR gets a link to download the snapshots, so the reviewer can see what broke.

Best practices

1. Keep diagrams close to code

Store architecture diagrams in docs/ alongside the services they describe. Store ER diagrams with your migrations. When code moves, diagrams move with it.

2. Version your diagram syntax

Mermaid evolves. A diagram written for v8 might render differently in v10. Pin @mermaid-js/mermaid-cli in package.json:

{
  "devDependencies": {
    "@mermaid-js/mermaid-cli": "10.6.1"
  }
}

Document breaking changes in your PR title or commit message.

3. Use diagram-specific PR labels

Tag diagram-heavy PRs with a diagrams or docs label so reviewers know to check the snapshots:

- uses: actions/github-script@v6
  with:
    script: |
      if (context.payload.pull_request.files.some(f => f.filename.includes('content/diagrams'))) {
        github.rest.issues.addLabels({
          issue_number: context.issue.number,
          owner: context.repo.owner,
          repo: context.repo.repo,
          labels: ['diagrams']
        });
      }

4. Fail loudly on diagram errors

Don't let a bad diagram slide. Make the build fail if:

A diagram has syntax errors
A render fails
Required nodes are missing
A label is too long

This builds a culture where "diagrams are kept in sync with code."

Example: Testing a breaking change

You upgrade Mermaid from v9 to v10. The loop syntax in sequence diagrams changed slightly. Without diagram testing, users discover this in production. With testing:

You run npm update @mermaid-js/mermaid-cli
CI runs render tests
Snapshots show visual differences in two diagrams
You spot the syntax change and fix the diagrams before merge
PR goes green

FAQ

Q: Isn't this overkill for simple diagrams? A: Depends on the stakes. A diagram in a blog post? Syntax validation is enough. A data model that engineers reference daily? Full testing is worth it.

Q: How do I avoid snapshot bloat? A: Store snapshots as Git LFS (git-lfs) or as separate artifacts, not in the main repo. Or, store only hashes of PNG outputs and compare them, not the full images.

Q: Can I test diagram rendering against different themes? A: Yes. Run render tests twice — once with the default theme, once with a dark theme — to catch theme-specific rendering bugs.

Q: What if a Mermaid version update changes diagram appearance but not validity? A: That's a visual regression — it's expected and worth reviewing. Update snapshots and commit them alongside the Mermaid version bump.

Automate diagram testing once and reap the benefits for every future PR. Diagrams stay accurate, and breaking changes are caught before production.

Mermaid