DevOps

Building Guardrails Around AI: Introducing Vigil

Austin H.•August 28, 2026•13 min read
#vigil#cicd#guardrails#ai-safety#security#testing

The Problem

AI-generated code ships fast but risks sink fast too:

  • [OK] Security: Hardcoded secrets, SQL injection, XSS, CSRF
  • [OK] Type errors: Runtime crashes that type checking would catch
  • [OK] Performance: N+1 queries, memory leaks, unbounded loops
  • [OK] Dead code: Unused imports, unreachable branches, orphaned functions
  • [OK] Inconsistent style: Lint failures block merge, CI fails
  • [OK] Missing tests: Code coverage drops, regression risk

Manual review catches some, but shipping fast means review is often rushed. Vigil automates the catch.

What Is Vigil?

Vigil is a GitHub Actions workflow that runs automated guardrails on every PR:

AI generates code |    push to GitHub |    Vigil runs:
  |-- Security scan
  |-- TypeScript typecheck
  |-- ESLint linting
  |-- Unit tests
  |-- Integration tests (optional)
  ||  Code coverage check
     ||o
   All pass? |    Auto-approve for merge
   Any fail? |    Block merge, flag issues

Architecture

# .github/workflows/vigil.yml
name: Vigil - AI Code Guardrails
on: [pull_request]

jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run scan:security
      - run: npm run scan:secrets

  types:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run typecheck

  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run test -- --coverage
      - run: npm run coverage:check  # Fail if < 80%

  # All must pass for PR to merge
status-check:
  runs-on: ubuntu-latest
  needs: [security, types, lint, test]
  if: always()
  steps:
    - run: |
        if [[ "${{ needs.security.result }}" != "success" ]] || \
           [[ "${{ needs.types.result }}" != "success" ]] || \
           [[ "${{ needs.lint.result }}" != "success" ]] || \
           [[ "${{ needs.test.result }}" != "success" ]]; then
          exit 1
        fi

Guardrail 1: Security Scanning

What It Detects

| Pattern | Risk | Severity | Example | | --- | --- | --- | --- | | Hardcoded secrets | Critical | Critical | const API_KEY = 'sk-1234567890' | | SQL injection | Critical | Critical | db.query('SELECT * FROM users WHERE id = ' + userId) | | XSS vulnerability | Critical | Critical | innerHTML = userInput | | Insecure random | High | High | Math.random() for tokens | | Eval / exec | High | High | eval(userCode) | | Insecure crypto | High | High | md5() instead of argon2 |

Implementation

// [OK]  WRONG: Hardcoded secret
const API_KEY = 'sk-proj-1234567890';
const password = 'MyPassword123';
const dbUrl = 'postgres://admin:secret@localhost:5432/db';

// |-- a RIGHT: Use environment variables
const API_KEY = process.env.OPENAI_API_KEY;
if (!API_KEY) throw new Error('OPENAI_API_KEY not set');

const password = process.env.DATABASE_PASSWORD;
if (!password) throw new Error('DATABASE_PASSWORD not set');
// [OK]  WRONG: SQL injection
const userId = req.query.id;
db.query(`SELECT * FROM users WHERE id = ${userId}`);

// |-- a RIGHT: Parameterized query
const userId = req.query.id;
db.query('SELECT * FROM users WHERE id = $1', [userId]);
// [OK]  WRONG: XSS
dom.innerHTML = userInput; // Attacker injects <script>alert(1)</script>

// |-- a RIGHT: Escape or use textContent
dom.textContent = userInput;
// or
dom.innerHTML = DOMPurify.sanitize(userInput);

Configure Security Scanner

// .vigil/security.json
{
  "scanners": {
    "secrets": {
      "enabled": true,
      "patterns": [
        "sk-[A-Za-z0-9]{20,}",     // OpenAI key format
        "AKIA[0-9A-Z]{16}",         // AWS key format
        "-----BEGIN PRIVATE KEY-----" // Private key
      ]
    },
    "sql-injection": {
      "enabled": true,
      "patterns": ["query(`.*\${.*}`)", "query('.*\\' \\+ .*')"] // Template strings in queries
    },
    "xss": {
      "enabled": true,
      "dangerous-methods": ["innerHTML", "eval", "Function"]
    }
  }
}

Guardrail 2: TypeScript Type Checking

Why It Matters

Types catch 10-15% of bugs before runtime. AI often generates vague types or skips them entirely.

// [OK]  WRONG: Vague types
function processUser(user) {
  return user.name.toUpperCase(); // Crashes if user is null
}

// |-- a RIGHT: Strict types
interface User {
  name: string;
  email: string;
}

function processUser(user: User | null): string {
  if (!user) throw new Error('User not provided');
  return user.name.toUpperCase();
}

TypeScript Config

// tsconfig.json
{
  "compilerOptions": {
    "strict": true,                    // Enable all strict checks
    "noImplicitAny": true,            // No implicit any
    "strictNullChecks": true,         // No null surprises
    "strictFunctionTypes": true,      // Function type checking
    "noImplicitThis": true,           // this context must be explicit
    "alwaysStrict": true,             // Use "use strict"
    "noUnusedLocals": true,           // Catch unused variables
    "noUnusedParameters": true,       // Catch unused params
    "noImplicitReturns": true,        // All code paths return
    "noFallthroughCasesInSwitch": true
  }
}

Script

// package.json
{
  "scripts": {
    "typecheck": "tsc --noEmit"
  }
}

Guardrail 3: Linting

ESLint Config

// eslint.config.js
import js from '@eslint/js';
import tsPlugin from '@typescript-eslint/eslint-plugin';
import tsParser from '@typescript-eslint/parser';

export default [
  {
    ignores: ['dist/**', 'node_modules/**'],
  },
  {
    files: ['src/**/*.{js,ts,jsx,tsx}'],
    languageOptions: {
      parser: tsParser,
      ecmaVersion: 2020,
      sourceType: 'module',
    },
    rules: {
      ...js.configs.recommended.rules,
      ...tsPlugin.configs.recommended.rules,
      'no-console': ['warn', { allow: ['warn', 'error'] }],
      'no-unused-vars': 'off', // TS handles this
      '@typescript-eslint/no-unused-vars': ['error'],
      '@typescript-eslint/explicit-function-return-types': 'error',
      '@typescript-eslint/no-explicit-any': 'error',
      'eqeqeq': ['error', 'always'], // Use === not ==
    },
  },
];

Fix Automatically

npm run lint -- --fix
# Fixes 80% of issues automatically (spacing, semicolons, quotes)

Guardrail 4: Testing

Unit Test Example

// src/utils/auth.test.ts
import { hashPassword, verifyPassword } from './auth';

describe('Auth Utils', () => {
  it('hashes passwords consistently', async () => {
    const password = 'MySecurePassword123';
    const hash1 = await hashPassword(password);
    const hash2 = await hashPassword(password);
    
    // Different hashes (salted)
    expect(hash1).not.toBe(hash2);
    
    // But both verify
    expect(await verifyPassword(password, hash1)).toBe(true);
    expect(await verifyPassword(password, hash2)).toBe(true);
  });

  it('rejects wrong password', async () => {
    const hash = await hashPassword('correct');
    expect(await verifyPassword('wrong', hash)).toBe(false);
  });

  it('handles edge cases', async () => {
    // Empty password
    await expect(hashPassword('')).rejects.toThrow();
    
    // Very long password
    const longPassword = 'a'.repeat(1000);
    expect(await hashPassword(longPassword)).toBeDefined();
  });
});

Coverage Check

# Run tests with coverage
npm run test -- --coverage

# Output
# ======= Coverage summary =======
# Statements   : 92.5% ( 148/160 )
# Branches     : 88.2% ( 75/85 )
# Functions    : 95.0% ( 38/40 )
# Lines        : 93.1% ( 150/161 )
# Fail if coverage drops below 80%
npm run coverage:check
# Error: Coverage below 80% threshold

Guardrail 5: Integration Tests

E2E Test Example

// tests/e2e/auth.test.ts
import { test, expect } from '@playwright/test';

test('User signup and login flow', async ({ page }) => {
  // Signup
  await page.goto('/signup');
  await page.fill('input[name=email]', 'test@example.com');
  await page.fill('input[name=password]', 'Password123!');
  await page.click('button[type=submit]');
  
  // Wait for redirect
  await expect(page).toHaveURL('/dashboard');
  
  // Logout
  await page.click('button:has-text("Logout")');
  
  // Verify redirected to login
  await expect(page).toHaveURL('/login');
  
  // Try login with wrong password
  await page.fill('input[name=email]', 'test@example.com');
  await page.fill('input[name=password]', 'WrongPassword');
  await page.click('button[type=submit]');
  
  // Error shown
  await expect(page.locator('.error-message')).toContainText('Invalid credentials');
});

Configuration

// .vigil/config.json
{
  "guardrails": {
    "security": {
      "enabled": true,
      "scanners": ["secrets", "sql-injection", "xss", "eval"],
      "failOnWarning": true
    },
    "types": {
      "enabled": true,
      "strict": true,
      "failOnError": true
    },
    "lint": {
      "enabled": true,
      "autoFix": true,
      "failOnError": true
    },
    "test": {
      "enabled": true,
      "unit": true,
      "integration": true,
      "minCoverage": 80,
      "failBelowThreshold": true
    }
  },
  "reporting": {
    "format": "json",
    "slack": {
      "enabled": true,
      "channel": "#ci-failures",
      "onFail": true
    }
  }
}

Using Vigil with AI Prompts

When asking Claude or Copilot to generate code, include guardrails:

## Prompt for AI

Write a function to authenticate users that passes all Vigil checks:

1. **Security:** No hardcoded secrets, use parameterized queries, escape all output
2. **Types:** Use strict TypeScript, no implicit any
3. **Lint:** Follow ESLint rules (===, no console, explicit returns)
4. **Tests:** Include unit tests with >80% coverage
5. **Performance:** Avoid N+1 queries, hash passwords properly

Here's the guardrails config:

AI then knows exactly what checks it must pass.

Real-World Example: auto-apply-plugin

The auto-apply-plugin repository uses Vigil to guard against:

  • XSS in DOM manipulation (content script runs on untrusted pages)
  • Scope pollution (extension script shouldn't leak to page globals)
  • Memory leaks in event listeners
  • API leaks of secrets (API key never logged or sent to third parties)
  • Regex ReDoS (catastrophic backtracking)
# .github/workflows/vigil.yml (auto-apply-plugin)
on: [pull_request]
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run scan:security
      - run: npm run scan:secrets
  types:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run typecheck
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run test -- --coverage --minCoverage=85

Every PR must pass all checks. No exceptions. This enforces quality at the source.

Benefits

  • Ship AI-generated code with confidence
  • Catch security bugs before code review
  • Educate AI on your standards (it learns from guardrails)
  • Reduce security debt
  • Maintain code quality at scale
  • Sleep better at night

Common Issues

CI Times Out (30+ minutes)

Your test suite is too slow. Parallelize:

jobs:
  security:
    runs-on: ubuntu-latest  # Can run in parallel with others
  types:
    runs-on: ubuntu-latest  # Independent job
  lint:
    runs-on: ubuntu-latest  # Independent job
  test:
    runs-on: ubuntu-latest  # Independent job (longest, run first)

False Positives (Linting Fails on Valid Code)

Adjust ESLint rule strictness:

// Too strict
'@typescript-eslint/no-explicit-any': 'error',

// More forgiving
'@typescript-eslint/no-explicit-any': ['warn', { fixToUnknown: true }],

Tests Pass Locally, Fail in CI

CI environment differs (different Node version, no git history, etc.):

# Use same Node version locally and in CI
node --version  # Should match CI

# Use same npm version
npm --version

# Clean install
rm -rf node_modules package-lock.json
npm ci

Deployment Checklist

  • [ ] .vigil/config.json committed
  • [ ] .github/workflows/vigil.yml in repo
  • [ ] Branch protection enabled (require Vigil checks pass)
  • [ ] Security scanning rules defined
  • [ ] TypeScript strict mode on
  • [ ] ESLint config applied
  • [ ] Unit tests for critical paths
  • [ ] Coverage threshold set (|eN80%)
  • [ ] CI passing on current main
  • [ ] Slack/email notifications configured

Takeaway

Vigil automates what humans forget: checking for security bugs, enforcing types, catching regressions. It transforms AI from a fast but risky tool into a fast and reliable tool. Every guardrail you add is one fewer thing humans need to remember during code review |-- and one more thing you ship with confidence.

Key Takeaways

  • Automated guardrails catch AI-generated code issues before merge
  • Security scanning (Semgrep/CodeQL) finds secrets, injection, XSS
  • TypeScript strict mode catches runtime crashes at compile time
  • Code coverage gates prevent untested code from merging
  • Auto-approve on green CI keeps velocity high for safe changes

Code References

Vigil GitHub Action

name: Vigil
on: [pull_request]
jobs:
  guard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint
      - run: npm test

Thanks for reading!

Read More Articles