Codity CI Setup Guide: GitHub Actions and Azure Pipelines

Overview

Codity automatically generates comprehensive unit tests for your code changes and can create pull requests containing those tests. To maximize the benefits of automated testing, set up Continuous Integration (CI) so tests run automatically for every pull request.

This guide covers CI setup for both GitHub Actions and Azure Pipelines.

How It Works

Test Generation

Codity analyzes your PR, detects changes in supported languages, and generates relevant tests (Go, Python, Ruby, JavaScript/TypeScript).

PR Creation

Codity commits the generated tests to a new branch and opens a pull request.

CI Execution

If CI is configured, tests are automatically executed on the PR.

Auto-Fix

Codity monitors CI results and automatically fixes failing generated tests (up to 3 attempts).

Merge

Once all tests pass, you can review and merge the PR.

Supported Languages

Go — Generates _test.go files using Go's testing package and Testify assertions.

Python — Generates test_*.py test files using pytest.

Ruby — Generates _spec.rb files using RSpec.

JavaScript / TypeScript — Generates .test.js/.test.ts files using Jest.

GitHub Actions Setup

Required GitHub App Permissions

Before configuring CI, ensure the Codity GitHub App has the correct permissions.

Steps:

1. Go to Repository Settings → GitHub Apps → Codity
2. Enable:
   • Read access: metadata
   • Read & write access: checks, contents, issues, pull requests, workflows

Without these permissions, Codity cannot create test PRs or push auto-fixes.

CI Workflow Configuration

The following minimal GitHub Actions workflows will run unit tests for each supported language. They trigger on:

• opened pull requests
• synchronize (new commits pushed to PR)
• reopened pull requests

Go Testing Workflow

Create: .github/workflows/go-tests.yml

# Minimal Go Tests CI
# Works for repositories with go.mod in root or subdirectories

name: Go Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: "1.21"

      - name: Run tests
        run: |
          failed=0
          while IFS= read -r -d '' gomod; do
            dir=$(dirname "$gomod")
            echo "Testing in $dir"
            (cd "$dir" && go mod download && go test -v ./...) || failed=1
          done < <(find . -name go.mod -print0)
          exit $failed

Python Testing Workflow

Create: .github/workflows/python-tests.yml

# Minimal Python Tests CI

name: Python Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies and run tests
        run: |
          python -m pip install --upgrade pip
          pip install pytest pytest-cov
          # Install from all requirements.txt files found
          while IFS= read -r -d '' reqfile; do
            echo "Installing from $reqfile"
            pip install -r "$reqfile"
          done < <(find . -name requirements.txt -print0)
          pytest -v

Ruby Testing Workflow

Create: .github/workflows/ruby-tests.yml

# Minimal Ruby Tests CI

name: Ruby Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: "3.2"

      - name: Run tests
        run: |
          failed=0
          while IFS= read -r -d '' gemfile; do
            dir=$(dirname "$gemfile")
            echo "Testing in $dir"
            (cd "$dir" && bundle install && bundle exec rspec) || failed=1
          done < <(find . -name Gemfile -print0)
          exit $failed

Multi-Language Projects (GitHub Actions)

If your repository uses multiple languages, you may combine workflows or create separate ones.

Example Combined Workflow:

name: Multi-Language Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  go-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.go')
    steps:
      # Go test steps here

  python-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.py')
    steps:
      # Python test steps here

  ruby-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.rb')
    steps:
      # Ruby test steps here

Troubleshooting

Tests not running?

• Ensure workflow files are under: .github/workflows/
• Verify PR triggers match your workflow YAML
• Check your Actions tab for logs and failures

Auto-Fix not working?

• Workflow names must include: test, ci, or build
• Ensure CI is reporting pass/fail status
• Verify Codity GitHub App permissions allow:
  • Writing to PR branches
  • Reading workflow statuses