Codity CI Setup Guide: GitHub Actions and Azure Pipelines

Overview

Codity automatically generates comprehensive unit tests for your code changes and can create pull requests containing those tests. To maximize the benefits of automated testing, set up Continuous Integration (CI) so tests run automatically for every pull request.

This guide covers CI setup for both GitHub Actions and Azure Pipelines.

---

How It Works

Test Generation

Codity analyzes your PR, detects changes in supported languages, and generates relevant tests (Go, Python, Ruby, JavaScript/TypeScript).

PR Creation

Codity commits the generated tests to a new branch and opens a pull request.

CI Execution

If CI is configured, tests are automatically executed on the PR.

Auto-Fix

Codity monitors CI results and automatically fixes failing generated tests (up to 3 attempts).

Merge

Once all tests pass, you can review and merge the PR.

---

Supported Languages

Go

Generates _test.go files using Go's testing package and Testify assertions.

Python

Generates test_*.py test files using pytest.

Ruby

Generates _spec.rb files using RSpec.

JavaScript / TypeScript

Generates .test.js/.test.ts files using Jest.

---

GitHub Actions Setup

Required GitHub App Permissions

Before configuring CI, ensure the Codity GitHub App has the correct permissions.

Steps:

Go to Repository Settings → GitHub Apps → Codity

Enable:

• Read access: metadata
• Read & write access: checks, contents, issues, pull requests, workflows

Without these permissions, Codity cannot create test PRs or push auto-fixes.

---

CI Workflow Configuration

The following minimal GitHub Actions workflows will run unit tests for each supported language. They trigger on:

• opened pull requests
• synchronize (new commits pushed to PR)
• reopened pull requests

---

Go Testing Workflow

Create: .github/workflows/go-tests.yml

# Minimal Go Tests CI

# Works for repositories with go.mod in root or subdirectories

name: Go Tests

on:

  pull_request:

    types: [opened, synchronize, reopened]

jobs:

  test:

    runs-on: ubuntu-latest

    steps:

      - name: Checkout code

        uses: actions/checkout@v4

      - name: Set up Go

        uses: actions/setup-go@v5

        with:

          go-version: "1.21"

      - name: Run tests

        run: |

          failed=0

          while IFS= read -r -d '' gomod; do

            dir=$(dirname "$gomod")

            echo "Testing in $dir"

            (cd "$dir" && go mod download && go test -v ./...) || failed=1

          done < <(find . -name go.mod -print0)

          exit $failed

---

Python Testing Workflow

Create: .github/workflows/python-tests.yml

# Minimal Python Tests CI

name: Python Tests

on:

  pull_request:

    types: [opened, synchronize, reopened]

jobs:

  test:

    runs-on: ubuntu-latest

    steps:

      - name: Checkout code

        uses: actions/checkout@v4

      - name: Set up Python

        uses: actions/setup-python@v5

        with:

          python-version: "3.11"

      - name: Install dependencies and run tests

        run: |

          python -m pip install --upgrade pip

          pip install pytest pytest-cov

          # Install from all requirements.txt files found

          while IFS= read -r -d '' reqfile; do

            echo "Installing from $reqfile"

            pip install -r "$reqfile"

          done < <(find . -name requirements.txt -print0)

          pytest -v

---

Ruby Testing Workflow

Create: .github/workflows/ruby-tests.yml

# Minimal Ruby Tests CI

name: Ruby Tests

on:

  pull_request:

    types: [opened, synchronize, reopened]

jobs:

  test:

    runs-on: ubuntu-latest

    steps:

      - name: Checkout code

        uses: actions/checkout@v4

      - name: Set up Ruby

        uses: ruby/setup-ruby@v1

        with:

          ruby-version: "3.2"

      - name: Run tests

        run: |

          failed=0

          while IFS= read -r -d '' gemfile; do

            dir=$(dirname "$gemfile")

            echo "Testing in $dir"

            (cd "$dir" && bundle install && bundle exec rspec) || failed=1

          done < <(find . -name Gemfile -print0)

          exit $failed

---

Multi-Language Projects (GitHub Actions)

If your repository uses multiple languages, you may combine workflows or create separate ones.

Example Combined Workflow:

name: Multi-Language Tests

on:

  pull_request:

    types: [opened, synchronize, reopened]

jobs:

  go-tests:

    runs-on: ubuntu-latest

    if: contains(github.event.pull_request.changed_files, '.go')

    steps:

      # Go test steps here

  python-tests:

    runs-on: ubuntu-latest

    if: contains(github.event.pull_request.changed_files, '.py')

    steps:

      # Python test steps here

  ruby-tests:

    runs-on: ubuntu-latest

    if: contains(github.event.pull_request.changed_files, '.rb')

    steps:

      # Ruby test steps here

---

Troubleshooting

Tests not running?

Ensure workflow files are under: .github/workflows/ Verify PR triggers match your workflow YAML Check your Actions tab for logs and failures

Auto-Fix not working?

Workflow names must include: test, ci, or build Ensure CI is reporting pass/fail status Verify Codity GitHub App permissions allow: Writing to PR branches Reading workflow statuses

---

Last updated: 2025-12-27