Bitbucket Pipelines Setup Guide for Codity.ai

This guide covers the important configuration points to ensure Codity.ai properly detects and monitors your Bitbucket Pipelines for automated test generation.

Critical Configuration Points

1. Step Name

Codity identifies test steps by checking the step name for keywords. The step name must contain at least one of:

test, ci, spec, unit, build, jest, pytest, rspec, go test, junit, maven, gradle, mvn

Steps whose names contain any of the following are skipped:

lint, security, leak, format, style, codetour, watch

Examples:

  • Run Python Tests
  • CI - pytest
  • Run Jest Unit Tests
  • Gradle Build and Test
  • Deploy (not detected)
  • Run Linter (skipped)

2. PR Triggers

Include a pull-requests: section to trigger pipelines on pull requests. Without this, CI won't run on PRs and Codity cannot monitor test results.

3. Verbose Output

Use -v or --verbose flags in test commands to provide detailed logs for auto-fix analysis.

4. App Password Scopes

The Bitbucket App Password used for Codity must have the following scopes: repository, pullrequest, pipeline.

Sample bitbucket-pipelines.yml Files

Python with pytest

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run Python Tests
          image: python:3.11
          script:
            - pip install -r requirements.txt
            - export PYTHONPATH="${PYTHONPATH}:$(pwd)"
            - pytest -v

Go

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run Go Tests
          image: golang:1.21
          script:
            - go test -v ./...

JavaScript/TypeScript with Jest

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run Jest Tests
          image: node:18
          script:
            - npm install
            - npm test -- --verbose

Ruby with RSpec

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run RSpec Tests
          image: ruby:3.2
          script:
            - gem install bundler
            - bundle install
            - bundle exec rspec --format documentation

Java with Gradle

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run JUnit Tests
          image: openjdk:17
          script:
            - chmod +x gradlew
            - ./gradlew test --info

Multi-language Monorepo

pipelines:
  pull-requests:
    '**':
      - parallel:
        - step:
            name: Run Python Backend Tests
            image: python:3.11
            script:
              - cd backend
              - pip install -r requirements.txt
              - export PYTHONPATH="${PYTHONPATH}:$(pwd)"
              - pytest -v
        - step:
            name: Run Jest Frontend Tests
            image: node:18
            script:
              - cd frontend
              - npm install
              - npm test -- --verbose

How Codity Detects Your Pipeline

  1. Commit lookup: Finds pipelines for the PR's source commit hash
  2. Branch fallback: If no match by commit, searches by source branch name
  3. Trigger fallback: Checks recent pipelines triggered by pull_request event
  4. Step filtering: Keeps only steps matching test keywords, skips lint/security/format steps
  5. Log retrieval: Fetches step logs using the Bitbucket Pipelines API
  6. Auto-fix: Analyzes failures and automatically fixes tests (up to 3 attempts)

Codity waits up to 180 seconds for a pipeline to start before declaring no CI found.

Common Issues and Solutions

"CI detection result: has_ci=False"

Step name does not contain any recognized test keywords. Rename the step to include test, ci, build, or a framework name like pytest, jest, rspec, gradle.

"ModuleNotFoundError" in Python tests

Set PYTHONPATH in your script: export PYTHONPATH="${PYTHONPATH}:$(pwd)"

Tests not triggered on PRs

Ensure pull-requests: section exists in bitbucket-pipelines.yml.

Permission errors fetching logs

The App Password is missing the pipeline scope. Re-generate with repository, pullrequest, and pipeline scopes.

"go.mod not found"

Add cd your-go-dir before running go test.

Checklist

  • [ ] Step name contains a recognized test keyword
  • [ ] pull-requests: trigger section is included
  • [ ] For Python: PYTHONPATH is set
  • [ ] Test commands use verbose flags (-v, --verbose)
  • [ ] Correct directory used if project files are in a subdirectory
  • [ ] App Password has repository, pullrequest, and pipeline scopes