Code Quality Scanning

Codity scans pull request diffs for code quality issues and posts findings as review comments. Scanning combines a fast pattern-based pass with LLM-based analysis, subject to model/service availability.

Overview

When an eligible quality scan runs, Codity analyzes the applicable changed lines and reports issues across eight quality categories:

  • Complexity: oversized functions, deep nesting, high cyclomatic complexity
  • Duplication: copy-pasted blocks, DRY violations
  • Error handling: bare except, swallowed exceptions, missing null checks
  • Performance: N+1 queries, blocking I/O in async contexts, unnecessary loops
  • Documentation: missing docstrings or JSDoc for public APIs
  • Testability: global state mutation, hard-coded dependencies
  • Maintainability: magic numbers, poorly named variables, dead code
  • Style: lines exceeding 120 characters

Two-Phase Analysis

Phase 1: Pattern Detection

When an eligible quality scan runs, Phase 1 uses static pattern matching on reviewable source files. It is fast and deterministic.

Rule Category Severity What it checks
CQ-001 Complexity Medium Function bodies over 200 lines, fully visible as consecutive added lines in the diff
CQ-003 Error Handling High Bare except or empty catch blocks
CQ-004 Error Handling High Exception caught and swallowed with pass
CQ-005 Maintainability Low TODO, FIXME, or HACK comments
CQ-006 Performance Medium SELECT * or .all() called inside a loop
CQ-007 Documentation Low Public function missing a docstring
CQ-008 Maintainability Low Magic numbers used directly in code
CQ-009 Style Low Lines longer than 120 characters
CQ-010 Testability Medium Global variable mutation
CQ-011 Duplication Low Duplicate import statements
CQ-012 Performance High Synchronous I/O (open, requests.get, sleep) in an async context

CQ-002 (deep nesting) is no longer a pattern rule. Context-dependent nesting concerns can be assessed by the LLM phase. Pattern rules also apply language-specific exclusions, and some rules skip test files.

Phase 2: LLM Analysis

Phase 2 uses an LLM to review the same diff with deeper reasoning. It catches nuanced issues that pattern matching misses, such as subtle logic errors, architectural concerns, and context-dependent duplication.

Phase 2 can use the platform model without BYOK. When the account's plan allows BYOK and a supported custom key is configured, the worker uses that provider instead. PR diffs are processed in batches within the model's input budget; the 10/50-file limits below apply to whole-repository quality scans.

Quality Score

Every scan produces a score from 0 to 100. Findings are grouped by rule type before scoring, so repeated instances of the same rule do not each add another penalty. The grouped rule severities determine the decay.

Penalty Weights

Severity Penalty per distinct rule type
Critical 10
High 5
Medium 2
Low / Info 0.5

Formula

penalty = (critical × 10) + (high × 5) + (medium × 2) + (low × 0.5)
score   = max(0, round(100 × (1 / (1 + penalty / 20))))

This is a harmonic decay, not linear subtraction. Here is why each part exists:

  • 1 +: prevents division by zero when penalty is 0, and sets the clean baseline. With zero findings the denominator is 1, giving 100 / 1 = 100. Every unit of penalty is measured relative to that 1.
  • penalty / 20: the 20 is the pivot point. When total penalty equals 20, the denominator becomes 2 and the score halves to 50. This means two critical issues (or four high issues) put you at exactly 50.
  • Harmonic decay: each additional finding hurts less than the previous one. Going from 0 to 1 critical issue drops the score by 33 points. Going from 10 to 11 critical issues drops it by roughly 1 point. This reflects reality: the gap between a clean codebase and one with a single critical issue is far more meaningful than the gap between a very poor codebase and a slightly worse one.
  • max(0, ...): a safety floor. The formula asymptotically approaches zero but never actually reaches it, so this guard exists only for floating-point edge cases.

Example scores

Findings Score
None 100
1 critical 67
2 critical 50
1 critical + 2 high 50
4 high + 5 medium 40

Finding Severity Levels

Each finding is assigned one of five severity levels:

Severity Description
Critical Must be fixed before merge
High Significant quality or correctness risk
Medium Should be addressed soon
Low Minor quality concern
Info Informational, no action required

Whole-Repository Scanning

Open Analytics → Code Quality. The current page separates PR Scans and Whole Repo Scans.

  • Choose Run PR Scan, select the repository and open PR, then Start Scan.
  • Choose Run Repo Scan, select the repository and branch in Run Whole Repo Scan, then Start Scan.

Check status and the completed result before downloading. This Analytics quality scan is separate from the paid one-time repository assessment.

Whole-repository quality scans analyze up to 10 reviewable source files with the platform model, or 50 with an applicable configured BYOK key. Generated files and other excluded file types do not consume those slots. A completed scan can therefore cover only part of a repository; it is not evidence that every file was analyzed.

Bring Your Own Key (BYOK)

The quality service supports account BYOK configuration when the plan includes it. Without an applicable key, the worker uses the platform model, including the reduced whole-repository file limit described above. The current scan dialogs select a repository and PR/branch; they do not contain a key-entry field or the legacy Settings → Whole Repository Scan switch. Ask the account administrator to check supported key setup and plan access when needed. Do not paste a key into Help.

Downloading Reports

Quality scan reports can be downloaded from the dashboard in two formats:

  • Markdown (.md): human-readable report
  • JSON (.json): machine-readable for integration with other tools

Next Steps