Nine Fixes and a Delete: A CI/CD Debugging Story

Analysis of a CI/CD debugging cycle in which nine iterative patches addressed a workflow that was ultimately unnecessary. Key findings cover assumption-driven engineering, AI-assisted tool chains, and the cost of deferred platform discovery.

Executive Summary

A standard CI/CD deploy workflow consumed approximately one full engineering day across nine iterative fix commits before a tenth commit deleted all 33 lines. The workflow was rendered unnecessary by a native Gitea integration built into the deployment platform — an integration that had been active since initial configuration. This analysis examines the specific failure mode responsible: a correct but unexamined assumption applied in a context where it did not hold. It further examines how AI-assisted debugging tools, while increasing fix velocity, amplified investment in an incorrect problem frame. Three procedural controls are identified that would have prevented the waste and that apply broadly to platform-adjacent engineering work.

---

Key Findings

• Assumption-driven engineering produces technically correct artifacts for the wrong problem. The workflow code that resulted from nine iterations was functionally sound; its defect was existential, not technical.
• AI-assisted debugging tools increase iteration velocity without evaluating problem validity. Each error message became a prompt; each prompt produced a valid patch; the loop never surfaced the question of whether the workflow was necessary.
• Platform discovery deferred past implementation is consistently more expensive than platform discovery prior to implementation. In this case, a five-minute dashboard review would have eliminated a full day of engineering effort.
• Diagnostic minimization — stripping a system to its smallest verifiable unit — is the correct first debugging step, not the seventh. The echo-hello commit that confirmed runner health should have preceded all workflow-specific fixes.
• Self-hosted CI environments introduce systematic incompatibilities with cloud-native action ecosystems. These incompatibilities are known, documented, and recur predictably; they warrant a pre-implementation compatibility checklist.
• Deletion is a valid and underutilized engineering output. Removing 33 lines of correct but unnecessary code is a strictly better outcome than maintaining them.

---

1. Background and System Context

The deployment target was a documentation platform with a native version control integration. The engineering environment used a self-hosted version control system running a local action runner rather than the cloud-hosted runner environment for which most community CI actions are designed.

The initial workflow followed a standard pattern: on push to the default branch, check out the repository and invoke the platform’s deployment action.

name: Deploy to Mintlify
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: mintlify/action@v1
        with:
          token: ${{ secrets.MINTLIFY_TOKEN }}

This workflow was entirely appropriate for a GitHub-hosted runner environment. It was entirely inappropriate for the self-hosted environment in which it was executed.

---

2. Failure Analysis: Commits 1 Through 9

2.1 The Checkout Incompatibility (Commits 1–3)

The first failure was immediate and predictable in retrospect. The actions/checkout@v4 action expects a GitHub runner environment and cannot execute on Gitea’s act_runner. This is a known, documented incompatibility — not a configuration error but a fundamental environmental mismatch.

Three commits addressed the consequences of this incompatibility in sequence:

1. Replace the managed checkout action with a manual git clone invocation.
2. Correct the SHA reference variable, which Gitea 1.24 exposes differently from the GitHub context model.
3. Switch to a depth-1 clone without specifying a ref, to avoid failures when fetching by commit SHA.

Each fix was technically correct in isolation. Each fix addressed a real error. None of the three commits questioned whether the workflow was the appropriate solution.

2.2 The Runner Configuration Failures (Commits 4–6)

With the checkout resolved, runner configuration issues emerged. The runner label ubuntu-latest was not registered on the self-hosted instance, causing the job to queue indefinitely. The platform deployment action required an external action runner context unavailable in the self-hosted environment. A workspace directory conflict caused the clone step to fail when prior files were present.

Each issue was isolated, diagnosed, and patched. The workflow grew from 22 lines to 33 lines. Code added to explain workarounds began to outnumber code that performed actual work.

(For practitioners establishing a self-hosted Gitea CI pipeline from baseline, this walkthrough of Gitea Actions, act_runner, and Docker-in-Docker inside k3s documents six of these same incompatibilities in systematic detail.)

2.3 Diagnostic Minimization — Applied Late (Commit 7)

Commit seven stripped the entire workflow to a single shell command to verify runner execution independent of workflow logic:

steps:
  - name: Hello
    run: echo hello

The runner executed successfully. This confirmed that the problem space was contained to workflow logic, not infrastructure. This diagnostic step — reducing the system to its minimum verifiable unit before layering complexity — is the canonical first step in debugging unfamiliar infrastructure. It was executed as the seventh step.

2.4 Final Restoration and Green State (Commits 8–9)

With runner health confirmed, the full workflow was restored with authentication headers added to accommodate a self-hosted setup that requires credentials even for public repository access. The workflow passed. Nine commits. Thirty-three lines. A functioning CI deploy pipeline.

2.5 The Delete (Commit 10)

Reviewing the deployment platform dashboard that evening revealed a native Gitea integration configured in January. The platform had been monitoring the repository and executing deploys on every push to the default branch since initial setup — silently, because there was nothing to break.

chore: remove deploy workflow — Mintlify syncs directly from Gitea

Auto-deploy via Mintlify's Gitea integration works without a CI workflow.
Removing the redundant workflow to avoid noise from failed runs.

The workflow that consumed a full engineering day was deleted. The actual deploy mechanism had been operational throughout.

---

3. Root Cause: The Dead Assumption

The failure mode was precise and is worth naming with precision. It was not inattention, and it was not incompetence. It was the application of a correct general rule — “CI deployments require a workflow” — in a context where that rule did not hold, without checking whether this was one of the cases where it does not hold.

This pattern recurs across engineering disciplines. A rule is correct in a sufficiently high proportion of cases that it is applied without examination. The cases where it fails are precisely the cases where examination would have been cheapest. This failure mode is examined further in the context of recurring debugging spirals at this analysis of cascading errors in AI-assisted development — the mechanism is identical regardless of scale.

The pattern is also documented as a general engineering failure mode at this discussion of dead assumptions.

---

4. AI Tool Chain: Amplification of Direction

AI assistance was used throughout the debugging process. Each failure produced an error message; each error message was provided as a prompt; each prompt produced a technically valid patch. The co-authorship was genuine — the AI did not make errors in the patches it generated.

The AI tool chain amplified the speed and consistency of movement in the chosen direction. It did not evaluate whether the direction was correct. This is an accurate description of the current capability boundary of AI-assisted debugging tools: they are optimizers, not navigators.

Dimension	AI Contribution	Human Responsibility
Error interpretation	High — rapid, accurate diagnosis	Low
Patch generation	High — syntactically and semantically correct	Low
Problem framing	None	Full
Platform discovery	None	Full
Question validity (“Do I need this?”)	None	Full

The practical implication is not that AI assistance is unreliable. It is that AI assistance requires the human operator to supply correct problem framing before engaging the tool chain. A precisely wrong question, asked efficiently, produces a precise wrong answer quickly.

The question driving every prompt in a debugging loop determines the quality of the loop’s outcome. “How do I fix this workflow?” is a different question from “Does this workflow need to exist?” AI assistance will follow the framing it is given. Responsibility for that framing rests entirely with the operator.

---

5. Comparison: Debugging Strategies

The following table compares the approach taken against two alternatives that would have produced faster resolution.

Strategy	Time to Resolution	Commits Required	Core Action
Approach used: fix each error as encountered	~1 day	9	Patch individual failures sequentially
Platform-first: review integration documentation before building	~5 minutes	0	Discover native integration in dashboard
Minimization-first: verify assumptions before fixing	~2 hours	2–3	Echo-hello first, then platform review

The platform-first approach was strictly superior: zero code written, zero time spent on incompatibilities, zero lines to delete. The minimization-first approach would have surfaced the question of platform coverage within a few commits. Both alternatives required only a change in procedural sequence, not additional capability.

---

6. Recommendations

Recommendation 1: Conduct platform integration review before implementing any tooling that operates on top of a platform.

Integration documentation — specifically the “how does this connect to X” category — must be reviewed prior to implementation, not during debugging. This review should take no longer than 15 minutes and should include checking for native integrations, webhooks, and existing automation configured during platform setup.

Recommendation 2: Define and apply an existence check prior to any infrastructure build.

For any proposed piece of new infrastructure, a documented question must be answered before implementation begins: “What already exists that may make this unnecessary?” This check should be treated as a blocking prerequisite. It takes minutes and occasionally saves days.

Recommendation 3: Execute diagnostic minimization as the first step in infrastructure debugging, not after exhausting other options.

Before addressing any specific error in an unfamiliar system, strip the system to its minimum executable unit and verify that the underlying infrastructure is functional. In CI contexts, this means a single-command workflow before any application-specific logic is restored.

Diagnostic minimization requires discipline to apply early. The instinct to fix the visible error is strong. The correct instinct is to verify that the visible error exists in a context where fixing it matters.

Recommendation 4: When using AI-assisted debugging, periodically restate the problem from first principles.

At regular intervals during a debugging session, the operator should set aside the immediate error and ask: “What am I trying to accomplish, and is the approach I am pursuing the correct path to that goal?” This discipline prevents efficient progress in a wrong direction.

Recommendation 5: Develop a self-hosted CI compatibility checklist for known action ecosystem incompatibilities.

Gitea’s act_runner has documented incompatibilities with community actions designed for GitHub-hosted runners. These incompatibilities recur predictably. A pre-implementation checklist covering runner labels, context variable differences, action compatibility, and workspace state should be maintained and applied before any new workflow is written.

Recommendation 6: Treat code deletion as a first-class engineering output.

The deletion of 33 lines of correct but unnecessary code was the most valuable commit of the day. Engineering process should not treat deletion as an admission of failure. It should treat it as the correct outcome when investigation reveals that a component is redundant.

Maintain a habit of reviewing existing platform capabilities after any significant debugging session involving platform-adjacent tooling. The knowledge gained during debugging often surfaces questions about platform coverage that were not visible at implementation time.

---

7. Conclusion and Forward-Looking Assessment

The nine-commit debugging cycle documented here represents a specific, reproducible failure mode in platform-adjacent engineering. The technical knowledge acquired — Gitea 1.24 context variable behavior, act_runner action compatibility constraints, self-hosted runner configuration patterns — is genuine and applicable. It was acquired at a cost disproportionate to the problem it addressed.

As AI-assisted development tools become standard components of engineering workflows, the leverage they provide on implementation and debugging velocity increases the cost of incorrect problem framing. The productivity multiplier AI applies to iteration amplifies both progress toward a valid goal and investment in an invalid one. This dynamic will become more pronounced as tool capability increases.

Engineering teams adopting AI-assisted workflows should expect that process discipline — particularly the discipline of problem framing, platform review, and assumption validation — will become more important, not less, as AI tools become more capable. The tools are optimizers. The choice of what to optimize remains a human responsibility.

---

All content represents personal learning from personal projects. Code examples are sanitized and generalized. No proprietary information is shared. Opinions are my own and do not reflect my employer’s views.