On-Premises & Self-Hosted Deployment¶

Estimated time: 16 min read · Part: CI/CD Across Cloud, On-Prem & Hybrid

Why On-Premises Still Exists¶

Every year, predictions of the death of on-premises computing fail to materialize. The reason is that for a significant portion of enterprise workloads, on-prem is not a legacy choice — it is the technically correct one. Understanding the four reasons it persists is essential for a consultant who will encounter clients in every quadrant.

Compliance and data residency is the most common hard constraint. GDPR requires that EU citizen data not leave EU jurisdictions without appropriate safeguards. HIPAA requires strict control over protected health information (PHI) and audit trails on every access. FedRAMP requires US government data to stay in FedRAMP-authorized environments. Financial regulations (MiFID II, SOX, PCI-DSS) impose audit requirements and residency controls. While cloud providers now offer regional data residency controls and have obtained many compliance certifications, some organizations’ legal or risk teams require on-prem as the only acceptable path — and in some jurisdictions and sectors, they are right.

Data gravity applies when the dataset is so large that moving it to the cloud is economically impractical. A genomics lab with 5 petabytes of sequencing data on local storage does not move that data to the cloud for analysis — it brings the compute to the data. An internet exchange point, a stock exchange, a weather modeling agency: each has terabyte-per-second data streams that gravity anchors to on-prem hardware.

Latency-critical applications include manufacturing automation (PLCs, SCADA systems), robotics, high-frequency trading, real-time control systems for medical devices, and edge computing for autonomous vehicles. These applications require single-digit millisecond response times or deterministic scheduling that cloud networks — with their inherent latency variability — cannot guarantee.

Economics at scale: at very large numbers of nodes (typically above 500–1,000 equivalent machines running continuously), the amortized cost of owned hardware often undercuts cloud pricing. Large streaming companies, social platforms, and financial institutions have made this calculation and operate their own hardware, sometimes alongside cloud bursting capacity.

The Self-Hosted Runner Model: Pull, Not Push¶

The key architectural insight for deploying to on-prem from a cloud-based CI system is the direction of the connection. A naive approach would be to have the CI system push code or commands into the on-prem network — which would require opening inbound firewall ports and creating a complex network exposure. The correct model inverts this.

A self-hosted runner (GitHub Actions terminology; analogous to a GitLab Runner, Jenkins agent, or Buildkite agent) is a process that runs inside the corporate network. It initiates an outbound HTTPS connection to the CI control plane (GitHub, GitLab.com, or your self-hosted CI server) and polls for available jobs. When a job matching its labels is queued, the runner pulls the job definition, executes it locally, and reports results back via the same outbound connection. The runner can reach any host inside the corporate network, including the production Kubernetes cluster, database hosts, or artifact servers — because it is physically inside the perimeter.

flowchart TB
    subgraph CLOUD["GitHub (Cloud)"]
        GH["GitHub Actions<br/>Control Plane<br/>(queue jobs)"]
    end
    subgraph CORP["Corporate Network (On-Prem)"]
        direction TB
        R["Self-Hosted Runner<br/>(outbound HTTPS only)"]
        K8S["On-Prem Kubernetes<br/>Cluster"]
        DB[("Regulated<br/>Database")]
        R -->|kubectl apply| K8S
        K8S --- DB
    end
    GH <-->|"Outbound HTTPS port 443<br/>(runner initiates)"| R
    style CLOUD fill:#1e3a5f,color:#e2e8f0
    style CORP fill:#2a1e3a,color:#e2e8f0

The firewall rule is simple: allow outbound HTTPS (port 443) from the runner host to api.github.com, *.actions.githubusercontent.com, and the container registry. No inbound rules required.

Installing and Registering a Self-Hosted Runner¶

GitHub provides a runner binary for Linux, macOS, and Windows. The registration process uses a short-lived registration token from the GitHub API; thereafter the runner communicates using a long-lived but non-privileged credential that cannot be used to authenticate as the repository owner.

# 1. Download the runner binary (Linux x64 example)
mkdir -p /opt/github-runner && cd /opt/github-runner
curl -o actions-runner-linux-x64.tar.gz -L \
  https://github.com/actions/runner/releases/download/v2.316.0/actions-runner-linux-x64-2.316.0.tar.gz
tar xzf actions-runner-linux-x64.tar.gz

# 2. Register against your repo or org
#    --token is retrieved from Settings > Actions > Runners > New self-hosted runner
./config.sh \
  --url https://github.com/myorg/myrepo \
  --token ABCDEFGTOKEN123 \
  --name prod-onprem-runner-01 \
  --labels self-hosted,linux,onprem,production \
  --runnergroup on-premises

# 3. Install as a systemd service so it survives reboots
sudo ./svc.sh install
sudo ./svc.sh start
sudo ./svc.sh status

In the workflow, the runner is targeted by label so cloud-intended jobs run on GitHub-hosted runners and on-prem jobs run on self-hosted runners:

name: Deploy to On-Prem Kubernetes

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest         # GitHub-hosted: build the image, push to Harbor

    steps:
      - uses: actions/checkout@v4
      - name: Build and push to Harbor
        run: |
          docker build -t harbor.corp.internal/apps/api:${{ github.sha }} .
          docker push harbor.corp.internal/apps/api:${{ github.sha }}

  deploy:
    runs-on: [self-hosted, onprem, production]   # Self-hosted: inside the firewall
    needs: build

    steps:
      # The kubeconfig for the on-prem cluster is stored as a CI secret
      # (encrypted in GitHub, decrypted into the runner's environment)
      - name: Set up kubeconfig
        run: |
          mkdir -p ~/.kube
          echo "${{ secrets.ONPREM_KUBECONFIG_B64 }}" | base64 -d > ~/.kube/config
          chmod 600 ~/.kube/config

      - name: Deploy to on-prem cluster
        run: |
          kubectl set image deployment/api \
            api=harbor.corp.internal/apps/api:${{ github.sha }} \
            --namespace production

      - name: Wait for rollout
        run: |
          kubectl rollout status deployment/api \
            --namespace production \
            --timeout=5m

Air-Gapped Environments and Internal Registries¶

Some environments go beyond “on-prem with internet access” to full air gaps: the network has no outbound internet connectivity at all. Every artifact must be pre-approved, scanned, and loaded onto internal mirrors before use. This is common in defense, intelligence, and the most security-conscious financial institutions.

Harbor is the CNCF-graduated on-premises container registry. It provides Docker Hub-compatible APIs, built-in vulnerability scanning (via Trivy or Clair), image signing, replication between Harbor instances, and robot accounts for CI authentication. In an air-gapped setup, Harbor is the single source of truth for container images: images are pulled from the internet by a designated admin workstation in a DMZ, scanned and signed, then mirrored into the air-gapped Harbor instance.

Artifactory or Nexus Repository serve as proxies and caches for language package managers: PyPI (for pip install), npm, Maven, NuGet, apt/yum/dnf. In an air-gapped environment, developers and CI runners point their package managers at the internal Artifactory rather than the internet. The Artifactory is pre-populated with approved packages by a separate process.

[global]
# Redirect all pip installs to internal Artifactory PyPI mirror
index-url = https://artifactory.corp.internal/artifactory/api/pypi/pypi-virtual/simple
trusted-host = artifactory.corp.internal

# Disable pip's check for newer pip versions (no internet)
disable-pip-version-check = 1

On-Premises Kubernetes Distributions¶

Once you’re deploying to on-prem Kubernetes, the deployment commands are identical to cloud K8s (kubectl apply, helm upgrade, kubectl set image). Only the kubeconfig endpoint changes. The distribution choice is an operational and commercial decision:

Rollout Strategies On-Premises¶

Kubernetes supports multiple rollout strategies that work identically on-prem and in the cloud:

Rolling update (default): Kubernetes replaces pods one by one (or in configurable batches), ensuring maxUnavailable pods are replaced at a time and maxSurge extra pods can run during the transition. Zero downtime for stateless services. Config in the Deployment spec.

Blue/green: two full Deployments exist simultaneously (blue = current, green = new). Traffic is flipped from blue to green by updating a Service selector. Rollback is instantaneous: flip the selector back. Higher resource cost during the transition.

Canary: a small subset of pods (e.g., 5%) runs the new version; the rest runs the old. Traffic is split by replica ratio or by an ingress controller (Nginx, Istio). Metrics are evaluated; if acceptable, the canary is graduated to 100%.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: production
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1    # At most 1 pod unavailable at any time
      maxSurge: 2          # At most 2 extra pods running during update
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
        version: "abc123f"   # Updated per release for traceability
    spec:
      containers:
        - name: api
          image: harbor.corp.internal/apps/api:abc123f
          ports:
            - containerPort: 8000
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

name: Deploy Worker On-Prem

on:
  workflow_dispatch:
    inputs:
      sha:
        description: Git SHA to deploy
        required: true

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.inputs.sha }}
      - name: Build and push to Harbor
        run: |
          docker build -t harbor.corp.internal/apps/worker:${{ github.event.inputs.sha }} .
          docker push harbor.corp.internal/apps/worker:${{ github.event.inputs.sha }}

  deploy:
    runs-on: [self-hosted, onprem, production]
    needs: build
    steps:
      - name: Configure kubeconfig
        run: |
          mkdir -p ~/.kube
          echo "${{ secrets.PROD_KUBECONFIG_B64 }}" | base64 -d > ~/.kube/config
          chmod 600 ~/.kube/config

      - name: Update deployment image
        run: |
          kubectl set image deployment/worker \
            worker=harbor.corp.internal/apps/worker:${{ github.event.inputs.sha }} \
            --namespace production

      - name: Wait for rollout to complete
        run: |
          kubectl rollout status deployment/worker \
            --namespace production \
            --timeout=10m