Skip to main content

How to Run Your Own Building Block Runner

What is this guide about?

This guide explains what building block runners are, why you might want to self-host one, and the high-level steps to get one running.

What is a Building Block Runner?​

A building block runner is a small service that connects meshStack to your CI/CD platform. When an application team adds a building block to their workspace or project, meshStack queues a building block run and routes it to the runner assigned to that building block definition. The runner picks up the queued run, triggers the corresponding automation (e.g. a Terraform script, a GitHub Actions workflow, a GitLab pipeline, or an Azure DevOps pipeline), and reports progress and status back to meshStack.

meshStack ships with hosted runners for Terraform, Manual, GitHub Actions, GitLab CI/CD, and Azure DevOps Pipelines. For most use cases, you don't need to run your own.

Should You Run Your Own Runner?​

You need a self-hosted runner when:

  • You need to reach a private network (private cloud or on-premise Git servers) that isn't reachable from meshStack's infrastructure
  • You want dedicated isolation — a private runner only picks up runs assigned to it, with no cross-workspace interference
  • Your organization's security policies require that pipeline credentials never leave your network

How Runners Work​

The runner polls meshStack every few seconds for pending building block runs. When a run is available, it triggers your pipeline and reports status back. Depending on the execution mode on the building block definition, the runner either polls the automation pipeline for completion (synchronous) or hands off and waits for the automation pipeline to report back (asynchronous).

Secret inputs are encrypted with the runner's RSA public key — only the runner, using the matching private key, can decrypt them at runtime.

meshStack building block runners are deployed as Docker containers. They are open sourced on GitHub, so you can run them on your own infrastructure, customize them for your needs, or simply use meshStack's hosted runners.

Changing the Runner's Key Pair

If you replace the key pair after building blocks have stored encrypted inputs, those inputs can no longer be decrypted. Static inputs need a new definition version; user inputs need to be re-submitted by workspace users. Plan key rotation carefully.

Setting Up a Self-Hosted Runner​

The easiest way to set up a runner is through the Platform Builder or Admin Area UI in meshStack — the interface walks you through each field and generates the configuration for you. The high-level steps are:

  1. Generate an RSA key pair. meshStack encrypts sensitive building block inputs with the runner's public key. Only your runner, with the private key, can decrypt them.
  2. Register the runner in meshStack. Create a Building Block Runner in the Platform Builder/Admin Area (or via the meshObject API), providing your public key and the implementation type. meshStack assigns the runner a UUID. If your runs use Workload Identity Federation, also declare the identity your runner presents - see Declaring the Runner's Own Identity.
  3. Create an API key for the runner. The runner needs an API key to poll meshStack for work. Once the runner is registered, provision one from the runner's detail page in meshStack — see Runner Visibility and API Key Permissions below for the exact permissions required.
  4. Deploy the runner container. Run the Docker image from the meshcloud container registry, passing the runner UUID, meshStack URL, API credentials, and private key as configuration. The runner starts polling immediately.
  5. Verify and assign. Confirm the runner shows as Active in meshStack, then select it on the Implementation tab of your building block definition.
Runner Status

meshStack marks a runner Active if it has polled within the last 5 minutes, Pending right after registration, and Error if it goes silent. You can see the last known runner version in the runner details view.

Declaring the Runner's Own Identity​

If your runs authenticate to a cloud provider with Workload Identity Federation instead of long-lived credentials, tell meshStack which identity your runner presents. You do that with the optional spec.workloadIdentityFederation section when you register or update the runner.

The tokens can come from wherever your runner gets them: the cluster it runs in, HashiCorp Vault, a SPIFFE workload API, your own identity provider, or your CI system's OIDC provider. meshStack never mints them and never checks who does. What the cloud provider sees is an issuer, a subject and an audience, and that is exactly what you declare:

  • issuer — the OIDC issuer URL of whoever signs the tokens
  • subjectTemplate — the subject claim your tokens carry
  • gcp, aws and azure — the audience and the file path each token is written to

Writing the Subject Template​

Most runners mint one identity per building block definition so the trust you set up in a cloud provider stays scoped to a single definition. Write subjectTemplate with the two placeholders that stand for the definition a run belongs to:

PlaceholderStands for
{{ workspaceIdentifier }}The identifier of the workspace that owns the building block definition
{{ buildingBlockDefinitionUuid }}The UUID of the building block definition

Whitespace inside the braces is optional, so {{workspaceIdentifier}} works too. No other name is accepted, and a template that names one is rejected when you register the runner.

A template like this one:

system:serviceaccount:my-runners:workspace.{{ workspaceIdentifier }}.buildingblockdefinition.{{ buildingBlockDefinitionUuid }}

resolves per definition to:

system:serviceaccount:my-runners:workspace.my-team.buildingblockdefinition.3f2a1c9e-7b40-4d8e-9c11-2e5a6d0f8b31

If your runner's token issuer cannot put the definition into the subject claim, for example a single-pod runner or a CI system whose subject is fixed by repository or pipeline, register a template without placeholders. Every building block definition on that runner then shares one identity, so everyone who can use the runner can use whatever you trust it with.

Reading the Resolved Values

meshStack resolves the template for each version of a building block definition and publishes the result. Platform builders see it under Implementation → Workload Identity Setup on the definition, and the meshObject API returns it as status.versions[].workloadIdentityFederation. Use those values in your cloud provider's trust policy rather than assembling the subject yourself. Everyone who can read the definition can read these values; they are not secrets, because the cloud provider only trusts a token that the declared issuer has signed. Building blocks stay on the version they were created with, so the trust should cover the subject of every version that building blocks still run on.

Runner Visibility and API Key Permissions​

Every runner is owned by a workspace and has a restriction that determines where it can be used:

  • Private runners (default) are restricted to the workspace that owns them. A private runner only executes runs for building block definitions owned by that same workspace. This is the only option available when registering a runner from a workspace's Platform Builder.
  • Public (shared) runners are owned by the admin workspace and available to every workspace in meshStack — they can execute runs for building block definitions owned by any workspace. Only admins can create public runners, from the Admin Area. meshStack's built-in hosted runners are public runners.

The API key that authenticates your runner needs different permissions depending on which type of runner it belongs to:

For a private runner owned by a platform team's workspace:

  • Read Runs — fetch run specifications and inputs
  • Write Runs — trigger and update runs
  • Write Run Sources — report run status, logs, and outputs

For a public runner owned by the admin workspace:

  • Read Runs (Admin) — fetch run specifications and inputs across all workspaces
  • Write Runs (Admin) — trigger and update runs across all workspaces
  • Write Run Sources (Admin) — report run status, logs, and outputs across all workspaces
Provisioning the Key

Use the Provision API Key action on the runner's detail page to have meshStack create the key for you with the correct permission set already applied, based on the runner's visibility. You don't need to pick permissions manually.

API Key and Runner Workspace

The API key must belong to the same workspace that owns the runner. For a private runner, that workspace must also own the building block definitions the runner should execute — for example, if workspace A owns a building block definition and a private runner assigned to it, a building block instantiated from that definition in workspace B is still executed using the runner (and API key) in workspace A. Public runners don't have this restriction, since they can execute definitions from any workspace.

What's Coming: Run Controller​

We're working on open-sourcing our run controller — a Kubernetes-native variant that spawns a short-lived Kubernetes Job for each building block run instead of a single continuously polling container. This is the same architecture we use for our built-in runners already.

Benefits over the Docker runner:

  • Parallelization — runs execute concurrently because each gets its own isolated job
  • All implementation types in one — a single run controller handles Terraform, GitHub Actions, GitLab CI/CD, Azure DevOps Pipelines, and Manual building blocks without deploying a separate container per type

If you set up a Docker-based runner today, you'll be able to migrate it to a run controller in the future without re-registering or rotating keys.

Concepts​

Guides​