How to manually integrate GCP as meshPlatform
The recommended way to set up GCP as a meshPlatform is via the public terraform GCP meshPlatform Module. The steps below are not needed if you decide to use it.
Set up the Service Account for Replication
meshStack needs a well-defined set of permissions for its automation. meshStack is designed so that it does not require access to workload. We highly recommend that permissions are configured according to the "least privilege" principle.
Platform Operators need to define a Custom IAM Role called meshfed-service
at the Organization Level with the following permissions
resourcemanager.folders.get
resourcemanager.folders.list
resourcemanager.organizations.get
resourcemanager.projects.create
resourcemanager.projects.get
resourcemanager.projects.getIamPolicy
resourcemanager.projects.list
resourcemanager.projects.move
resourcemanager.projects.setIamPolicy
resourcemanager.projects.update
resourcemanager.projects.createBillingAssignment
resourcemanager.projects.deleteBillingAssignment
billing.resourceAssociations.create
serviceusage.services.enable
serviceusage.services.get
deploymentmanager.deployments.delete
deploymentmanager.deployments.create
deploymentmanager.deployments.update
deploymentmanager.deployments.get
In order to enable meshStack to delete GCP Projects as part of tenant deletion, please also include the following permission. We strongly recommend you assign this permission only on those Folders where you want to allow automated tenant deletion.
resourcemanager.project.delete
Configure the Root Project
meshStack requires a project in GCP for some of the resources it uses. It is reserved for use by meshstack and Platform Operators. For this guide, we’ll call the project meshstack-root
.
Enable APIs
Enable the following APIs on the meshstack-root
project from the API Library
Create meshfed-service Service Account
Create a meshfed-service
Service Account in the meshstack-root
project.
Enable the Service Account for “G Suite Domain-wide Delegation” and note the generated Client Id
.
The Service Account will be identified by an email address like
meshfed-service@meshstack-root.iam.gserviceaccount.com
Next, configure either a service account key or workload identity federation for authentication purposes.
Service Account Key
Generate and Download a Service Account Key which will be used by meshStack.
Workload Identity Federation
Setting up Workload Identity Federation is more involved but allows meshStack to use short lived credentials without explicitly handing over any secrets or requiring manual rotation.
To use Workload Identity Federation you must create a Workload Identity Pool and Provider (IAM → Workload Identity Federation) as part of the meshStack root project.
- For the provider, select "OpenID Connect" and add provider details for "Issuer" and "Allowed audiences" as they're specified by the panel when selecting "Workload Identity Federation" as authentication type for a GCP platform.
- Create an attribute condition to restrict access to replicator and/or metering service accounts:
google.subject in ['<replicator_subject_from_panel>', '<kraken_subject_from_panel']
. Make sure to replace the placeholders.
After setting up identity pool and provider you need to grant access to the meshStack workload service accounts to access the GCP service accounts.
- Select the workload identity pool and click "grant access".
- Select the GCP service account (e.g. for replication).
- Define the appropriate principal by mapping "subject" to the subject name you received from the panel.
- When prompted, enter anything for OIDC ID token path and download the generated configuration file. Open it to get the
audience
value that is required by meshStack.
In addition to the permissions required to perform resource management or metering the replicator/metering service account must also be permitted to create ID tokens for itself.
ID tokens must be used when calling cloud functions and authentication via workload identity federation does not automatically create such tokens.
Add this permissions by assigning the roles roles/iam.serviceAccountOpenIdTokenCreator
and roles/iam.serviceAccountTokenCreator
to the the service account.
Grant Resource Permissions
The Service Account will be used by meshStack to perform project replication. Operators thus need to grant it the permissions of the
meshfed-service
IAM role on those folders of the GCP resource hierarchy
that make up the Landing Zones for client projects.
It's a best practice to segregate "user" and "infrastructure" projects in GCP using the resource hierarchy. By setting granular permissions (instead of organization-wide permissions) this can limit the access of meshStack's replicator to only the parts of the resource hierarchy that it needs to actively manage (principle of least privilege).
Grant Billing Account Permissions to the Service Account
In order to associate created projects with a Billing Account, the replicator needs to be granted the
billing.resourceAssociations.create
permission on the Billing Account. This is best achieved by assigning the
meshfed-service
IAM Role to the meshfed-service
Service Account on the Billing Account in the Billing Account's permissions.
Optional: Billing Account owned by a different organization
In order to use a billing account that is owned by a different organization the permissons for meshfed-service
user need to be adjusted.
Platform Operators create a custom role meshfed-billing-creator
in the organization that owns the target billing account with the following permisson
billing.resourceAssociations.create
The meshfed-service
user needs to be granted the meshfed-billing-creator
role in the organization that owns the target billing account.
Following the principle of least privilege, operators should remove the billing.resourceAssociations.create
permisson from the custom role meshfed-service
created in meshfed-service IAM Role.
Set up Cloud Identity
Authorize the Service Account
In order to perform certain group related administrative tasks the previously created meshfed-service
service account needs the "Groups Admin" role from the Admin Console (G Suite).
To authorize the Service Account via the Google Admin Console navigate to @Account
in the sidebar and then Admin Roles → Groups Admin
and click Assign Service Accounts
. In the prompt that appears, enter the service account email, which looks like user@project.iam.gserviceaccount.com
.
You can alternatively authorize the Service Account via the Cloud Identity Groups API. Please find the instructions in for this in the official Google guide.
Set up the Service Account for Metering
meshStack needs a separate service account to provide metering functionality. Create a new service account kraken-service
and
assign it the following predefined roles:
roles/bigquery.jobUser (on the project of the ServiceAccount)
roles/bigquery.dataViewer (on the project that holds the bigquery billing/carbon export dataset)
To enable meshStack to periodically collect active projects, create an IAM role with the following permissions and assign it to the service account.
resourcemanager.folders.get
resourcemanager.folders.list
resourcemanager.projects.get
resourcemanager.projects.list
Set up GCP Billing data export
Complete the GCP instructions to set up Cloud Billing data export to BigQuery. We recommend using the "standard usage cost data" export with meshStack.
Once billing export has been setup as explained in the GCP documentation linked above, set up the kraken-service
service account with
access to the exported billing dataset. You can set up this permission either on the project containing the dataset
or directly on the dataset itself.
roles/bigquery.dataViewer
Optional: Provide a custom BigQuery table
Beyond the default BigQuery Billing Export, meshStack can also support custom BigQuery tables. You might want to apply some custom transformations or logic on the data. As long as the view or table is given in the same format as the default BigQuery Billing Export, meshStack can handle and process the data.
Below, the columns are listed that are required by meshStack, and in which format. Make sure that the data types are the same as the default export.
- invoice.month
- project.id
- service.id
- service.description
- sku.id
- sku.description
- currency
- usage.pricing_unit
- cost
- credits.amount
- usage.amount_in_pricing_units
- cost_type
- Partition Time*
*Since BigQuery doesn't allow column names that start with an underscore character, you have to map the _PARTITIONTIME
column to a different name (see example query below). This column name has to be configured in the platform configuration
under 'Partition Time Column'.
Make sure that the kraken-service
service account has permission on the table or view as described above.
Then, enter the BigQuery reference in the GCP meshPlatform configuration in meshStack, which you can find under
'Metering Configuration' → 'BigQuery Table'.
Example: multiple billing accounts for the same GCP organization
There can be situations where you want the projects inside a single GCP meshPlatform to be associated with multiple billing accounts. One such example is when you have one billing account which has Spend-based committed use discounts applied and another billing account which contains free credit applied. To ensure that meshStack collects data from both the accounts, you can to create a view in BigQuery which is a union of the two billing data exports from the two billing accounts. Such a union can be created with a query which looks similar to the following.
(SELECT _PARTITIONTIME as PARTITIONTIME, billing_account_id,service, sku, STRUCT(project.id as id, project.ancestry_numbers as ancestry_numbers) as project, labels, system_labels,
location, cost, currency, usage, credits, invoice, cost_type, adjustment_info
from `project-name-root.billing_export.gcp_billing_export_general`)
UNION ALL
(SELECT _PARTITIONTIME as PARTITIONTIME,billing_account_id,service, sku, STRUCT(project.id as id, project.ancestry_numbers as ancestry_numbers) as project, labels, system_labels,
location, cost, currency, usage, credits, invoice, cost_type, adjustment_info
from `project-name.billing_export.gcp_billing_export_credit`)
Configure this view as mentioned in the section above and meshStack will start using this data for cost collection.
Optional: Filter billable projects from the export
The GCP platform configuration offers an "additional filter" setting. meshStack will include configured filters in the
WHERE
clause of its query to read billing data from the export table. This enables scenarios like excluding projects
under a certain part of the organizational hiearchy from meshStack.
The following example excludes all projects nested underneath folder 123456789
in your resource hierarchy.
AND "123456789" IN UNNEST(SPLIT(project.ancestry_number, "/"))
Optional: Enable GCP Cloud Carbon Footprint Export
meshStack metering supports general environmental reports based on GCP Cloud Carbon Footprint data. To enable this feature,set up a big query data set containing the carbon footprint export data, see Export your carbon footprint on the GCP documentation.
Then enable the kraken-service
service account to read this dataset.
roles/bigquery.dataViewer
Finally configure the table id containing the exported carbon data in the meshPlatform configuration.
Setting up GCP Big Query Transfer configuration will execute the transfer job on behalf of the user setting up the transfer config. The user must therefore grant permission to the big query service to access their account. This is a limitation of the big query data transfer service authorization model.
Optional: Enable Audit Logs for meshfed-service User
The actions of the meshfed-service
User can be monitored via Audit Logs. This allows an in-depth view meshStack activities for GCP project at any moment.
Enable Audit Logs
Enabling Audit Logs may incur charges.
meshcloud recommends to enable Audit Logs on the organizational level for monitoring meshfed-service
User. This is achivied by following these steps:
- Navigate to the organizational level in GCP Cloud Console
- Navigate to IAM & Admin → Audit logs
- Filter the table for
Cloud Resource Manager API
and select the resulting entry - Enable all log types
You may want to check the official Google instructions on enabling Auit Logs for further information.
The below screen shot show how to set up the Audit Logs for the organization dev.meshcloud.io
Query Audit Logs in Google Cloud Console
Please consult Google docs for options to querying Audit Logs.