Edit

How to manually integrate AWS as meshPlatform

The recommended way to set up AWS as a meshPlatform is via the public terraform AWS meshPlatform Module. The steps below are not needed if you decide to use it.

Set up AWS Account 1: meshcloud

The meshStack AWS Connector uses a dedicated set of IAM credentials to work with AWS APIs on behalf of meshStack. To create these credentials, create a user in IAM with these specifications:

  • User name: meshfed-service-user
  • AWS access type: Programmatic access - with an access key

This user should have the following policy attached to assume the respective roles in the management account and in the automation account. Replace the MANAGEMENT_ACCOUNT_ID, AUTOMATION_ACCOUNT_ID and EXTERNAL_ID placeholders accordingly.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<<MANAGEMENT_ACCOUNT_ID>>:role/MeshfedServiceRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "<<EXTERNAL_ID>>"
                }
            }
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<<AUTOMATION_ACCOUNT_ID>>:role/MeshfedAutomationRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "<<EXTERNAL_ID>>"
                }
            }
        }
    ]
}

Operators should generate a unique and random value for EXTERNAL_ID, e.g. a GUID. meshStack AWS Connector will supply this ExternalId only when accessing organization member accounts from a privileged (system) context. Using the ExternalId therefore increases the security of member accounts in your organization.

Operators need to securely inject the generated credentials and EXTERNAL_ID into the configuration of the AWS Connector.

Set up AWS Account 2: Management

Security Note: The demonstrated IAM Policies implement the minimum of configuration required to produce a working AWS integration using meshStack AWS Connector. This setup is based on the default AWS Organization configuration. We advise operators to determine the specific needs and requirements for their usage of AWS and implement more restrictive roles and policies.

This MeshfedServiceRole should be created in the management account with the following policy attached.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "StsAccessMemberAccount",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::*:role/MeshstackAccountAccessRole"
        },
        {
            "Sid": "OrgManagementAccess1",
            "Effect": "Allow",
            "Action": [
                "organizations:UntagResource",
                "organizations:TagResource",
                "organizations:MoveAccount",
                "organizations:ListTagsForResource",
                "organizations:ListParents",
                "organizations:ListOrganizationalUnitsForParent",
                "organizations:DescribeOrganizationalUnit",
                "organizations:DescribeAccount",
                "organizations:CreateOrganizationalUnit"
            ],
            "Resource": [
                "arn:aws:organizations::<<MANAGEMENT_ACCOUNT_ID>>:root/o-*/r-*",
                "arn:aws:organizations::*:ou/o-*/ou-*",
                "arn:aws:organizations::*:account/o-*/*"
            ]
        },
        {
            "Sid": "OrgManagementAccess2",
            "Effect": "Allow",
            "Action": [
                "organizations:ListRoots",
                "organizations:ListAccounts",
                "organizations:DescribeCreateAccountStatus",
                "organizations:CreateAccount"
            ],
            "Resource": "*"
        },
        {
            "Sid": "OrgManagementAccessSSO",
            "Effect": "Allow",
            "Action": [
                "sso:ListAccountAssignments",
                "sso:CreateAccountAssignment",
                "sso:DescribeAccountAssignmentCreationStatus"
            ],
            "Resource": [
              "<<AWS_SSO_INSTANCE_ARN>>",
              "arn:aws:sso:::permissionSet/*/*",
              "arn:aws:sso:::account/*"
            ]
        }
    ]
}

The following trust relationship needs to be attached to the MeshfedServiceRole so that the meshfed-service-user can assume the role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<<MESHCLOUD_ACCOUNT_ID>>:user/meshfed-service-user"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<<EXTERNAL_ID>>"
        }
      }
    }
  ]
}

Replace MESHCLOUD_ACCOUNT_ID with the dedicated meshcloud AWS account id where the meshfed-service-user lives. Replace EXTERNAL_ID accordingly as well.

For Enrollment with AWS Control Tower, the MeshfedServiceRole needs to have extra permissions to invoke the Account Factory. More information on that can be found at AWS Guide. To make use of these permissions, there must be an available launch path defined in AWS Service Catalog, as stated also in the prerequisites.

Set up AWS Account 3: Automation

The automation account should contain a MeshfedAutomationRole.

The following policy and trust relationship should be attached to the role so that meshfed-service-user can assume it in order to roll out CloudFormation stack instances in the newly provisioned accounts or to invoke a Lambda that would trigger account bootstrapping.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AutomationPolicy",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "cloudformation:UpdateStackInstances",
                "cloudformation:ListStackInstances",
                "cloudformation:DescribeStackSet",
                "cloudformation:CreateStackInstances"
            ],
            "Resource": "*"
        }
    ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
         "AWS": "arn:aws:iam::<<MESHCLOUD_ACCOUNT_ID>>:user/meshfed-service-user"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<<EXTERNAL_ID>>"
        }
      }
    }
  ]
}

In order to roll out CloudFormation Stack Instances in the newly provisioned accounts, create the AWSCloudFormationStackSetAdministrationRole as specified in the documentation and attach the following policy and trust relationship.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAssumeCloudFormationExecutionOnAllAccounts",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::*:role/AWSCloudFormationStackSetExecutionRole"
        }
    ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudformation.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Set up IAM

Currently meshStack supports 2 different ways of integrating AWS IAM with meshStack, either via AWS SSO or meshIdB (deprecated). The AWS SSO integration is the preferred integration as it allows using your company's central IdP to log in to AWS. This simplifies integration with meshStack, gives you more control over the AuthN part and improves UX for end-users when logging in to AWS.

Using AWS SSO (recommended)

The integration with AWS SSO basically works like this: AuthN is done via the company's IdP. Additionally users will be synced via AWS SSO Automated Provisioning (SCIM) to AWS SSO. meshStack takes care of AuthZ. That means meshStack will create groups for every project role on a meshTenant in AWS SSO. meshStack will assign the according users to these groups. As a last step, meshStack assigns the created groups to the respective AWS account with configured PermissionSets.

Details about what needs to be configured inside AWS SSO can be found here.

An important precondition, regarding the automated user provisioning to AWS SSO, is that the userName in AWS SSO has to be set to the euid. This limitation is caused by AWS SSO only allowing to filter userNames to find users. If an AAD is used as the IdP, that means the userPrincipalName in the AAD must be set to the euid, as AAD will always set the userName in AWS SSO to its userPrincipalName.

The following configuration options are available in the AWS Platform Connection Config:

AWS SSO Configuration

Using meshIdB (deprecated)

As AWS SSO is a rather new AWS feature, meshStack integrated IAM for AWS differently in the past. This AWS IAM integration should not be used for new integrations anymore. During replication meshStack configures meshIdB as an IdP within the managed account. Additionally according IAM roles are created during replication (dependent on configuration either my meshStack or by a CF template or a lambda function that are configured. in the Landing Zone). meshStack also sets up a trust relationship to meshIdB in order to allow SSO for the project users. meshStack additionally creates according roles in the meshIdB so the AuthZ information on which accounts can be accessed by which user are then part of the SAML token AWS receives after logging in via meshIdB.

The following configuration options are available in the AWS Platform Connection Config:

AWS meshIdB Configuration

Decide on Naming Patterns

You can define naming patterns based on the String Templating syntax of meshStack for the following properties:

  • Account Email Address: Please make sure to consider that this is limited to 64 characters
  • Account Alias Pattern: The account alias must be unique across all of AWS. Operators should therefore consider using a company-specific prefix together with a combination of meshCustomer and meshProject identifier. You can decide if you want to enforce setting the account alias on every replication via a flag in the configuration.

Identifier Configuration

meshStack operators that want to use AWS must configure their deployment to restrict identifier lengths to meet AWS requirements. The maximum allowed lengths are:

customer_identifier_length: 16
project_identifier_length: 30

Integrate AWS Control Tower

(also refer to AWS Management Account Setup)

A PlatformInstance can be configured to integrate with an existing AWS Control Tower setup. In order to manage accounts created by meshStack with AWS Control Tower, these need to be "enrolled". AWS Control Tower utilizes an Account Factory, to provide new accounts or enroll other accounts with it. AWS Control Tower will create an Account Factory Product in AWS Service Catalog, that will be invoked to enroll accounts. meshStack can be enabled to trigger the Account Factory via AWS Service Catalog. The correct Id of the Account Factory Product needs to be specified in the enrollment configuration besides the management account Id. meshStack will create new accounts as usual and in a later step will enroll them via the Account Factory with AWS Control Tower.

The following configuration options are available in the AWS Platform Connection Config:

AWS Control Tower Enrollment

In order to enroll created accounts with AWS Control Tower, a Landing Zone must be configured. The Target Organization Unit Id from the Landing Zone configuration must belong to a OU that is already enrolled with AWS Control Tower. Refer to Landing Zone Configuration for more information.

The following prerequisites must be fulfilled for the enrollment to work:

  1. A configured AWS Control Tower Service
  2. All AWS prerequisites
  3. AWS Service Catalog needs to have an AWS Control Tower Account Factory Portfolio
  4. The portfolio needs to contain an AWS Control Tower Account Factory Product
  5. The AWS Control Tower Account Factory Product needs to have at least one active Provisioned Artifact (active version)
  6. There needs to be a launch path with permissions to invoke the AWS Account Factory Product for meshStack access
  7. MeshfedServiceRole requires the attached policies AWSServiceCatalogEndUserFullAccess and AWSServiceCatalogAdminReadOnlyAccess and in addition some extra permissions. These are also described in the AWS Docs. We recommend attaching another custom policy to the MeshfedServiceRole that contains the following permissions:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AWSControlTowerAccountFactoryAccess",
      "Effect": "Allow",
      "Action": [
        "sso:GetProfile",
        "sso:CreateProfile",
        "sso:UpdateProfile",
        "sso:AssociateProfile",
        "sso:CreateApplicationInstance",
        "sso:GetSSOStatus",
        "sso:GetTrust",
        "sso:CreateTrust",
        "sso:UpdateTrust",
        "sso:GetPeregrineStatus",
        "sso:GetApplicationInstance",
        "sso:ListDirectoryAssociations",
        "sso:ListPermissionSets",
        "sso:GetPermissionSet",
        "sso:ProvisionApplicationInstanceForAWSAccount",
        "sso:ProvisionApplicationProfileForAWSAccountInstance",
        "sso:ProvisionSAMLProvider",
        "sso:ListProfileAssociations",
        "sso-directory:ListMembersInGroup",
        "sso-directory:AddMemberToGroup",
        "sso-directory:SearchGroups",
        "sso-directory:SearchGroupsWithGroupName",
        "sso-directory:SearchUsers",
        "sso-directory:CreateUser",
        "sso-directory:DescribeGroups",
        "sso-directory:DescribeDirectory",
        "sso-directory:GetUserPoolInfo",
        "controltower:CreateManagedAccount",
        "controltower:DescribeManagedAccount",
        "controltower:DeregisterManagedAccount",
        "s3:GetObject",
        "organizations:describeOrganization",
        "sso:DescribeRegisteredRegions"
      ],
      "Resource": "*"
    }
  ]
}

During the enrollment process, we create a new role in the tenant account that grants permissions to management account to perform AWS Control Tower execution steps. You have to make sure that there is no Service Control Policy (SCP) enabled in AWS Organizations that prevents that.

Set Minimum Access Rights on Provisioned Accounts

When provisioning a new account, a default role with administration privileges will be created in the new account. This role is by default named OrganizationAccountAccessRole but the name can be configured via meshStack. The MeshfedServiceRole will assume this newly created role in the provisioned account to perform tasks such as setting the account alias or setting up the user roles. meshStack has the capability to self downgrade this role's permissions to a minimum. This can be configured via the self-downgrade-access-role configuration flag. The downgraded role will have the following final policy attached (Note that the accountId and accessRole placeholders need to be populated with the value of the current account id and the configured access role name).

{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "iam:CreateSAMLProvider",
            "iam:GetSAMLProvider",
            "iam:UpdateSAMLProvider",
            "iam:DeleteSAMLProvider",
            "iam:ListSAMLProviders"
          ],
          "Resource": [
            "arn:aws:iam::{accountId}:saml-provider/*",
            "arn:aws:cloudformation:*:{accountId}:stack/meshstack-cf-access*"
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
            "iam:ListAttachedRolePolicies",
            "iam:CreateAccountAlias",
            "iam:ListAccountAliases",
            "iam:DeleteAccountAlias",
            "iam:GetRole",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:UpdateAssumeRolePolicy"
          ],
          "Resource": "*"
        },
        {
          "Effect": "Allow",
          "Action": [
            "cloudformation:DescribeStacks"
          ],
          "Resource": "*"
        },
        {
          "Effect": "Allow",
          "Action": [
            "iam:DetachRolePolicy",
            "iam:DeleteRolePolicy",
            "iam:PutRolePolicy",
            "iam:ListRolePolicies"
          ],
          "Resource": "arn:aws:iam::{accountId}:role/{accessRole}"
        }
      ]
    }

As you can see, meshStack has the permissions to upgrade this role if needed. This is to ensure that any new features added to the product will have the required permissions. Any newly added permissions will be notified via the product release notes.

If you prefer that meshStack does not have the capability to upgrade its own role, you can choose to implement your own role downgrade mechanism, for example via a Lambda call, which is also supported by meshLandingZones. In this case the role auto downgrade feature can be disabled.

If you would like to audit the actions taken by this role, you can enable AWS CloudTrail on all the accounts provisioned by meshStack by using an AWS CloudFormation StackSet, which is also supported by meshLandingZones. With the auditing enabled, it will always be possible to identify at which point in time meshStack added additional rights to its role. It will help to easily identify that meshStack was only able to do certain actions given the rights assigned at a certain point in time.

Set up IAM User for metering

graph LR; subgraph Organization Account meshfedServiceRole["MeshfedServiceRole"]; costExplorerServiceRole["MeshCostExplorerServiceRole"]; end subgraph meshcloud Account replicatorUser["ReplicatorUser & AccessKey"]; costExplorerUser["CostExplorerUser & AccessKey"]; end replicatorUser--Trusted Entity with External-id-->meshfedServiceRole; costExplorerUser--Trusted Entity with External-id-->costExplorerServiceRole; subgraph Automation Account meshfedAutomationRole["MeshfedAutomationRole"]; end replicatorUser--Trusted Entity with External-id-->meshfedAutomationRole

For the purpose of metering, meshStack requires the AWS Access Key and Secret Key of a user created in the meshcloud AWS account. A role should be created in the AWS management account which has the following policies attached (This role will be referred to as MeteringRole from now on).

  1. MeshCostExplorerServiceRole's Access Policy: This policy allows the Metering IAM user to call the AWS Cost Explorer API to read data required for metering. Note that Savings Plan and Reserved Instance related permissions are needed only if you have specific meshCustomers buying those directly, and you need to implement a cash-flow based Chargeback process for those. See Reserved Reserved Instances & Savings Plans Guide for more details.
  2. CostExplorerUser's Assume Role Policy: This policy allows CostExplorerUser IAM user to assume the IAM Role MeshCostExplorerServiceRole
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "ce:GetSavingsPlansUtilizationDetails",
                "ce:GetSavingsPlansUtilization",
                "ce:GetSavingsPlansCoverage",
                "ce:GetReservationUtilization",
                "ce:GetReservationCoverage",
                "ce:GetDimensionValues",
                "ce:GetCostAndUsage",
                "organizations:ListAccounts"
            ],
            "Resource": "*"
        }
    ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<<ORGANIZATION_ACCOUNT_ID>>:role/MeshCostExplorerServiceRole"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<<EXTERNAL_ID>>"
        }
      }
    }
  ]
}

Leverage Reserved Instances & Savings Plans

This section applies only if your meshCustomers pay you (the Cloud Foundation team), upfront to purchase Reserved Instances and Savings Plans directly on their AWS accounts, which give them priority for consuming the RI or SP. If this is the case, you can enable reservedInstanceFairChargeback and savingsPlanFairChargeback feature flags to achieve the following. See Reserved Reserved Instances & Savings Plans Guide for more details.

The upfront payments will be shown as line items in the tenant usage report for the month on which the Reserved Instance or Savings Plan starts.

meshStack also adds a discount which is equal to the amortized upfront cost to the relevant tenant usage report. This line item will be added to the report every month until the end of the Reserved Instances or Savings Plan period.

To achieve this, meshStack collects the information about Reserved Instances and Savings Plans via the cost explorer API.

Last updated on 3/21/2023
← How to integrate a meshPlatform into meshStackHow to manually integrate Azure as meshPlatform →