meshcloud

meshcloud

  • User Docs
  • Operator Docs
  • API Docs
  • Release Notes
  • Help

›Amazon Web Services

Architecture

  • Overview
  • Identity Federation
  • Platforms & Locations

Metering & Billing

  • Architecture
  • Configuration

System

  • Managed Service
  • Configuration
  • Authorization
  • User Revocation
  • Logging & Auditing
  • Metrics & Monitoring
  • meshStack APIs
  • Managing Tags
  • meshCustomer Group Synchronisation

Amazon Web Services

  • Integration
  • Landing Zones
  • Metering

Microsoft Azure

  • Integration Guide
  • Landing Zones
  • Metering
  • Cloud Inventory
  • Configuration Reference

Google Cloud Platform

  • Integration
  • Landing Zones
  • Metering

Cloud Foundry

  • Integration
  • Metering

Kubernetes

  • Integration
  • Metering

OpenShift

  • Integration
  • Landing Zones
  • Metering

OpenStack

  • Integration
  • VMware Integrated OpenStack
  • Metering

meshMarketplace

  • Integration
  • Marketplace Development
  • Metering
  • Tutorial: Implement a Broker
  • Tutorial: Dashboards
  • meshcloud OSB API Profile
  • Metrics-based Metering
Edit

Metering

meshStack imports metering data from AWS Cost and Usage Reports

An AWS Cost and Usage Report should be created as specified in the documentation with the following properties

  • Daily time granularity
  • gzip compression
  • txt/csv file format
  • Should include resource ids
  • Versioning should be setup to overwrite existing report

Note: If an S3 bucket is not already set up, it should be created as specified in AWS documentation

The following parameters are required to configure meshStack to process the AWS Cost and Usage Reports [More Details]

  • AWS credentials that can access the S3 bucket where the reports are written
  • The name of the S3 bucket where the report is written
  • The region of the S3 bucket
  • The name of the report
  • The "Report path prefix" configured when creating the report

When processing the AWS Cost and Usage Report to generate the Usage Report in the meshPanel,

  • You can configure which line item types should be considered in the calculations. You can also configure whether you want to consider discounts or not. We include the following columns in the calculations in order to come up with an amortized cost that should be charged to each account(the discount column is taken only if configured to consider discounts).
    1. If the line item type is RIFee, we take reservation/UnusedAmortizedUpfrontFeeForBillingPeriod, reservation/UnusedRecurringFee and discounts/TotalDiscount column
    2. If the line item type is SavingsPlanRecurringFee then we take the discounts/TotalDiscount column
    3. For other line item types, we take which ever is available from the columns savingsPlan/SavingsPlanEffectiveCost, reservation/EffectiveCost and lineItem/UnblendedCost in that order plus the discounts/TotalDiscount column
  • Only the line items with bill type Anniversary are taken into the calculation. In other words, line items with bill type Purchase and Refund are excluded.

IAM User Configuration

Policies

In order for meshStack to process AWS Cost and Usage Reports, following policies are required:

  1. Bucket Access Policy: This policy allows the Metering IAM user to fetch AWS cost and usage reports
  2. Organization Access Policy: This policy allows the Metering IAM user to list all accounts in the organization
  3. Assume Role Policy: This policy allows Metering IAM user to assume the IAM Role MeteringRole
Bucket Access Policy
Organization Access Policy
Assume Role Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-cur-bucket-name",
"arn:aws:s3:::my-cur-bucket-name/*"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "organizations:ListAccounts",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::123456789123:role/MeteringRole"
}
]
}

Role

The 3 policies in the previous section should be attached to this role.

In order for the Metering IAM user to assume this role, following trust policy is required:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789123:user/Metering"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

Configuration Reference

This section describes the configuration of a AWS Platform Instance in the meshStack configuration model at mesh.platforms for AWS metering.

Dhall Type
Example
let AwsPlatformKrakenConfiguration =
{ platform :
{- The combination of meshPlatform and meshLocation identifiers, i.e. <platform-id>.<location-id> -}
Text
, region :
{- The region of the IAM user -}
Text
, meshfed-service-user :
{-
Metering IAM user credentials
access-key: "<AWS_ACCESS_KEY>"
secret-key: "<AWS_SECRET_KEY>"
-}

{ access-key : Secret, secret-key : Secret }
, cost-and-usage-report :
{-
Parameters required to process AWS cost and usage reports
name: The name of the report
bucket-name: The name of the S3 bucket where the report is written
bucket-region: The region of the S3 bucket
report-key-prefix: The report path prefix.
Generated reports will be located at <bucket-name>/<report-key-prefix>
-}

{ name : Text
, bucket-name : Text
, bucket-region : Text
, report-key-prefix : Text
}
, organization-root-account-role :
{- The role assigned to the metering IAM user -}
Text
, organization-root-account-external-id :
{- The external ID passed to "sts:AssumeRole" when used in privileged context -}
Optional Text
, cost-item-write-batch-size :
{- Cost items written per batch -}
Natural
, max-report-file-processing-retries :
{- The maximum number of attempts to process or download a single file in a report -}
Natural
, report-file-process-retry-delay-millis :
{- Time in milliseconds before an operation (report file processing) is retried -}
Natural
, max-bulk-insert-retries :
{- The maximum number of attempts to process a batch of cost items to the database -}
Natural
, bulk-insert-retry-delay-millis :
{- Time in milliseconds before an operation (cost item batch processing) is retried -}
Natural
, cost-item-read-page-size :
{-
Cost items to read per query when generating tenant usage reports.
This setting is useful to prevent read timeout errors in the case of accounts with
a large number of cost items.
-}

Natural
, apply-discounts :
{-
Whether the discounts contained in the AWS Cost and Usage Reports should be applied
when generating the tenant usage reports or not
-}

Bool
, reported-line-item-types :
{-
What line item types should be included in the report. See https://docs.aws.amazon.com/cur/latest/userguide/Lineitem-columns.html#l-L
for a list of options. Note that if you remove items from this list, some manual cleanup is still needed to remove
the already collected data from the database.
-}

List Text
}
let example
: AwsPlatformKrakenConfiguration
= { platform = "aws.aws-location"
, region = "eu-central-1"
, meshfed-service-user =
{ access-key = Secret.Native "AWS_ACCESS_KEY_KRAKEN"
, secret-key = Secret.Native "AWS_SECRET_KEY_KRAKEN"
}
, cost-and-usage-report =
{ name = "my-cur-name"
, bucket-name = "my-cur-bucket-name"
, bucket-region = "eu-central-1"
, report-key-prefix = "/my-cur-prefix"
}
, organization-root-account-role =
"arn:aws:iam::123456789123:role/MeteringRole"
, organization-root-account-external-id = Some
"abcd1234-12ab-12ab-12ab-abcdef123456"
, cost-item-write-batch-size = 10000
, max-report-file-processing-retries = 5
, report-file-process-retry-delay-millis = 120000
, max-bulk-insert-retries = 5
, bulk-insert-retry-delay-millis = 5000
, cost-item-read-page-size = 2000
, apply-discounts = False
, reported-line-item-types =
[ "DiscountedUsage", "Fee", "Usage", "SavingsPlanCoveredUsage" ]
}

Configuring Seller Information

If you would like to see the AWS costs attributed to a seller in the chargeback statments, a product needs to be created manually in the meshcloud Product Catalog. This product should have a resourceType of sellerInfo and should be scoped to a platformType of Aws . An example is shown below.

{
    "resourceType": "sellerInfo",
    "displayName": "AWS Seller Info",
    "scope": {
        "platformType": "Aws",
        "location": null,
        "platformInstance": null,
        "localProjectId": null
    },
    "usageTypes": [],
    "description": "",
    "sellerId": "AWS",
    "sellerProductGroup": null,
    "@metadata": {
        "@collection": "Products",
        "Raven-Java-Type": "io.meshcloud.kraken.core.metering.Product"
    }
}
Last updated on 4/14/2021
← Landing ZonesIntegration Guide →
  • IAM User Configuration
    • Policies
    • Role
  • Configuration Reference
  • Configuring Seller Information
meshcloud
Docs
User DocumentationOperator Documentation
Community & Follow us
TwitterFacebookLinkedInXING
More
Release NotesGitHub
Copyright © 2021 meshcloud GmbH