Amazon Managed Grafana setup (#133)

* Managed Grafana Workspace with Identity Centre Users (#83)

* update kuberenetes and instance type

* initial setup of managed grafana workspace and identity centre identities

* cleanup

* run precommit

* output grafana workspace ID

* add identity store id variable

* remove API key

* update outputs naming convention as per terraform guidelines

* update docs and add versions

* update variables type

* update naming conventions

* add managed policy arn for querying promethues

* update readme

* update workshop references to this

* add role arn type

* cleanup and simplification

* remove workshop

---------

Co-authored-by: charlie keegan <chakeega@amazon.com>
Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>

* Rename example

* Update grafana example and base module references

* Update example's reference

* Cleanup and docs ref

* Add docs

* Update docs

* TODO: add link after merge

* Update managed-grafana.md

---------

Co-authored-by: Charlie Keegan <91210223+charliekeeegan@users.noreply.github.com>
Co-authored-by: charlie keegan <chakeega@amazon.com>
Co-authored-by: Mark Beacom <7315957+mbeacom@users.noreply.github.com>

README.md

@@ -91,7 +91,7 @@ View all the configuration options in the module documentation below.
 ### Workload modules
 
 [Workloads modules](./modules) are provided, which essentially provide curated
-metrics collection, alerting rule and Grafana dashboards.
+metrics collection, alerting rules and Grafana dashboards.
 
 #### Infrastructure monitoring
 
@@ -151,9 +151,7 @@ If you are interested in contributing, see the [Contribution guide](https://gith
 
 ## Modules
 
-| Name | Source | Version |
+No modules.
-|------|--------|---------|
-| <a name="module_managed_grafana"></a> [managed\_grafana](#module\_managed\_grafana) | terraform-aws-modules/managed-service-grafana/aws | ~> 1.3 |
 
 ## Resources
 
@@ -172,7 +170,6 @@ If you are interested in contributing, see the [Contribution guide](https://gith
 |------|-------------|------|---------|:--------:|
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
 | <a name="input_enable_alertmanager"></a> [enable\_alertmanager](#input\_enable\_alertmanager) | Creates Amazon Managed Service for Prometheus AlertManager for all workloads | `bool` | `false` | no |
-| <a name="input_enable_managed_grafana"></a> [enable\_managed\_grafana](#input\_enable\_managed\_grafana) | Creates a new Amazon Managed Grafana Workspace | `bool` | `true` | no |
 | <a name="input_enable_managed_prometheus"></a> [enable\_managed\_prometheus](#input\_enable\_managed\_prometheus) | Creates a new Amazon Managed Service for Prometheus Workspace | `bool` | `true` | no |
 | <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace | `string` | n/a | yes |
 | <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana Workspace ID | `string` | `""` | no |

docs/eks/index.md

@@ -72,9 +72,9 @@ aws amp create-workspace --alias observability-accelerator --query '.workspaceId
 
 #### 5. Amazon Managed Grafana workspace
 
-To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, create an environment variable as described below.
+To run this example you need an Amazon Managed Grafana workspace. If you have
-To create a new workspace, visit our Amazon Managed Grafana [documentation](https://docs.aws.amazon.com/grafana/latest/userguide/getting-started-with-AMG.html).
+an existing workspace, create an environment variable as described below.
-Make sure to provide the workspace with Amazon Managed Service for Prometheus read permissions.
+To create a new workspace, visit our supporting example for Grafana.
 
 !!! note
     For the URL `https://g-xyz.grafana-workspace.eu-central-1.amazonaws.com`, the workspace ID would be `g-xyz`

docs/helpers/managed-grafana.md

@@ -0,0 +1,57 @@
+# Creating a new Amazon Managed Grafana Workspace
+
+This example creates an Amazon Managed Grafana Workspace with
+Amazon CloudWatch, AWS X-Ray and Amazon Managed Service for Prometheus
+datasources.
+
+The authentication method chosen for this example is with IAM Identity
+Center (former SSO). You can extend this example to add SAML.
+
+## Prerequisites
+
+!!! note
+    Make sure to complete the [prerequisites section](https://aws-observability.github.io/terraform-aws-observability-accelerator/concepts/#prerequisites) before proceeding.
+
+## Setup
+
+### 1. Download sources and initialize Terraform
+
+```
+git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
+cd examples/managed-grafana-workspace
+terraform init
+```
+
+### 2. AWS Region
+
+Specify the AWS Region where the resources will be deployed:
+
+```bash
+export TF_VAR_aws_region=xxx
+```
+
+## Deploy
+
+Simply run this command to deploy the example
+
+```bash
+terraform apply
+```
+
+## Authentication
+
+After apply, Terraform will output the Worksapce's URL, but you need to:
+
+- [Setup user(s)](https://docs.aws.amazon.com/singlesignon/latest/userguide/getting-started.html) in the IAM Identity Center (former SSO)
+- [Assign the user(s) to the workspace](https://docs.aws.amazon.com/grafana/latest/userguide/AMG-manage-users-and-groups-AMG.html) with proper permissions
+
+<img width="1936" alt="Screenshot 2023-03-19 at 12 04 45" src="https://user-images.githubusercontent.com/10175027/226172947-f8588ed3-3751-47c1-a3ed-fb4c2d4d847e.png">
+
+
+## Cleanup
+
+To clean up your environment, destroy the Terraform example by running
+
+```sh
+terraform destroy
+```

examples/existing-cluster-java/README.md

@@ -50,9 +50,11 @@ Add your cluster name for `eks_cluster_id="..."` to the `terraform.tfvars` or us
 
 4. Amazon Managed Grafana workspace
 
-To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, create an environment variable `export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+To run this example you need an Amazon Managed Grafana workspace. If you have
-To create a new one, visit our Amazon Managed Grafana [documentation](https://docs.aws.amazon.com/grafana/latest/userguide/getting-started-with-AMG.html).
+an existing workspace, create an environment variable
-Make sure to provide the workspace with Amazon Managed Service for Prometheus read permissions.
+`export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+
+To create a new one, visit [this example](../managed-grafana-workspace).
 
 > In the URL `https://g-xyz.grafana-workspace.eu-central-1.amazonaws.com`, the workspace ID would be `g-xyz`
 

examples/existing-cluster-java/main.tf

@@ -47,7 +47,6 @@ module "aws_observability_accelerator" {
   managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
 
   # reusing existing Amazon Managed Grafana workspace
-  enable_managed_grafana       = false
   managed_grafana_workspace_id = var.managed_grafana_workspace_id
   grafana_api_key              = var.grafana_api_key
 

examples/existing-cluster-nginx/README.md

@@ -49,9 +49,11 @@ Add your cluster name for `eks_cluster_id="..."` to the `terraform.tfvars` or us
 
 4. Amazon Managed Grafana workspace
 
-To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, create an environment variable `export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+To run this example you need an Amazon Managed Grafana workspace. If you have
-To create a new one, visit our Amazon Managed Grafana [documentation](https://docs.aws.amazon.com/grafana/latest/userguide/getting-started-with-AMG.html).
+an existing workspace, create an environment variable
-Make sure to provide the workspace with Amazon Managed Service for Prometheus read permissions.
+`export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+
+To create a new one, visit [this example](../managed-grafana-workspace).
 
 > In the URL `https://g-xyz.grafana-workspace.eu-central-1.amazonaws.com`, the workspace ID would be `g-xyz`
 

examples/existing-cluster-nginx/main.tf

@@ -47,7 +47,6 @@ module "aws_observability_accelerator" {
   managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
 
   # reusing existing Amazon Managed Grafana workspace
-  enable_managed_grafana       = false
   managed_grafana_workspace_id = var.managed_grafana_workspace_id
   grafana_api_key              = var.grafana_api_key
 

examples/existing-cluster-with-base-and-infra/README.md

@@ -47,9 +47,11 @@ Add your cluster name for `eks_cluster_id="..."` to the `terraform.tfvars` or us
 
 4. Amazon Managed Grafana workspace
 
-To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, create an environment variable `export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+To run this example you need an Amazon Managed Grafana workspace. If you have
-To create a new one, visit our Amazon Managed Grafana [documentation](https://docs.aws.amazon.com/grafana/latest/userguide/getting-started-with-AMG.html).
+an existing workspace, create an environment variable
-Make sure to provide the workspace with Amazon Managed Service for Prometheus read permissions.
+`export TF_VAR_managed_grafana_workspace_id=g-xxx`.
+
+To create a new one, visit [this example](../managed-grafana-workspace).
 
 > In the URL `https://g-xyz.grafana-workspace.eu-central-1.amazonaws.com`, the workspace ID would be `g-xyz`
 

examples/existing-cluster-with-base-and-infra/main.tf

@@ -50,7 +50,6 @@ module "aws_observability_accelerator" {
   enable_alertmanager = true
 
   # reusing existing Amazon Managed Grafana workspace
-  enable_managed_grafana       = false
   managed_grafana_workspace_id = var.managed_grafana_workspace_id
   grafana_api_key              = var.grafana_api_key
 

examples/managed-grafana-workspace/main.tf

@@ -0,0 +1,47 @@
+provider "aws" {
+  region = var.aws_region
+}
+
+locals {
+  name        = "aws-observability-accelerator"
+  description = "Amazon Managed Grafana workspace for ${local.name}"
+
+  tags = {
+    GithubRepo = "terraform-aws-observability-accelerator"
+    GithubOrg  = "aws-observability"
+  }
+}
+
+module "managed_grafana" {
+  source  = "terraform-aws-modules/managed-service-grafana/aws"
+  version = "1.8.0"
+
+  name                      = local.name
+  associate_license         = false
+  description               = local.description
+  account_access_type       = "CURRENT_ACCOUNT"
+  authentication_providers  = ["AWS_SSO"]
+  permission_type           = "SERVICE_MANAGED"
+  data_sources              = ["CLOUDWATCH", "PROMETHEUS", "XRAY"]
+  notification_destinations = ["SNS"]
+  stack_set_name            = local.name
+
+  configuration = jsonencode({
+    unifiedAlerting = {
+      enabled = true
+    }
+  })
+
+
+  # Workspace IAM role
+  create_iam_role                = true
+  iam_role_name                  = local.name
+  use_iam_role_name_prefix       = true
+  iam_role_description           = local.description
+  iam_role_path                  = "/grafana/"
+  iam_role_force_detach_policies = true
+  iam_role_max_session_duration  = 7200
+  iam_role_tags                  = local.tags
+
+  tags = local.tags
+}

examples/managed-grafana-workspace/outputs.tf

@@ -0,0 +1,14 @@
+output "grafana_workspace_endpoint" {
+  description = "Amazon Managed Grafana Workspace endpoint"
+  value       = "https://${module.managed_grafana.workspace_endpoint}"
+}
+
+output "grafana_workspace_id" {
+  description = "Amazon Managed Grafana Workspace ID"
+  value       = module.managed_grafana.workspace_id
+}
+
+output "grafana_workspace_iam_role_arn" {
+  description = "Amazon Managed Grafana Workspace's IAM Role ARN"
+  value       = module.managed_grafana.workspace_iam_role_arn
+}

examples/managed-grafana-workspace/readme.md

@@ -0,0 +1,48 @@
+# Amazon Managed Grafana Workspace Setup
+
+This example creates an Amazon Managed Grafana Workspace with
+Amazon CloudWatch, AWS X-Ray and Amazon Managed Service for Prometheus
+datasources
+
+The authentication method chosen for this example is with IAM Identity
+Center (former SSO). You can extend this example to add SAML.
+
+Step-by-step instructions available on our [docs site](https://aws-observability.github.io/terraform-aws-observability-accelerator/)
+under **Supporting Examples**
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_managed_grafana"></a> [managed\_grafana](#module\_managed\_grafana) | terraform-aws-modules/managed-service-grafana/aws | 1.8.0 |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_grafana_workspace_endpoint"></a> [grafana\_workspace\_endpoint](#output\_grafana\_workspace\_endpoint) | Amazon Managed Grafana Workspace endpoint |
+| <a name="output_grafana_workspace_iam_role_arn"></a> [grafana\_workspace\_iam\_role\_arn](#output\_grafana\_workspace\_iam\_role\_arn) | Amazon Managed Grafana Workspace's IAM Role ARN |
+| <a name="output_grafana_workspace_id"></a> [grafana\_workspace\_id](#output\_grafana\_workspace\_id) | Amazon Managed Grafana Workspace ID |
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/managed-grafana-workspace/variables.tf

@@ -0,0 +1,4 @@
+variable "aws_region" {
+  description = "AWS Region"
+  type        = string
+}

examples/managed-grafana-workspace/versions.tf

@@ -0,0 +1,10 @@
+terraform {
+  required_version = ">= 1.1.0"
+
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 4.0.0"
+    }
+  }
+}

examples/managed-prometheus-monitoring/main.tf

@@ -8,13 +8,12 @@ provider "grafana" {
 }
 
 data "aws_grafana_workspace" "this" {
-  count        = var.managed_grafana_workspace_id == "" ? 0 : 1
   workspace_id = var.managed_grafana_workspace_id
 }
 
 locals {
   region          = var.aws_region
-  amg_ws_endpoint = "https://${data.aws_grafana_workspace.this[0].endpoint}"
+  amg_ws_endpoint = "https://${data.aws_grafana_workspace.this.endpoint}"
 }
 
 resource "grafana_folder" "this" {

locals.tf

@@ -1,7 +1,6 @@
 data "aws_region" "current" {}
 
 data "aws_grafana_workspace" "this" {
-  count        = var.managed_grafana_workspace_id == "" ? 0 : 1
   workspace_id = var.managed_grafana_workspace_id
 }
 
@@ -11,10 +10,8 @@ locals {
   amp_ws_id       = var.enable_managed_prometheus ? aws_prometheus_workspace.this[0].id : var.managed_prometheus_workspace_id
   amp_ws_endpoint = "https://aps-workspaces.${local.amp_ws_region}.amazonaws.com/workspaces/${local.amp_ws_id}/"
 
-  # if grafana_workspace_id is supplied, we infer the endpoint from
+  amg_ws_endpoint = "https://${data.aws_grafana_workspace.this.endpoint}"
-  # computed region, else we create a new workspace
+  amg_ws_id       = var.managed_grafana_workspace_id
-  amg_ws_endpoint = var.managed_grafana_workspace_id == "" ? "https://${module.managed_grafana[0].workspace_endpoint}" : "https://${data.aws_grafana_workspace.this[0].endpoint}"
-  amg_ws_id       = var.managed_grafana_workspace_id == "" ? split(".", module.managed_grafana[0].workspace_endpoint)[0] : var.managed_grafana_workspace_id
 
   name = "aws-observability-accelerator"
 }

main.tf

@@ -19,20 +19,6 @@ alertmanager_config: |
 EOF
 }
 
-module "managed_grafana" {
-  count   = var.enable_managed_grafana ? 1 : 0
-  source  = "terraform-aws-modules/managed-service-grafana/aws"
-  version = "~> 1.3"
-
-  # Workspace
-  name              = local.name
-  stack_set_name    = local.name
-  data_sources      = ["PROMETHEUS"]
-  associate_license = false
-
-  tags = var.tags
-}
-
 provider "grafana" {
   url  = local.amg_ws_endpoint
   auth = var.grafana_api_key

mkdocs.yml

@@ -33,8 +33,8 @@ nav:
   - Monitoring Managed Service for Prometheus Workspaces: workloads/managed-prometheus.md
   - Supporting Examples:
       - EKS Cluster with VPC: helpers/new-eks-cluster.md
-        # - Amazon Managed Grafana setup: helpers/managed-grafana.md
+      - Amazon Managed Grafana setup: helpers/managed-grafana.md
-  - Support & feedback: support.md
+  - Support & Feedback: support.md
   - Contributors: contributors.md
 
 markdown_extensions:

variables.tf

@@ -27,17 +27,12 @@ variable "enable_alertmanager" {
   default     = false
 }
 
-variable "enable_managed_grafana" {
-  description = "Creates a new Amazon Managed Grafana Workspace"
-  type        = bool
-  default     = true
-}
-
 variable "managed_grafana_workspace_id" {
   description = "Amazon Managed Grafana Workspace ID"
   type        = string
   default     = ""
 }
+
 variable "grafana_api_key" {
   description = "Grafana API key for the Amazon Managed Grafana workspace"
   type        = string