Move all dashboards to GitOps (#175)

* Typo

* Remove Grafana provider

* Temp: move dashbaords to gitOps

* Move external labels to resource attributes

* Avoid DDoS with using 0.0.0.0

* Pre-commit

* Transition in two steps

Will need to remove provider in a separate version to provide a transition path as removing this will break terraform and leave orphans in the state

* Move patterns' dashboards creation to gitOps

Standardize config objects for patterns as well

* Pre-commit

* Create AMP dashboard from external source with Grafana provider

* Fix deprecated option

* Fix Flux requirements

* Run pre-commit

* Update example with operator

* Cleanup examples

* Update multicluster example

* Update multicluster example

* Drop dead variable

* Update docs

* Change GitOps branch name

* Update docs

* Replacing Secrets Manager to SSM to store Grafana API Key (#178)

* Fixing SSM

* Fixing SSM

* Replacing Secrets Manager with SSM

* Replacing Secrets Manager with SSM

* Update architecture diagram

* Update architecture diagram

* Update README.md

* Update index.md

* Fixing Grafana Operator Version

* Fix multicluster example

* Update docs

---------

Co-authored-by: Ela AWS <51791117+elamaran11@users.noreply.github.com>
Co-authored-by: Elamaran Shanmugam <elamaran.shan@gmail.com>

README.md

@@ -17,7 +17,8 @@ your custom applications.
 You also can monitor your Amazon Managed Service for Prometheus workspaces ingestion,
 costs, active series with [this module](./modules/managed-prometheus-monitoring).
 
-<img width="1501" alt="image" src="docs/images/dark-o11y-accelerator-amp-xray.png">
+![image](https://github.com/aws-observability/terraform-aws-observability-accelerator/assets/10175027/e83f8709-f754-4192-90f2-e3de96d2e26c)
+
 
 ## Documentation
 
@@ -33,15 +34,6 @@ visit the [Amazon EKS cluster monitoring documentation](https://aws-observabilit
 The sections below demonstrate how you can leverage AWS Observability Accelerator
 to enable monitoring to an existing EKS cluster.
 
-### v2.x changes
-
-v2+ releases introduces couple of breaking changes compared to previous versions:
-
-- `modules/workloads/infra` module moves to `modules/eks-monitoring`
-- All EKS configuration options moves from the base  module to the `eks-monitoring` module
-- All EKS workload modules `modules/workloads/{java,nginx}` merge into `eks-monitoring` as configuration options (patterns), see [examples](./examples) to provide a more complete visibility
-- All examples have been updated to reflect these changes
-- Introducing GitOps for Grafana contents (Dashboards, Folders and Data sources) with [Grafana Operator](https://github.com/grafana-operator/grafana-operator) and [Flux CD](https://fluxcd.io/)
 
 ### Base Module
 
@@ -161,14 +153,13 @@ If you are interested in contributing, see the
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
 | <a name="requirement_awscc"></a> [awscc](#requirement\_awscc) | >= 0.24.0 |
-| <a name="requirement_grafana"></a> [grafana](#requirement\_grafana) | 1.25.0 |
+| <a name="requirement_grafana"></a> [grafana](#requirement\_grafana) | >= 1.25.0 |
 
 ## Providers
 
 | Name | Version |
 |------|---------|
 | <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0.0 |
-| <a name="provider_grafana"></a> [grafana](#provider\_grafana) | 1.25.0 |
 
 ## Modules
 
@@ -180,8 +171,6 @@ No modules.
 |------|------|
 | [aws_prometheus_alert_manager_definition.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_alert_manager_definition) | resource |
 | [aws_prometheus_workspace.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_workspace) | resource |
-| [grafana_data_source.amp](https://registry.terraform.io/providers/grafana/grafana/1.25.0/docs/resources/data_source) | resource |
-| [grafana_folder.this](https://registry.terraform.io/providers/grafana/grafana/1.25.0/docs/resources/folder) | resource |
 | [aws_grafana_workspace.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/grafana_workspace) | data source |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
 
@@ -190,12 +179,10 @@ No modules.
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
-| <a name="input_create_dashboard_folder"></a> [create\_dashboard\_folder](#input\_create\_dashboard\_folder) | Boolean flag to enable Amazon Managed Grafana folder and dashboards | `bool` | `true` | no |
-| <a name="input_create_prometheus_data_source"></a> [create\_prometheus\_data\_source](#input\_create\_prometheus\_data\_source) | Boolean flag to enable Amazon Managed Grafana datasource | `bool` | `true` | no |
 | <a name="input_enable_alertmanager"></a> [enable\_alertmanager](#input\_enable\_alertmanager) | Creates Amazon Managed Service for Prometheus AlertManager for all workloads | `bool` | `false` | no |
 | <a name="input_enable_managed_prometheus"></a> [enable\_managed\_prometheus](#input\_enable\_managed\_prometheus) | Creates a new Amazon Managed Service for Prometheus Workspace | `bool` | `true` | no |
 | <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace | `string` | n/a | yes |
-| <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana Workspace ID | `string` | `""` | no |
+| <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana Workspace ID | `string` | n/a | yes |
 | <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Service for Prometheus Workspace ID | `string` | `""` | no |
 | <a name="input_managed_prometheus_workspace_region"></a> [managed\_prometheus\_workspace\_region](#input\_managed\_prometheus\_workspace\_region) | Region where Amazon Managed Service for Prometheus is deployed | `string` | `null` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
@@ -205,15 +192,10 @@ No modules.
 | Name | Description |
 |------|-------------|
 | <a name="output_aws_region"></a> [aws\_region](#output\_aws\_region) | AWS Region |
-| <a name="output_grafana_dashboard_folder_created"></a> [grafana\_dashboard\_folder\_created](#output\_grafana\_dashboard\_folder\_created) | Boolean value indicating if the module created a dashboard folder in Amazon Managed Grafana |
-| <a name="output_grafana_dashboards_folder_id"></a> [grafana\_dashboards\_folder\_id](#output\_grafana\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards. Required by workload modules |
-| <a name="output_grafana_prometheus_datasource_test"></a> [grafana\_prometheus\_datasource\_test](#output\_grafana\_prometheus\_datasource\_test) | Grafana save & test URL for Amazon Managed Prometheus workspace |
 | <a name="output_managed_grafana_workspace_endpoint"></a> [managed\_grafana\_workspace\_endpoint](#output\_managed\_grafana\_workspace\_endpoint) | Amazon Managed Grafana workspace endpoint |
-| <a name="output_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#output\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana workspace ID |
 | <a name="output_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#output\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus workspace endpoint |
 | <a name="output_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#output\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus workspace ID |
 | <a name="output_managed_prometheus_workspace_region"></a> [managed\_prometheus\_workspace\_region](#output\_managed\_prometheus\_workspace\_region) | Amazon Managed Prometheus workspace region |
-| <a name="output_prometheus_data_source_created"></a> [prometheus\_data\_source\_created](#output\_prometheus\_data\_source\_created) | Boolean value indicating if the module created a prometheus data source in Amazon Managed Grafana |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 
 ## Contributing

docs/concepts.md

@@ -39,22 +39,24 @@ The grafana-operator is a Kubernetes operator built to help you manage your Graf
 
 GitOps is a way of managing application and infrastructure deployment so that the whole system is described declaratively in a Git repository. It is an operational model that offers you the ability to manage the state of multiple Kubernetes clusters leveraging the best practices of version control, immutable artifacts, and automation. Flux  is a declarative, GitOps-based continuous delivery tool that can be integrated into any CI/CD pipeline. It gives users the flexibility of choosing their Git provider (GitHub, GitLab, BitBucket). Now, with grafana-operator supporting the management of external Grafana instances such as Amazon Managed Grafana, operations personas can use GitOps mechanisms using CNCF projects such as Flux to create and manage the lifecycle of resources in Amazon Managed Grafana.
 
-We have setup a [GitRepository](https://fluxcd.io/flux/components/source/gitrepositories/) and [Kustomization](https://fluxcd.io/flux/components/kustomize/kustomization/) using flux to sync our GitHub Repository to add Grafana Datasources, folder and Dashboards to Amazon Managed Grafana using Grafana Operator. GitRepository defines a Source to produce an Artifact for a Git repository revision. Kustomization defines a pipeline for fetching, decrypting, building, validating and applying Kustomize overlays or plain Kubernetes manifests. we are also using [Flux Post build variable substitution](https://fluxcd.io/flux/components/kustomize/kustomization/#post-build-variable-substitution) to dynamically render variables such as AMG_AWS_REGION, AMP_ENDPOINT_URL, AMG_ENDPOINT_URL,GRAFANA_NODEEXP_DASH_URL on the YAML manifests during deployment time to avoid hardcoding on the YAML manifests stored in Git repo.
+We have setup a [GitRepository](https://fluxcd.io/flux/components/source/gitrepositories/) and [Kustomization](https://fluxcd.io/flux/components/kustomize/kustomization/) using Flux to sync our GitHub Repository to add Grafana Datasources, folder and Dashboards to Amazon Managed Grafana using Grafana Operator. GitRepository defines a Source to produce an Artifact for a Git repository revision. Kustomization defines a pipeline for fetching, decrypting, building, validating and applying Kustomize overlays or plain Kubernetes manifests. we are also using [Flux Post build variable substitution](https://fluxcd.io/flux/components/kustomize/kustomization/#post-build-variable-substitution) to dynamically render variables such as AMG_AWS_REGION, AMP_ENDPOINT_URL, AMG_ENDPOINT_URL,GRAFANA_NODEEXP_DASH_URL on the YAML manifests during deployment time to avoid hardcoding on the YAML manifests stored in Git repo.
 
-We have placed our declarative code snippet to create an Amazon Managed Service For Promethes datasource and Grafana Dashboard in Amazon Managed Grafana in our [AWS Observabiity Accelerator GitHub Repository](https://github.com/aws-observability/aws-observability-accelerator/tree/main/artifacts/grafana-operator-manifests). We have setup a GitRepository to point to the AWS Observabiity Accelerator GitHub Repository and `Kustomization` for flux to sync Git Repository with artifacts in `./artifacts/grafana-operator-manifests` path in the AWS Observabiity Accelerator GitHub Repository. You can use this extension of our solution to point your own Kubernetes manifests to create Grafana Datasources and personified Grafana Dashboards of your choice using GitOps with Grafana Operator and Flux in Kubernetes native way with altering and redeploying this solution for changes to Grafana resources.
+We have placed our declarative code snippet to create an Amazon Managed Service For Promethes datasource and Grafana Dashboard in Amazon Managed Grafana in our [AWS Observabiity Accelerator GitHub Repository](https://github.com/aws-observability/aws-observability-accelerator). We have setup a GitRepository to point to the AWS Observabiity Accelerator GitHub Repository and `Kustomization` for flux to sync Git Repository with artifacts in `./artifacts/grafana-operator-manifests/*` path in the AWS Observabiity Accelerator GitHub Repository. You can use this extension of our solution to point your own Kubernetes manifests to create Grafana Datasources and personified Grafana Dashboards of your choice using GitOps with Grafana Operator and Flux in Kubernetes native way with altering and redeploying this solution for changes to Grafana resources.
 
 
 
-## v2.x changes
+## Release notes
 
-v2.x [releases](https://github.com/aws-observability/terraform-aws-observability-accelerator/releases) introduce
+We encourage you to use our [release versions](https://github.com/aws-observability/terraform-aws-observability-accelerator/releases)
-couple of breaking changes compared to previous versions:
+as much as possible to avoid breaking changes when deploying Terraform modules. You can
+read also our change log on the releases page. Here's an example of using a fixed version:
+
+```hcl
+module "eks_monitoring" {
+    source = "github.com/aws-observability/terraform-aws-observability-accelerator//modules/managed-prometheus-monitoring?ref=v2.5.0"
+}
+```
 
-- `modules/workloads/infra` module moves to `modules/eks-monitoring`
-- EKS configuration options moves from the base  module to the `eks-monitoring` module
-- EKS workload modules **java,nginx** merge into `eks-monitoring` as configuration options (patterns),
-see [examples](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/examples)
-- Examples have been updated to reflect these changes
 
 ## Base module
 
@@ -138,4 +140,4 @@ classDiagram
 
 If you are new to AWS Observability services, or want to dive deeper into them,
 check our [One Observability Workshop](https://catalog.workshops.aws/observability/)
-for a hands-on experience in a self-paced environement or at an AWS venue.
+for a hands-on experience in a self-paced environment or at an AWS venue.

docs/eks/index.md

@@ -111,26 +111,41 @@ terraform apply
 
 ## Visualization
 
-#### 1. Prometheus data source on Grafana
 
-Make sure to open the link in the output. After a successful deployment, this will open
+#### 1. Grafana dashboards
-the Prometheus data source configuration on Grafana.
-Click `Save & test` and you should see a notification confirming that the Amazon Managed Service for Prometheus workspace is ready to be used on Grafana.
-
-```bash
-terraform output grafana_prometheus_datasource_test
-```
-
-#### 2. Grafana dashboards
-
-Go to the Dashboards panel of your Grafana workspace. You should see a list of dashboards under the `Observability Accelerator Dashboards`
 
+Login to your Grafana workspace and navigate to the Dashboards panel. You should see a list of dashboards under the `Observability Accelerator Dashboards`
 <img width="1540" alt="image" src="https://user-images.githubusercontent.com/10175027/190000716-29e16698-7c90-49d6-8c37-79ca1790e2cc.png">
 
 Open a specific dashboard and you should be able to view its visualization
-
 <img width="2056" alt="cluster headlines" src="https://user-images.githubusercontent.com/10175027/199110753-9bc7a9b7-1b45-4598-89d3-32980154080e.png">
 
+With v2.5 and above, the dashboards are managed with a Grafana Operator running in your cluster.
+From the cluster to view all dashboards as Kubernetes objects, run
+
+```console
+kubectl get grafanadashboards -A
+NAMESPACE          NAME                                   AGE
+grafana-operator   cluster-grafanadashboard               138m
+grafana-operator   java-grafanadashboard                  143m
+grafana-operator   kubelet-grafanadashboard               13h
+grafana-operator   namespace-workloads-grafanadashboard   13h
+grafana-operator   nginx-grafanadashboard                 134m
+grafana-operator   node-exporter-grafanadashboard         13h
+grafana-operator   nodes-grafanadashboard                 13h
+grafana-operator   workloads-grafanadashboard             13h
+```
+
+You can inspect more details per dashboard using this command
+
+```console
+kubectl describe grafanadashboards cluster-grafanadashboard -n grafana-operator
+```
+
+Grafana Operator and Flux always work together to synchronize your dashboards with Git.
+If you delete your dashboards by accident, they will be re-provisioned automatically.
+
+
 #### 3. Amazon Managed Service for Prometheus rules and alerts
 
 Open the Amazon Managed Service for Prometheus console and view the details of your workspace. Under the `Rules management` tab, you should find new rules deployed.
@@ -216,21 +231,22 @@ export GO_AMG_API_KEY=$(aws grafana create-workspace-api-key \
   --output text)
 ```
 
-- Next, lets grab the Grafana API key secret name from AWS Secrets Manager. The keyname should start with `terraform-..`
-
-```bash
-aws secretsmanager list-secrets
-```
-
 - Finally, update the Grafana API key secret in AWS Secrets Manager using the above new Grafana API key:
 
 ```bash
-aws secretsmanager update-secret \
+aws aws ssm put-parameter \
-    --secret-id  <Your Secret Name> \
+    --name "/terraform-accelerator/grafana-api-key" \
-    --secret-string "{\"GF_SECURITY_ADMIN_APIKEY\": \"${GO_AMG_API_KEY}\"}" \
+    --type "SecureString" \
+    --value "{\"GF_SECURITY_ADMIN_APIKEY\": \"${GO_AMG_API_KEY}\"}" \
     --region <Your AWS Region>
 ```
 
+- If the issue persists, you can force the synchronization by deleting the `externalsecret` Kubernetes object.
+
+```bash
+kubectl delete externalsecret/external-secrets-sm -n grafana-operator
+```
+
 ### 2. Upgrade from 2.1.0 or earlier
 
 When you upgrade the eks-monitoring module from v2.1.0 or earlier, the following error may occur.

docs/eks/java.md

@@ -32,7 +32,7 @@ Make sure to refresh your temporary Grafana API key
 
 ```bash
 export TF_VAR_managed_grafana_workspace_id=g-xxx
-export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
+export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 7200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
 ```
 
 ## Deploy

docs/eks/multicluster.md

@@ -11,7 +11,7 @@ Using the example [eks-cluster-with-vpc](https://aws-observability.github.io/ter
    1. `eks-cluster-1`
    2. `eks-cluster-2`
 
-#### 2. Amazon Managed Serivce for Prometheus (AMP) workspace
+#### 2. Amazon Managed Service for Prometheus (AMP) workspace
 
 We recommend that you create a new AMP workspace. To do that you can run the following command.
 
@@ -48,7 +48,7 @@ Ensure you have the following necessary IAM permissions
 * `grafana.DeleteWorkspaceApiKey`
 
 ```sh
-export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
+export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 7200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
 ```
 
 ## Setup
@@ -70,8 +70,8 @@ Verify by looking at the file `variables.tf` that there are two EKS clusters tar
 
 The difference in deployment between these clusters is that Terraform, when setting up the EKS cluster behind variable `eks_cluster_1_id` for observability, also sets up:
 
-* Dashboard folder and files in `AMG`
+* Dashboard folder and files in Amazon Managed Grafana
-* Prometheus and Java, alerting and recording rules in `AMP`
+* Prometheus and Java, alerting and recording rules in Amazon Managed Service for Prometheus
 
 !!! warning
     To override the defaults, create a `terraform.tfvars` and change the default values of the variables.

docs/index.md

@@ -14,7 +14,7 @@ your custom applications.
 You also can monitor your Amazon Managed Service for Prometheus workspaces ingestion,
 costs, active series with [this module](https://aws-observability.github.io/terraform-aws-observability-accelerator/workloads/managed-prometheus/).
 
-<img width="1501" alt="image" src="images/dark-o11y-accelerator-amp-xray.png">
+![image](https://github.com/aws-observability/terraform-aws-observability-accelerator/assets/10175027/e83f8709-f754-4192-90f2-e3de96d2e26c)
 
 ## Getting started
 
@@ -26,7 +26,7 @@ traces collection, dashboards and alerts for monitoring:
 - Java/JMX workloads (running on Amazon EKS)
 - Amazon Managed Service for Prometheus workspaces with Amazon CloudWatch
 - [Grafana Operator](https://github.com/grafana-operator/grafana-operator) and [Flux CD](https://fluxcd.io/) to manage Grafana contents (AWS data sources, Grafana Dashboards) with GitOps
-- External Secrets Operator to retrieve and Sync the Grafana API keys
+- External Secrets Operator to retrieve and sync the Grafana API keys
 
 These modules can be directly configured in your existing Terraform
 configurations or ready to be deployed in our packaged

examples/eks-multicluster/main.tf

@@ -3,8 +3,6 @@ module "aws_observability_accelerator" {
   aws_region                          = var.eks_cluster_1_region
   enable_managed_prometheus           = false
   enable_alertmanager                 = true
-  create_dashboard_folder             = true
-  create_prometheus_data_source       = true
   grafana_api_key                     = var.grafana_api_key
   managed_prometheus_workspace_region = null
   managed_prometheus_workspace_id     = var.managed_prometheus_workspace_id
@@ -24,23 +22,18 @@ module "eks_cluster_1_monitoring" {
 
   # This configuration section results in actions performed on AMG and AMP; and it needs to be done just once
   # And hence, this in performed in conjunction with the setup of the eks_cluster_1 EKS cluster
-  enable_dashboards      = true
+  enable_dashboards       = true
-  enable_alerting_rules  = true
+  enable_external_secrets = true
-  enable_recording_rules = true
+  enable_fluxcd           = true
+  enable_alerting_rules   = true
+  enable_recording_rules  = true
 
   grafana_api_key                       = var.grafana_api_key
-  dashboards_folder_id                  = module.aws_observability_accelerator.grafana_dashboards_folder_id
   managed_prometheus_workspace_id       = module.aws_observability_accelerator.managed_prometheus_workspace_id
   managed_prometheus_workspace_endpoint = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint
   managed_prometheus_workspace_region   = module.aws_observability_accelerator.managed_prometheus_workspace_region
   grafana_url                           = module.aws_observability_accelerator.managed_grafana_workspace_endpoint
 
-  java_config = {
-    enable_alerting_rules  = true
-    enable_recording_rules = true
-    scrape_sample_limit    = 1
-  }
-
   prometheus_config = {
     global_scrape_interval = "60s"
     global_scrape_timeout  = "15s"
@@ -68,23 +61,17 @@ module "eks_cluster_2_monitoring" {
 
   # Since the following were enabled in conjunction with the set up of the
   # eks_cluster_1 EKS cluster, we will skip them with the eks_cluster_2 EKS cluster
-  enable_dashboards      = false
+  enable_dashboards       = false
-  enable_alerting_rules  = false
+  enable_external_secrets = false
-  enable_recording_rules = false
+  enable_fluxcd           = false
+  enable_alerting_rules   = false
+  enable_recording_rules  = false
 
-  grafana_api_key                       = var.grafana_api_key
-  dashboards_folder_id                  = module.aws_observability_accelerator.grafana_dashboards_folder_id
   managed_prometheus_workspace_id       = module.aws_observability_accelerator.managed_prometheus_workspace_id
   managed_prometheus_workspace_endpoint = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint
   managed_prometheus_workspace_region   = module.aws_observability_accelerator.managed_prometheus_workspace_region
   grafana_url                           = module.aws_observability_accelerator.managed_grafana_workspace_endpoint
 
-  java_config = {
-    enable_alerting_rules  = false # addressed while setting up the eks_cluster_1 EKS cluster
-    enable_recording_rules = false # addressed while setting up the eks_cluster_1 EKS cluster
-    scrape_sample_limit    = 1
-  }
-
   prometheus_config = {
     global_scrape_interval = "60s"
     global_scrape_timeout  = "15s"

examples/existing-cluster-java/README.md

@@ -236,8 +236,6 @@ terraform destroy -var-file=terraform.tfvars
 | <a name="output_aws_region"></a> [aws\_region](#output\_aws\_region) | AWS Region |
 | <a name="output_eks_cluster_id"></a> [eks\_cluster\_id](#output\_eks\_cluster\_id) | EKS Cluster Id |
 | <a name="output_eks_cluster_version"></a> [eks\_cluster\_version](#output\_eks\_cluster\_version) | EKS Cluster version |
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
-| <a name="output_grafana_prometheus_datasource_test"></a> [grafana\_prometheus\_datasource\_test](#output\_grafana\_prometheus\_datasource\_test) | Grafana save & test URL for Amazon Managed Prometheus workspace |
 | <a name="output_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#output\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus workspace endpoint |
 | <a name="output_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#output\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus workspace ID |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/existing-cluster-java/main.tf

@@ -79,10 +79,8 @@ module "eks_monitoring" {
   eks_cluster_id = var.eks_cluster_id
 
   # control the publishing of dashboards by specifying the boolean value for the variable 'enable_dashboards', default is 'true'
-  # the intention to publish is overruled depending upon whether grafana dashboard folder is created by the observability accelerator
+  enable_dashboards = var.enable_dashboards
-  enable_dashboards = module.aws_observability_accelerator.grafana_dashboard_folder_created ? var.enable_dashboards : false
 
-  dashboards_folder_id            = module.aws_observability_accelerator.grafana_dashboards_folder_id
   managed_prometheus_workspace_id = module.aws_observability_accelerator.managed_prometheus_workspace_id
 
   managed_prometheus_workspace_endpoint = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint

examples/existing-cluster-java/outputs.tf

@@ -13,16 +13,6 @@ output "managed_prometheus_workspace_id" {
   value       = module.aws_observability_accelerator.managed_prometheus_workspace_id
 }
 
-output "grafana_dashboard_urls" {
-  description = "URLs for dashboards created"
-  value       = module.eks_monitoring.grafana_dashboard_urls
-}
-
-output "grafana_prometheus_datasource_test" {
-  description = "Grafana save & test URL for Amazon Managed Prometheus workspace"
-  value       = module.aws_observability_accelerator.grafana_prometheus_datasource_test
-}
-
 output "eks_cluster_version" {
   description = "EKS Cluster version"
   value       = module.eks_monitoring.eks_cluster_version

examples/existing-cluster-nginx/README.md

@@ -247,8 +247,6 @@ add this `managed_prometheus_region=xxx` and `managed_prometheus_workspace_id=ws
 | <a name="output_aws_region"></a> [aws\_region](#output\_aws\_region) | AWS Region |
 | <a name="output_eks_cluster_id"></a> [eks\_cluster\_id](#output\_eks\_cluster\_id) | EKS Cluster Id |
 | <a name="output_eks_cluster_version"></a> [eks\_cluster\_version](#output\_eks\_cluster\_version) | EKS Cluster version |
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
-| <a name="output_grafana_prometheus_datasource_test"></a> [grafana\_prometheus\_datasource\_test](#output\_grafana\_prometheus\_datasource\_test) | Grafana save & test URL for Amazon Managed Prometheus workspace |
 | <a name="output_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#output\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus workspace endpoint |
 | <a name="output_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#output\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus workspace ID |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/existing-cluster-nginx/main.tf

@@ -75,10 +75,8 @@ module "eks_monitoring" {
   grafana_url             = module.aws_observability_accelerator.managed_grafana_workspace_endpoint
 
   # control the publishing of dashboards by specifying the boolean value for the variable 'enable_dashboards', default is 'true'
-  # the intention to publish is overruled depending upon whether grafana dashboard folder is created by the observability accelerator
+  enable_dashboards = var.enable_dashboards
-  enable_dashboards = module.aws_observability_accelerator.grafana_dashboard_folder_created ? var.enable_dashboards : false
 
-  dashboards_folder_id            = module.aws_observability_accelerator.grafana_dashboards_folder_id
   managed_prometheus_workspace_id = module.aws_observability_accelerator.managed_prometheus_workspace_id
 
   managed_prometheus_workspace_endpoint = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint

examples/existing-cluster-nginx/outputs.tf

@@ -13,16 +13,6 @@ output "managed_prometheus_workspace_id" {
   value       = module.aws_observability_accelerator.managed_prometheus_workspace_id
 }
 
-output "grafana_dashboard_urls" {
-  description = "URLs for dashboards created"
-  value       = module.eks_monitoring.grafana_dashboard_urls
-}
-
-output "grafana_prometheus_datasource_test" {
-  description = "Grafana save & test URL for Amazon Managed Prometheus workspace"
-  value       = module.aws_observability_accelerator.grafana_prometheus_datasource_test
-}
-
 output "eks_cluster_version" {
   description = "EKS Cluster version"
   value       = module.eks_monitoring.eks_cluster_version

examples/existing-cluster-with-base-and-infra/README.md

@@ -52,7 +52,7 @@ View the full documentation for this example [here](https://aws-observability.gi
 |------|-------------|------|---------|:--------:|
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
 | <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | Name of the EKS cluster | `string` | `"eks-cluster-with-vpc"` | no |
-| <a name="input_enable_dashboards"></a> [enable\_dashboards](#input\_enable\_dashboards) | Enables or disables curated dashboards | `bool` | `true` | no |
+| <a name="input_enable_dashboards"></a> [enable\_dashboards](#input\_enable\_dashboards) | Enables or disables curated dashboards. Dashboards are managed by the Grafana Operator | `bool` | `true` | no |
 | <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | API key for authorizing the Grafana provider to make changes to Amazon Managed Grafana | `string` | n/a | yes |
 | <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana Workspace ID | `string` | n/a | yes |
 | <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Service for Prometheus Workspace ID | `string` | `""` | no |
@@ -64,9 +64,4 @@ View the full documentation for this example [here](https://aws-observability.gi
 | <a name="output_aws_region"></a> [aws\_region](#output\_aws\_region) | AWS Region |
 | <a name="output_eks_cluster_id"></a> [eks\_cluster\_id](#output\_eks\_cluster\_id) | EKS Cluster Id |
 | <a name="output_eks_cluster_version"></a> [eks\_cluster\_version](#output\_eks\_cluster\_version) | EKS Cluster version |
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
-| <a name="output_grafana_prometheus_datasource_test"></a> [grafana\_prometheus\_datasource\_test](#output\_grafana\_prometheus\_datasource\_test) | Grafana save & test URL for Amazon Managed Prometheus workspace |
-| <a name="output_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#output\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana workspace ID |
-| <a name="output_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#output\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus workspace endpoint |
-| <a name="output_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#output\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus workspace ID |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/existing-cluster-with-base-and-infra/main.tf

@@ -50,6 +50,8 @@ module "aws_observability_accelerator" {
   enable_alertmanager = true
 
   # reusing existing Amazon Managed Grafana workspace
+  # This is not needed anymore but kept here for a two step transition into
+  # removing the Terraform Grafana provider
   managed_grafana_workspace_id = var.managed_grafana_workspace_id
   grafana_api_key              = var.grafana_api_key
 
@@ -85,10 +87,8 @@ module "eks_monitoring" {
   grafana_url             = module.aws_observability_accelerator.managed_grafana_workspace_endpoint
 
   # control the publishing of dashboards by specifying the boolean value for the variable 'enable_dashboards', default is 'true'
-  # the intention to publish is overruled depending upon whether grafana dashboard folder is created by the observability accelerator
+  enable_dashboards = var.enable_dashboards
-  enable_dashboards = module.aws_observability_accelerator.grafana_dashboard_folder_created ? var.enable_dashboards : false
 
-  dashboards_folder_id            = module.aws_observability_accelerator.grafana_dashboards_folder_id
   managed_prometheus_workspace_id = module.aws_observability_accelerator.managed_prometheus_workspace_id
 
   managed_prometheus_workspace_endpoint = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint

examples/existing-cluster-with-base-and-infra/outputs.tf

@@ -3,30 +3,6 @@ output "aws_region" {
   value       = module.aws_observability_accelerator.aws_region
 }
 
-output "managed_prometheus_workspace_endpoint" {
-  description = "Amazon Managed Prometheus workspace endpoint"
-  value       = module.aws_observability_accelerator.managed_prometheus_workspace_endpoint
-}
-
-output "managed_prometheus_workspace_id" {
-  description = "Amazon Managed Prometheus workspace ID"
-  value       = module.aws_observability_accelerator.managed_prometheus_workspace_id
-}
-
-output "managed_grafana_workspace_id" {
-  description = "Amazon Managed Grafana workspace ID"
-  value       = module.aws_observability_accelerator.managed_grafana_workspace_id
-}
-
-output "grafana_dashboard_urls" {
-  description = "URLs for dashboards created"
-  value       = module.eks_monitoring.grafana_dashboard_urls
-}
-
-output "grafana_prometheus_datasource_test" {
-  description = "Grafana save & test URL for Amazon Managed Prometheus workspace"
-  value       = module.aws_observability_accelerator.grafana_prometheus_datasource_test
-}
 output "eks_cluster_version" {
   description = "EKS Cluster version"
   value       = module.eks_monitoring.eks_cluster_version

examples/existing-cluster-with-base-and-infra/variables.tf

@@ -27,7 +27,7 @@ variable "grafana_api_key" {
 }
 
 variable "enable_dashboards" {
-  description = "Enables or disables curated dashboards"
+  description = "Enables or disables curated dashboards. Dashboards are managed by the Grafana Operator"
   type        = bool
   default     = true
 }

locals.tf

@@ -4,6 +4,7 @@ data "aws_grafana_workspace" "this" {
   workspace_id = var.managed_grafana_workspace_id
 }
 
+
 locals {
   # if region is not passed, we assume the current one
   amp_ws_region   = coalesce(var.managed_prometheus_workspace_region, data.aws_region.current.name)
@@ -11,7 +12,6 @@ locals {
   amp_ws_endpoint = "https://aps-workspaces.${local.amp_ws_region}.amazonaws.com/workspaces/${local.amp_ws_id}/"
 
   amg_ws_endpoint = "https://${data.aws_grafana_workspace.this.endpoint}"
-  amg_ws_id       = var.managed_grafana_workspace_id
 
   name = "aws-observability-accelerator"
 }

main.tf

@@ -5,6 +5,11 @@ resource "aws_prometheus_workspace" "this" {
   tags  = var.tags
 }
 
+provider "grafana" {
+  url  = local.amg_ws_endpoint
+  auth = var.grafana_api_key
+}
+
 resource "aws_prometheus_alert_manager_definition" "this" {
   count = var.enable_alertmanager ? 1 : 0
 
@@ -18,28 +23,3 @@ alertmanager_config: |
       - name: 'default'
 EOF
 }
-
-provider "grafana" {
-  url  = local.amg_ws_endpoint
-  auth = var.grafana_api_key
-}
-
-resource "grafana_data_source" "amp" {
-  count      = var.create_prometheus_data_source ? 1 : 0
-  type       = "prometheus"
-  name       = local.name
-  is_default = true
-  url        = local.amp_ws_endpoint
-  json_data {
-    http_method     = "GET"
-    sigv4_auth      = true
-    sigv4_auth_type = "workspace-iam-role"
-    sigv4_region    = local.amp_ws_region
-  }
-}
-
-# dashboards
-resource "grafana_folder" "this" {
-  count = var.create_dashboard_folder ? 1 : 0
-  title = "Observability Accelerator Dashboards"
-}

modules/eks-monitoring/README.md

@@ -67,7 +67,6 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_custom_metrics_config"></a> [custom\_metrics\_config](#input\_custom\_metrics\_config) | Configuration object to enable custom metrics collection | <pre>object({<br>    ports = list(number)<br>    # paths = optional(list(string), ["/metrics"])<br>    # list of samples to be dropped by label prefix, ex: go_ -> discards go_.*<br>    dropped_series_prefixes = list(string)<br>  })</pre> | <pre>{<br>  "dropped_series_prefixes": [<br>    "unspecified"<br>  ],<br>  "ports": []<br>}</pre> | no |
-| <a name="input_dashboards_folder_id"></a> [dashboards\_folder\_id](#input\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards | `string` | n/a | yes |
 | <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
 | <a name="input_enable_alerting_rules"></a> [enable\_alerting\_rules](#input\_enable\_alerting\_rules) | Enables or disables Managed Prometheus alerting rules | `bool` | `true` | no |
 | <a name="input_enable_amazon_eks_adot"></a> [enable\_amazon\_eks\_adot](#input\_enable\_amazon\_eks\_adot) | Enables the ADOT Operator on the EKS Cluster | `bool` | `true` | no |
@@ -86,11 +85,12 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | <a name="input_enable_tracing"></a> [enable\_tracing](#input\_enable\_tracing) | (Experimental) Enables tracing with AWS X-Ray. This changes the deploy mode of the collector to daemon set. Requirement: adot add-on <= 0.58-build.0 | `bool` | `false` | no |
 | <a name="input_flux_config"></a> [flux\_config](#input\_flux\_config) | FluxCD configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "flux2",<br>  "helm_chart_version": "2.7.0",<br>  "helm_release_name": "observability-fluxcd-addon",<br>  "helm_repo_url": "https://fluxcd-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "flux-system"<br>}</pre> | no |
 | <a name="input_flux_gitrepository_branch"></a> [flux\_gitrepository\_branch](#input\_flux\_gitrepository\_branch) | Flux GitRepository Branch | `string` | `"main"` | no |
+| <a name="input_flux_gitrepository_name"></a> [flux\_gitrepository\_name](#input\_flux\_gitrepository\_name) | Flux GitRepository name | `string` | `"aws-observability-accelerator"` | no |
 | <a name="input_flux_gitrepository_url"></a> [flux\_gitrepository\_url](#input\_flux\_gitrepository\_url) | Flux GitRepository URL | `string` | `"https://github.com/aws-observability/aws-observability-accelerator"` | no |
-| <a name="input_flux_kustomization_path"></a> [flux\_kustomization\_path](#input\_flux\_kustomization\_path) | Flux Kustomization Path | `string` | `"./artifacts/grafana-operator-manifests"` | no |
+| <a name="input_flux_kustomization_name"></a> [flux\_kustomization\_name](#input\_flux\_kustomization\_name) | Flux Kustomization name | `string` | `"grafana-dashboards-infrastructure"` | no |
-| <a name="input_flux_name"></a> [flux\_name](#input\_flux\_name) | Flux GitRepository and Kustomization Name | `string` | `"grafana-dashboards"` | no |
+| <a name="input_flux_kustomization_path"></a> [flux\_kustomization\_path](#input\_flux\_kustomization\_path) | Flux Kustomization Path | `string` | `"./artifacts/grafana-operator-manifests/eks/infrastructure"` | no |
-| <a name="input_go_config"></a> [go\_config](#input\_go\_config) | Grafana Operator configuration | <pre>object({<br>    create_namespace   = bool<br>    helm_chart         = string<br>    helm_name          = string<br>    k8s_namespace      = string<br>    helm_release_name  = string<br>    helm_chart_version = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",<br>  "helm_chart_version": "v5.0.0-rc1",<br>  "helm_name": "grafana-operator",<br>  "helm_release_name": "grafana-operator",<br>  "k8s_namespace": "grafana-operator"<br>}</pre> | no |
+| <a name="input_go_config"></a> [go\_config](#input\_go\_config) | Grafana Operator configuration | <pre>object({<br>    create_namespace   = bool<br>    helm_chart         = string<br>    helm_name          = string<br>    k8s_namespace      = string<br>    helm_release_name  = string<br>    helm_chart_version = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",<br>  "helm_chart_version": "v5.0.0-rc3",<br>  "helm_name": "grafana-operator",<br>  "helm_release_name": "grafana-operator",<br>  "k8s_namespace": "grafana-operator"<br>}</pre> | no |
-| <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace | `string` | n/a | yes |
+| <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace | `string` | `""` | no |
 | <a name="input_grafana_cluster_dashboard_url"></a> [grafana\_cluster\_dashboard\_url](#input\_grafana\_cluster\_dashboard\_url) | Dashboard URL for Cluster Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json"` | no |
 | <a name="input_grafana_kubelet_dashboard_url"></a> [grafana\_kubelet\_dashboard\_url](#input\_grafana\_kubelet\_dashboard\_url) | Dashboard URL for Kubelet Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json"` | no |
 | <a name="input_grafana_namespace_workloads_dashboard_url"></a> [grafana\_namespace\_workloads\_dashboard\_url](#input\_grafana\_namespace\_workloads\_dashboard\_url) | Dashboard URL for Namespace Workloads Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json"` | no |
@@ -101,14 +101,14 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm Config for Prometheus | `any` | `{}` | no |
 | <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
 | <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
-| <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    scrape_sample_limit    = number<br>  })</pre> | <pre>{<br>  "enable_alerting_rules": true,<br>  "enable_recording_rules": true,<br>  "scrape_sample_limit": 1000<br>}</pre> | no |
+| <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    enable_dashboards      = bool<br>    scrape_sample_limit    = number<br><br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    grafana_dashboard_url = string<br><br>    prometheus_metrics_endpoint = string<br>  })</pre> | `null` | no |
 | <a name="input_ksm_config"></a> [ksm\_config](#input\_ksm\_config) | Kube State metrics configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br><br>    scrape_interval = string<br>    scrape_timeout  = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "kube-state-metrics",<br>  "helm_chart_version": "4.24.0",<br>  "helm_release_name": "kube-state-metrics",<br>  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "kube-system",<br>  "scrape_interval": "60s",<br>  "scrape_timeout": "15s"<br>}</pre> | no |
 | <a name="input_logs_config"></a> [logs\_config](#input\_logs\_config) | Configuration object for logs collection | <pre>object({<br>    cw_log_retention_days = number<br>  })</pre> | <pre>{<br>  "cw_log_retention_days": 90<br>}</pre> | no |
 | <a name="input_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#input\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus Workspace Endpoint | `string` | `""` | no |
 | <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |
 | <a name="input_managed_prometheus_workspace_region"></a> [managed\_prometheus\_workspace\_region](#input\_managed\_prometheus\_workspace\_region) | Amazon Managed Prometheus Workspace's Region | `string` | `null` | no |
 | <a name="input_ne_config"></a> [ne\_config](#input\_ne\_config) | Node exporter configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br><br>    scrape_interval = string<br>    scrape_timeout  = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "prometheus-node-exporter",<br>  "helm_chart_version": "4.14.0",<br>  "helm_release_name": "prometheus-node-exporter",<br>  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "prometheus-node-exporter",<br>  "scrape_interval": "60s",<br>  "scrape_timeout": "60s"<br>}</pre> | no |
-| <a name="input_nginx_config"></a> [nginx\_config](#input\_nginx\_config) | Configuration object for NGINX monitoring | <pre>object({<br>    enable_alerting_rules       = bool<br>    scrape_sample_limit         = number<br>    prometheus_metrics_endpoint = string<br>  })</pre> | <pre>{<br>  "enable_alerting_rules": true,<br>  "prometheus_metrics_endpoint": "metrics",<br>  "scrape_sample_limit": 1000<br>}</pre> | no |
+| <a name="input_nginx_config"></a> [nginx\_config](#input\_nginx\_config) | Configuration object for NGINX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    enable_dashboards      = bool<br>    scrape_sample_limit    = number<br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    grafana_dashboard_url = string<br><br>    prometheus_metrics_endpoint = string<br>  })</pre> | `null` | no |
 | <a name="input_prometheus_config"></a> [prometheus\_config](#input\_prometheus\_config) | Controls default values such as scrape interval, timeouts and ports globally | <pre>object({<br>    global_scrape_interval = string<br>    global_scrape_timeout  = string<br>  })</pre> | <pre>{<br>  "global_scrape_interval": "60s",<br>  "global_scrape_timeout": "15s"<br>}</pre> | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
 | <a name="input_target_secret_name"></a> [target\_secret\_name](#input\_target\_secret\_name) | Target secret in Kubernetes to store the Grafana API Key Secret | `string` | `"grafana-admin-credentials"` | no |
@@ -121,5 +121,4 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 |------|-------------|
 | <a name="output_eks_cluster_id"></a> [eks\_cluster\_id](#output\_eks\_cluster\_id) | EKS Cluster Id |
 | <a name="output_eks_cluster_version"></a> [eks\_cluster\_version](#output\_eks\_cluster\_version) | EKS Cluster version |
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

modules/eks-monitoring/add-ons/external-secrets/README.md

@@ -32,8 +32,7 @@ This deploys an EKS Cluster with the External Secrets Operator. The cluster is p
 |------|------|
 | [aws_iam_policy.cluster_secretstore](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
 | [aws_kms_key.secrets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
-| [aws_secretsmanager_secret.secret](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) | resource |
+| [aws_ssm_parameter.secret](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource |
-| [aws_secretsmanager_secret_version.secret](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) | resource |
 | [kubectl_manifest.cluster_secretstore](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 | [kubectl_manifest.secret](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

modules/eks-monitoring/add-ons/external-secrets/main.tf

@@ -36,12 +36,13 @@ resource "aws_iam_policy" "cluster_secretstore" {
     {
       "Effect": "Allow",
       "Action": [
-        "secretsmanager:GetResourcePolicy",
+        "ssm:DescribeParameters",
-        "secretsmanager:GetSecretValue",
+        "ssm:GetParameter",
-        "secretsmanager:DescribeSecret",
+        "ssm:GetParameters",
-        "secretsmanager:ListSecretVersionIds"
+        "ssm:GetParametersByPath",
+        "ssm:GetParameterHistory"
       ],
-      "Resource": "${aws_secretsmanager_secret.secret.arn}"
+      "Resource": "${aws_ssm_parameter.secret.arn}"
     },
     {
       "Effect": "Allow",
@@ -64,7 +65,7 @@ metadata:
 spec:
   provider:
     aws:
-      service: SecretsManager
+      service: ParameterStore
       region: ${data.aws_region.current.name}
       auth:
         jwt:
@@ -75,16 +76,15 @@ YAML
   depends_on = [module.external_secrets]
 }
 
-resource "aws_secretsmanager_secret" "secret" {
+resource "aws_ssm_parameter" "secret" {
-  recovery_window_in_days = 0
+  name        = "/terraform-accelerator/grafana-api-key"
-  kms_key_id              = aws_kms_key.secrets.arn
+  description = "SSM Secret to store grafana API Key"
-}
+  type        = "SecureString"
-
+  value = jsonencode({
-resource "aws_secretsmanager_secret_version" "secret" {
-  secret_id = aws_secretsmanager_secret.secret.id
-  secret_string = jsonencode({
     GF_SECURITY_ADMIN_APIKEY = var.grafana_api_key
   })
+  key_id    = aws_kms_key.secrets.id
+  overwrite = true
 }
 
 resource "kubectl_manifest" "secret" {
@@ -103,7 +103,7 @@ spec:
     name: ${var.target_secret_name}
   dataFrom:
   - extract:
-      key: ${aws_secretsmanager_secret.secret.name}
+      key: ${aws_ssm_parameter.secret.name}
 YAML
   depends_on = [module.external_secrets]
 }

modules/eks-monitoring/dashboards.tf

@@ -1,9 +1,11 @@
 resource "kubectl_manifest" "flux_gitrepository" {
-  yaml_body  = <<YAML
+  count = var.enable_dashboards ? 1 : 0
+
+  yaml_body = <<YAML
 apiVersion: source.toolkit.fluxcd.io/v1beta2
 kind: GitRepository
 metadata:
-  name: ${var.flux_name}
+  name: ${var.flux_gitrepository_name}
   namespace: flux-system
 spec:
   interval: 5m0s
@@ -11,9 +13,8 @@ spec:
   ref:
     branch: ${var.flux_gitrepository_branch}
 YAML
-  count      = var.enable_dashboards ? 1 : 0
-  depends_on = [module.external_secrets]
 
+  depends_on = [module.external_secrets]
 }
 
 resource "kubectl_manifest" "flux_kustomization" {
@@ -21,7 +22,7 @@ resource "kubectl_manifest" "flux_kustomization" {
 apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
 kind: Kustomization
 metadata:
-  name: ${var.flux_name}
+  name: ${var.flux_kustomization_name}
   namespace: flux-system
 spec:
   interval: 1m0s
@@ -29,7 +30,7 @@ spec:
   prune: true
   sourceRef:
     kind: GitRepository
-    name: ${var.flux_name}
+    name: ${var.flux_gitrepository_name}
   postBuild:
     substitute:
       AMG_AWS_REGION: ${var.managed_prometheus_workspace_region}

modules/eks-monitoring/locals.tf

@@ -29,4 +29,52 @@ locals {
     irsa_iam_role_path             = var.irsa_iam_role_path
     irsa_iam_permissions_boundary  = var.irsa_iam_permissions_boundary
   }
+
+  java_pattern_config = {
+    # disabled if options from module are disabled, by default
+    # can be overriden by providing a config
+    enable_alerting_rules  = var.enable_alerting_rules
+    enable_recording_rules = var.enable_recording_rules
+    enable_dashboards      = var.enable_dashboards # disable flux kustomization if dashboards are disabled
+
+    scrape_sample_limit = 1000
+
+    flux_gitrepository_name   = "aws-observability-accelerator"
+    flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
+    flux_gitrepository_branch = "main"
+    flux_kustomization_name   = "grafana-dashboards-java"
+    flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/java"
+
+    managed_prometheus_workspace_id       = var.managed_prometheus_workspace_id
+    managed_prometheus_workspace_region   = var.managed_prometheus_workspace_region
+    managed_prometheus_workspace_endpoint = var.managed_prometheus_workspace_endpoint
+    prometheus_metrics_endpoint           = "/metrics"
+
+    grafana_url           = var.grafana_url
+    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/java/default.json"
+  }
+
+  nginx_pattern_config = {
+    # disabled if options from module are disabled, by default
+    # can be overriden by providing a config
+    enable_alerting_rules  = var.enable_alerting_rules
+    enable_recording_rules = var.enable_recording_rules
+    enable_dashboards      = var.enable_dashboards
+
+    scrape_sample_limit = 1000
+
+    flux_gitrepository_name   = "aws-observability-accelerator"
+    flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
+    flux_gitrepository_branch = "main"
+    flux_kustomization_name   = "grafana-dashboards-nginx"
+    flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/nginx"
+
+    managed_prometheus_workspace_id       = var.managed_prometheus_workspace_id
+    managed_prometheus_workspace_region   = var.managed_prometheus_workspace_region
+    managed_prometheus_workspace_endpoint = var.managed_prometheus_workspace_endpoint
+    prometheus_metrics_endpoint           = "/metrics"
+
+    grafana_url           = var.grafana_url
+    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/nginx/nginx.json"
+  }
 }

modules/eks-monitoring/main.tf

@@ -148,7 +148,11 @@ module "helm_addon" {
     },
     {
       name  = "javaScrapeSampleLimit"
-      value = var.java_config.scrape_sample_limit
+      value = try(var.java_config.scrape_sample_limit, local.java_pattern_config.scrape_sample_limit)
+    },
+    {
+      name  = "javaPrometheusMetricsEndpoint"
+      value = try(var.java_config.prometheus_metrics_endpoint, local.java_pattern_config.prometheus_metrics_endpoint)
     },
     {
       name  = "enable_nginx"
@@ -156,12 +160,12 @@ module "helm_addon" {
     },
     {
       name  = "nginxScrapeSampleLimit"
-      value = var.nginx_config.scrape_sample_limit
+      value = try(var.nginx_config.scrape_sample_limit, local.nginx_pattern_config.scrape_sample_limit)
     },
     {
       name  = "nginxPrometheusMetricsEndpoint"
-      value = var.nginx_config.prometheus_metrics_endpoint
+      value = try(var.nginx_config.prometheus_metrics_endpoint, local.nginx_pattern_config.prometheus_metrics_endpoint)
-    }
+    },
   ]
 
   irsa_config = {
@@ -181,23 +185,18 @@ module "helm_addon" {
 }
 
 module "java_monitoring" {
-  source            = "./patterns/java"
+  source = "./patterns/java"
-  count             = var.enable_java ? 1 : 0
+  count  = var.enable_java ? 1 : 0
-  enable_dashboards = var.enable_dashboards
+
+  pattern_config = coalesce(var.java_config, local.java_pattern_config)
 
-  managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
-  enable_alerting_rules           = var.java_config.enable_alerting_rules
-  enable_recording_rules          = var.java_config.enable_recording_rules
-  dashboards_folder_id            = var.dashboards_folder_id
 }
 
 module "nginx_monitoring" {
   source = "./patterns/nginx"
   count  = var.enable_nginx ? 1 : 0
 
-  managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
+  pattern_config = coalesce(var.nginx_config, local.nginx_pattern_config)
-  enable_alerting_rules           = var.nginx_config.enable_alerting_rules
-  dashboards_folder_id            = var.dashboards_folder_id
 }
 
 module "fluentbit_logs" {

modules/eks-monitoring/otel-config/templates/opentelemetrycollector.yaml

@@ -1701,6 +1701,7 @@ spec:
             {{ if .Values.enableJava }}
             - job_name: 'kubernetes-java-jmx'
               sample_limit: {{ .Values.javaScrapeSampleLimit }}
+              metrics_path: {{ .Values.javaPrometheusMetricsEndpoint }}
               kubernetes_sd_configs:
                 - role: pod
               relabel_configs:
@@ -1733,7 +1734,7 @@ spec:
             {{ if .Values.enableNginx }}
             - job_name: 'kubernetes-nginx'
               sample_limit: {{ .Values.nginxScrapeSampleLimit }}
-              metrics_path: /{{ .Values.nginxPrometheusMetricsEndpoint }}
+              metrics_path: {{ .Values.nginxPrometheusMetricsEndpoint }}
               kubernetes_sd_configs:
                 - role: pod
               relabel_configs:

modules/eks-monitoring/outputs.tf

@@ -1,15 +1,3 @@
-output "grafana_dashboard_urls" {
-  value = [concat(
-    # grafana_dashboard.workloads[*].url,
-    # grafana_dashboard.nodes[*].url,
-    # grafana_dashboard.nsworkload[*].url,
-    # grafana_dashboard.kubelet[*].url,
-    # grafana_dashboard.cluster[*].url,
-    flatten(module.java_monitoring[*].grafana_dashboard_urls),
-    flatten(module.nginx_monitoring[*].grafana_dashboard_urls),
-  )]
-  description = "URLs for dashboards created"
-}
 output "eks_cluster_version" {
   description = "EKS Cluster version"
   value       = data.aws_eks_cluster.eks_cluster.version

modules/eks-monitoring/patterns/java/README.md

@@ -22,7 +22,7 @@ Provides monitoring for Java based workloads with the following resources:
 | Name | Version |
 |------|---------|
 | <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0.0 |
-| <a name="provider_grafana"></a> [grafana](#provider\_grafana) | >= 1.25.0 |
+| <a name="provider_kubectl"></a> [kubectl](#provider\_kubectl) | >= 1.14 |
 
 ## Modules
 
@@ -34,21 +34,15 @@ No modules.
 |------|------|
 | [aws_prometheus_rule_group_namespace.alerting_rules](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_rule_group_namespace) | resource |
 | [aws_prometheus_rule_group_namespace.recording_rules](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_rule_group_namespace) | resource |
-| [grafana_dashboard.this](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/dashboard) | resource |
+| [kubectl_manifest.flux_kustomization](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_dashboards_folder_id"></a> [dashboards\_folder\_id](#input\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards | `string` | n/a | yes |
+| <a name="input_pattern_config"></a> [pattern\_config](#input\_pattern\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    scrape_sample_limit    = number<br><br>    enable_recording_rules = bool<br><br>    enable_dashboards = bool<br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    managed_prometheus_workspace_id       = string<br>    managed_prometheus_workspace_region   = string<br>    managed_prometheus_workspace_endpoint = string<br><br>    grafana_url           = string<br>    grafana_dashboard_url = string<br>  })</pre> | n/a | yes |
-| <a name="input_enable_alerting_rules"></a> [enable\_alerting\_rules](#input\_enable\_alerting\_rules) | Enables or disables Managed Prometheus alerting rules | `bool` | `true` | no |
-| <a name="input_enable_dashboards"></a> [enable\_dashboards](#input\_enable\_dashboards) | Enables or disables curated dashboards | `bool` | `true` | no |
-| <a name="input_enable_recording_rules"></a> [enable\_recording\_rules](#input\_enable\_recording\_rules) | Enables or disables Managed Prometheus recording rules | `bool` | `true` | no |
-| <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |
 
 ## Outputs
 
-| Name | Description |
+No outputs.
-|------|-------------|
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

modules/eks-monitoring/patterns/java/main.tf

@@ -1,7 +1,8 @@
 resource "aws_prometheus_rule_group_namespace" "recording_rules" {
-  count        = var.enable_recording_rules ? 1 : 0
+  count = var.pattern_config.enable_recording_rules ? 1 : 0
+
   name         = "accelerator-java-rules"
-  workspace_id = var.managed_prometheus_workspace_id
+  workspace_id = var.pattern_config.managed_prometheus_workspace_id
   data         = <<EOF
 groups:
   - name: default-metric
@@ -12,10 +13,10 @@ EOF
 }
 
 resource "aws_prometheus_rule_group_namespace" "alerting_rules" {
-  count = var.enable_alerting_rules ? 1 : 0
+  count = var.pattern_config.enable_alerting_rules ? 1 : 0
 
   name         = "accelerator-java-alerting"
-  workspace_id = var.managed_prometheus_workspace_id
+  workspace_id = var.pattern_config.managed_prometheus_workspace_id
   data         = <<EOF
 groups:
   - name: default-alert
@@ -31,8 +32,27 @@ groups:
 EOF
 }
 
-resource "grafana_dashboard" "this" {
+resource "kubectl_manifest" "flux_kustomization" {
-  count       = var.enable_dashboards ? 1 : 0
+  count = var.pattern_config.enable_dashboards ? 1 : 0
-  folder      = var.dashboards_folder_id
+
-  config_json = file("${path.module}/dashboards/default.json")
+  yaml_body = <<YAML
+apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
+kind: Kustomization
+metadata:
+  name: ${var.pattern_config.flux_kustomization_name}
+  namespace: flux-system
+spec:
+  interval: 1m0s
+  path: ${var.pattern_config.flux_kustomization_path}
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: ${var.pattern_config.flux_gitrepository_name}
+  postBuild:
+    substitute:
+      AMG_AWS_REGION: ${var.pattern_config.managed_prometheus_workspace_region}
+      AMP_ENDPOINT_URL: ${var.pattern_config.managed_prometheus_workspace_endpoint}
+      AMG_ENDPOINT_URL: ${var.pattern_config.grafana_url}
+      GRAFANA_JAVA_JMX_DASH_URL: ${var.pattern_config.grafana_dashboard_url}
+YAML
 }

modules/eks-monitoring/patterns/java/outputs.tf

@@ -1,6 +0,0 @@
-output "grafana_dashboard_urls" {
-  value = [concat(
-    grafana_dashboard.this[*].url,
-  )]
-  description = "URLs for dashboards created"
-}

modules/eks-monitoring/patterns/java/variables.tf

@@ -1,28 +1,26 @@
-variable "enable_alerting_rules" {
+variable "pattern_config" {
-  description = "Enables or disables Managed Prometheus alerting rules"
+  description = "Configuration object for Java/JMX monitoring"
-  type        = bool
+  type = object({
-  default     = true
+    enable_alerting_rules  = bool
-}
+    enable_recording_rules = bool
+    scrape_sample_limit    = number
 
-variable "enable_recording_rules" {
+    enable_recording_rules = bool
-  description = "Enables or disables Managed Prometheus recording rules"
-  type        = bool
-  default     = true
-}
 
-variable "managed_prometheus_workspace_id" {
+    enable_dashboards = bool
-  description = "Amazon Managed Prometheus Workspace ID"
-  type        = string
-  default     = null
-}
 
-variable "dashboards_folder_id" {
+    flux_gitrepository_name   = string
-  description = "Grafana folder ID for automatic dashboards"
+    flux_gitrepository_url    = string
-  type        = string
+    flux_gitrepository_branch = string
-}
+    flux_kustomization_name   = string
+    flux_kustomization_path   = string
 
-variable "enable_dashboards" {
+    managed_prometheus_workspace_id       = string
-  description = "Enables or disables curated dashboards"
+    managed_prometheus_workspace_region   = string
-  type        = bool
+    managed_prometheus_workspace_endpoint = string
-  default     = true
+
+    grafana_url           = string
+    grafana_dashboard_url = string
+  })
+  nullable = false
 }

modules/eks-monitoring/patterns/nginx/README.md

@@ -15,6 +15,7 @@ It provides the following resources:
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
 | <a name="requirement_grafana"></a> [grafana](#requirement\_grafana) | >= 1.25.0 |
+| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 1.14 |
 | <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
 
 ## Providers
@@ -22,7 +23,7 @@ It provides the following resources:
 | Name | Version |
 |------|---------|
 | <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0.0 |
-| <a name="provider_grafana"></a> [grafana](#provider\_grafana) | >= 1.25.0 |
+| <a name="provider_kubectl"></a> [kubectl](#provider\_kubectl) | >= 1.14 |
 
 ## Modules
 
@@ -33,19 +34,15 @@ No modules.
 | Name | Type |
 |------|------|
 | [aws_prometheus_rule_group_namespace.alerting_rules](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_rule_group_namespace) | resource |
-| [grafana_dashboard.workloads](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/dashboard) | resource |
+| [kubectl_manifest.flux_kustomization](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_dashboards_folder_id"></a> [dashboards\_folder\_id](#input\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards | `string` | n/a | yes |
+| <a name="input_pattern_config"></a> [pattern\_config](#input\_pattern\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    scrape_sample_limit    = number<br><br>    enable_recording_rules = bool<br><br>    enable_dashboards = bool<br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    managed_prometheus_workspace_id       = string<br>    managed_prometheus_workspace_region   = string<br>    managed_prometheus_workspace_endpoint = string<br><br>    grafana_url           = string<br>    grafana_dashboard_url = string<br>  })</pre> | n/a | yes |
-| <a name="input_enable_alerting_rules"></a> [enable\_alerting\_rules](#input\_enable\_alerting\_rules) | Enables or disables Managed Prometheus alerting rules | `bool` | `true` | no |
-| <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |
 
 ## Outputs
 
-| Name | Description |
+No outputs.
-|------|-------------|
-| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

modules/eks-monitoring/patterns/nginx/main.tf

@@ -1,8 +1,8 @@
 resource "aws_prometheus_rule_group_namespace" "alerting_rules" {
-  count = var.enable_alerting_rules ? 1 : 0
+  count = var.pattern_config.enable_alerting_rules ? 1 : 0
 
   name         = "accelerator-nginx-alerting"
-  workspace_id = var.managed_prometheus_workspace_id
+  workspace_id = var.pattern_config.managed_prometheus_workspace_id
   data         = <<EOF
 groups:
     - name: Nginx-HTTP-4xx-error-rate
@@ -37,8 +37,29 @@ groups:
          description: "Nginx p99 latency is higher than 3 seconds\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"
 EOF
 }
-resource "grafana_dashboard" "workloads" {
+
-  folder      = var.dashboards_folder_id
+resource "kubectl_manifest" "flux_kustomization" {
-  config_json = file("${path.module}/dashboards/nginx.json")
+  count = var.pattern_config.enable_dashboards ? 1 : 0
+
+  yaml_body = <<YAML
+apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
+kind: Kustomization
+metadata:
+  name: ${var.pattern_config.flux_kustomization_name}
+  namespace: flux-system
+spec:
+  interval: 1m0s
+  path: ${var.pattern_config.flux_kustomization_path}
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: ${var.pattern_config.flux_gitrepository_name}
+  postBuild:
+    substitute:
+      AMG_AWS_REGION: ${var.pattern_config.managed_prometheus_workspace_region}
+      AMP_ENDPOINT_URL: ${var.pattern_config.managed_prometheus_workspace_endpoint}
+      AMG_ENDPOINT_URL: ${var.pattern_config.grafana_url}
+      GRAFANA_NGINX_DASH_URL: ${var.pattern_config.grafana_dashboard_url}
+YAML
 
 }

modules/eks-monitoring/patterns/nginx/outputs.tf

@@ -1,4 +0,0 @@
-output "grafana_dashboard_urls" {
-  value       = [concat(grafana_dashboard.workloads[*].url)]
-  description = "URLs for dashboards created"
-}

modules/eks-monitoring/patterns/nginx/variables.tf

@@ -1,16 +1,26 @@
-variable "managed_prometheus_workspace_id" {
+variable "pattern_config" {
-  description = "Amazon Managed Prometheus Workspace ID"
+  description = "Configuration object for Java/JMX monitoring"
-  type        = string
+  type = object({
-  default     = null
+    enable_alerting_rules  = bool
-}
+    enable_recording_rules = bool
+    scrape_sample_limit    = number
 
-variable "dashboards_folder_id" {
+    enable_recording_rules = bool
-  type        = string
-  description = "Grafana folder ID for automatic dashboards"
-}
 
-variable "enable_alerting_rules" {
+    enable_dashboards = bool
-  type        = bool
+
-  default     = true
+    flux_gitrepository_name   = string
-  description = "Enables or disables Managed Prometheus alerting rules"
+    flux_gitrepository_url    = string
+    flux_gitrepository_branch = string
+    flux_kustomization_name   = string
+    flux_kustomization_path   = string
+
+    managed_prometheus_workspace_id       = string
+    managed_prometheus_workspace_region   = string
+    managed_prometheus_workspace_endpoint = string
+
+    grafana_url           = string
+    grafana_dashboard_url = string
+  })
+  nullable = false
 }

modules/eks-monitoring/patterns/nginx/versions.tf

@@ -10,6 +10,10 @@ terraform {
       source  = "hashicorp/kubernetes"
       version = ">= 2.10"
     }
+    kubectl = {
+      source  = "gavinbunney/kubectl"
+      version = ">= 1.14"
+    }
     grafana = {
       source  = "grafana/grafana"
       version = ">= 1.25.0"

modules/eks-monitoring/variables.tf

@@ -51,11 +51,6 @@ variable "managed_prometheus_workspace_region" {
   default     = null
 }
 
-variable "dashboards_folder_id" {
-  description = "Grafana folder ID for automatic dashboards"
-  type        = string
-}
-
 variable "enable_alerting_rules" {
   description = "Enables or disables Managed Prometheus alerting rules"
   type        = bool
@@ -74,10 +69,16 @@ variable "enable_dashboards" {
   default     = true
 }
 
-variable "flux_name" {
+variable "flux_kustomization_name" {
-  description = "Flux GitRepository and Kustomization Name"
+  description = "Flux Kustomization name"
   type        = string
-  default     = "grafana-dashboards"
+  default     = "grafana-dashboards-infrastructure"
+}
+
+variable "flux_gitrepository_name" {
+  description = "Flux GitRepository name"
+  type        = string
+  default     = "aws-observability-accelerator"
 }
 
 variable "flux_gitrepository_url" {
@@ -95,7 +96,7 @@ variable "flux_gitrepository_branch" {
 variable "flux_kustomization_path" {
   description = "Flux Kustomization Path"
   type        = string
-  default     = "./artifacts/grafana-operator-manifests"
+  default     = "./artifacts/grafana-operator-manifests/eks/infrastructure"
 }
 
 variable "enable_kube_state_metrics" {
@@ -249,14 +250,23 @@ variable "java_config" {
   type = object({
     enable_alerting_rules  = bool
     enable_recording_rules = bool
+    enable_dashboards      = bool
     scrape_sample_limit    = number
+
+
+    flux_gitrepository_name   = string
+    flux_gitrepository_url    = string
+    flux_gitrepository_branch = string
+    flux_kustomization_name   = string
+    flux_kustomization_path   = string
+
+    grafana_dashboard_url = string
+
+    prometheus_metrics_endpoint = string
   })
 
-  default = {
+  # defaults are pre-computed in locals.tf, provide a full definition to override
-    enable_alerting_rules  = true
+  default = null
-    enable_recording_rules = true
-    scrape_sample_limit    = 1000
-  }
 }
 
 variable "enable_nginx" {
@@ -265,19 +275,28 @@ variable "enable_nginx" {
   default     = false
 }
 
+
 variable "nginx_config" {
   description = "Configuration object for NGINX monitoring"
   type = object({
-    enable_alerting_rules       = bool
+    enable_alerting_rules  = bool
-    scrape_sample_limit         = number
+    enable_recording_rules = bool
+    enable_dashboards      = bool
+    scrape_sample_limit    = number
+
+    flux_gitrepository_name   = string
+    flux_gitrepository_url    = string
+    flux_gitrepository_branch = string
+    flux_kustomization_name   = string
+    flux_kustomization_path   = string
+
+    grafana_dashboard_url = string
+
     prometheus_metrics_endpoint = string
   })
 
-  default = {
+  # defaults are pre-computed in locals.tf, provide a full definition to override
-    enable_alerting_rules       = true
+  default = null
-    scrape_sample_limit         = 1000
-    prometheus_metrics_endpoint = "metrics"
-  }
 }
 
 variable "enable_logs" {
@@ -353,7 +372,7 @@ variable "go_config" {
     helm_name          = "grafana-operator"
     k8s_namespace      = "grafana-operator"
     helm_release_name  = "grafana-operator"
-    helm_chart_version = "v5.0.0-rc1"
+    helm_chart_version = "v5.0.0-rc3"
   }
   nullable = false
 }
@@ -367,6 +386,7 @@ variable "enable_external_secrets" {
 variable "grafana_api_key" {
   description = "Grafana API key for the Amazon Managed Grafana workspace"
   type        = string
+  default     = ""
 }
 
 variable "grafana_url" {

modules/managed-prometheus-monitoring/dashboards/amp-dashboard.json

@@ -1,795 +0,0 @@
-{
-  "annotations": {
-    "list": [
-      {
-        "builtIn": 1,
-        "datasource": "-- Grafana --",
-        "enable": true,
-        "hide": true,
-        "iconColor": "rgba(0, 211, 255, 1)",
-        "name": "Annotations & Alerts",
-        "target": {
-          "limit": 100,
-          "matchAny": false,
-          "tags": [],
-          "type": "dashboard"
-        },
-        "type": "dashboard"
-      }
-    ]
-  },
-  "description": "Dashboard for Amazon Managed Prometheus",
-  "editable": true,
-  "fiscalYearStartMonth": 0,
-  "graphTooltip": 0,
-  "id": 51,
-  "iteration": 1666292684202,
-  "links": [],
-  "liveNow": false,
-  "panels": [
-    {
-      "gridPos": {
-        "h": 7,
-        "w": 5,
-        "x": 0,
-        "y": 0
-      },
-      "id": 16,
-      "options": {
-        "content": "# Ingestion Usage Metrics\n\nMetrics relating to ingestion usage of the AMP service",
-        "mode": "markdown"
-      },
-      "pluginVersion": "8.4.7",
-      "title": "Usage",
-      "type": "text"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "palette-classic"
-          },
-          "custom": {
-            "axisLabel": "",
-            "axisPlacement": "auto",
-            "barAlignment": 0,
-            "drawStyle": "line",
-            "fillOpacity": 0,
-            "gradientMode": "none",
-            "hideFrom": {
-              "legend": false,
-              "tooltip": false,
-              "viz": false
-            },
-            "lineInterpolation": "linear",
-            "lineWidth": 1,
-            "pointSize": 5,
-            "scaleDistribution": {
-              "type": "linear"
-            },
-            "showPoints": "auto",
-            "spanNulls": false,
-            "stacking": {
-              "group": "A",
-              "mode": "none"
-            },
-            "thresholdsStyle": {
-              "mode": "off"
-            }
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 7,
-        "w": 9,
-        "x": 5,
-        "y": 0
-      },
-      "id": 6,
-      "options": {
-        "legend": {
-          "calcs": [],
-          "displayMode": "list",
-          "placement": "bottom"
-        },
-        "tooltip": {
-          "mode": "single",
-          "sort": "none"
-        }
-      },
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT SUM(ResourceCount) FROM SCHEMA(\"AWS/Usage\", Class,Resource,ResourceId,Service,Type) WHERE Type = 'Resource' AND ResourceId = '$WorkspaceID' AND Resource = 'ActiveSeries' AND Service = 'Prometheus' AND Class = 'None'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Active Series Metrics",
-      "type": "timeseries"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "palette-classic"
-          },
-          "custom": {
-            "axisLabel": "",
-            "axisPlacement": "auto",
-            "barAlignment": 0,
-            "drawStyle": "line",
-            "fillOpacity": 0,
-            "gradientMode": "none",
-            "hideFrom": {
-              "legend": false,
-              "tooltip": false,
-              "viz": false
-            },
-            "lineInterpolation": "linear",
-            "lineWidth": 1,
-            "pointSize": 5,
-            "scaleDistribution": {
-              "type": "linear"
-            },
-            "showPoints": "auto",
-            "spanNulls": false,
-            "stacking": {
-              "group": "A",
-              "mode": "none"
-            },
-            "thresholdsStyle": {
-              "mode": "off"
-            }
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 7,
-        "w": 9,
-        "x": 14,
-        "y": 0
-      },
-      "id": 2,
-      "options": {
-        "legend": {
-          "calcs": [],
-          "displayMode": "list",
-          "placement": "bottom"
-        },
-        "tooltip": {
-          "mode": "single",
-          "sort": "none"
-        }
-      },
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT AVG(ResourceCount) FROM SCHEMA(\"AWS/Usage\", Class,Resource,ResourceId,Service,Type) WHERE Type = 'Resource' AND ResourceId = '$WorkspaceID' AND Resource = 'IngestionRate' AND Service = 'Prometheus' AND Class = 'None'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Workspace Ingestion Rate",
-      "type": "timeseries"
-    },
-    {
-      "gridPos": {
-        "h": 8,
-        "w": 5,
-        "x": 0,
-        "y": 7
-      },
-      "id": 22,
-      "options": {
-        "content": "# Billing\n\nContains information relating to the cost of AMP\n\n",
-        "mode": "markdown"
-      },
-      "pluginVersion": "8.4.7",
-      "title": "Billing",
-      "type": "text"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "palette-classic"
-          },
-          "custom": {
-            "axisLabel": "",
-            "axisPlacement": "auto",
-            "barAlignment": 0,
-            "drawStyle": "line",
-            "fillOpacity": 0,
-            "gradientMode": "none",
-            "hideFrom": {
-              "legend": false,
-              "tooltip": false,
-              "viz": false
-            },
-            "lineInterpolation": "linear",
-            "lineWidth": 1,
-            "pointSize": 5,
-            "scaleDistribution": {
-              "type": "linear"
-            },
-            "showPoints": "auto",
-            "spanNulls": false,
-            "stacking": {
-              "group": "A",
-              "mode": "none"
-            },
-            "thresholdsStyle": {
-              "mode": "off"
-            }
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 8,
-        "w": 18,
-        "x": 5,
-        "y": 7
-      },
-      "id": 24,
-      "options": {
-        "legend": {
-          "calcs": [],
-          "displayMode": "list",
-          "placement": "bottom"
-        },
-        "tooltip": {
-          "mode": "single",
-          "sort": "none"
-        }
-      },
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT SUM(EstimatedCharges) FROM SCHEMA(\"AWS/Billing\", Currency,ServiceName) WHERE ServiceName = 'AmazonPrometheus'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Sum of Estimated AMP Charges (total)",
-      "type": "timeseries"
-    },
-    {
-      "gridPos": {
-        "h": 9,
-        "w": 5,
-        "x": 0,
-        "y": 15
-      },
-      "id": 14,
-      "options": {
-        "content": "# Alert Usage Metrics\n\nMetrics associated with Alertmanager Alert Usage",
-        "mode": "markdown"
-      },
-      "pluginVersion": "8.4.7",
-      "title": "Alerts",
-      "type": "text"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 1
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 9,
-        "w": 4,
-        "x": 5,
-        "y": 15
-      },
-      "id": 4,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "textMode": "auto"
-      },
-      "pluginVersion": "8.4.7",
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT AVG(ResourceCount) FROM SCHEMA(\"AWS/Usage\", Class,Resource,ResourceId,Service,Type) WHERE Type = 'Resource' AND ResourceId = '$WorkspaceID' AND Resource = 'ActiveAlerts' AND Service = 'Prometheus' AND Class = 'None'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Active Alerts",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 1
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 9,
-        "w": 5,
-        "x": 9,
-        "y": 15
-      },
-      "id": 12,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "textMode": "auto"
-      },
-      "pluginVersion": "8.4.7",
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT AVG(AlertManagerNotificationsFailed) FROM SCHEMA(\"AWS/Prometheus\", Workspace) WHERE Workspace = '$WorkspaceID'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Alert Manager Notifications Failed",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 9,
-        "w": 4,
-        "x": 14,
-        "y": 15
-      },
-      "id": 10,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "textMode": "auto"
-      },
-      "pluginVersion": "8.4.7",
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT AVG(AlertManagerAlertsReceived) FROM SCHEMA(\"AWS/Prometheus\", Workspace) WHERE Workspace = '$WorkspaceID'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Alert Manager Alerts Received",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 9,
-        "w": 5,
-        "x": 18,
-        "y": 15
-      },
-      "id": 8,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "textMode": "auto"
-      },
-      "pluginVersion": "8.4.7",
-      "targets": [
-        {
-          "alias": "",
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "dimensions": {},
-          "expression": "SELECT AVG(ResourceCount) FROM SCHEMA(\"AWS/Usage\", Class,Resource,ResourceId,Service,Type) WHERE Type = 'Resource' AND ResourceId = '$WorkspaceID' AND Resource = 'SizeOfAlerts' AND Service = 'Prometheus' AND Class = 'None'",
-          "id": "",
-          "matchExact": true,
-          "metricEditorMode": 1,
-          "metricName": "",
-          "metricQueryType": 0,
-          "namespace": "",
-          "period": "",
-          "queryMode": "Metrics",
-          "refId": "A",
-          "region": "default",
-          "sqlExpression": "",
-          "statistic": "Average"
-        }
-      ],
-      "title": "Size of Alerts",
-      "type": "stat"
-    },
-    {
-      "gridPos": {
-        "h": 7,
-        "w": 5,
-        "x": 0,
-        "y": 24
-      },
-      "id": 20,
-      "options": {
-        "content": "# AMP Vended Logs\n\nLast 25 log events from AMP Vended Logs for alert and rule evaluation",
-        "mode": "markdown"
-      },
-      "pluginVersion": "8.4.7",
-      "title": "AMP Logs",
-      "type": "text"
-    },
-    {
-      "datasource": {
-        "type": "cloudwatch",
-        "uid": "$datasource"
-      },
-      "gridPos": {
-        "h": 7,
-        "w": 18,
-        "x": 5,
-        "y": 24
-      },
-      "id": 18,
-      "options": {
-        "dedupStrategy": "none",
-        "enableLogDetails": true,
-        "prettifyLogMessage": false,
-        "showCommonLabels": false,
-        "showLabels": false,
-        "showTime": false,
-        "sortOrder": "Descending",
-        "wrapLogMessage": false
-      },
-      "targets": [
-        {
-          "datasource": {
-            "type": "cloudwatch",
-            "uid": "$datasource"
-          },
-          "expression": "fields @timestamp, @message\n| sort @timestamp desc\n| limit 25",
-          "id": "",
-          "logGroupNames": [
-            "/aws/vendedlogs/amp"
-          ],
-          "namespace": "",
-          "queryMode": "Logs",
-          "refId": "A",
-          "region": "default",
-          "statsGroups": []
-        }
-      ],
-      "timeFrom": "6h",
-      "timeShift": "6h",
-      "title": "AMP Vended Logs",
-      "type": "logs"
-    }
-  ],
-  "refresh": "",
-  "schemaVersion": 35,
-  "style": "dark",
-  "tags": [],
-  "templating": {
-    "list": [
-      {
-        "current": {
-          "selected": true,
-          "text": [
-            "ws-e8b003eb-0528-4208-b31c-edf4598d5f66"
-          ],
-          "value": [
-            "ws-e8b003eb-0528-4208-b31c-edf4598d5f66"
-          ]
-        },
-        "datasource": {
-          "type": "cloudwatch",
-          "uid": "$datasource"
-        },
-        "definition": "dimension_values(default,AWS/Prometheus,RuleEvaluations,Workspace)",
-        "hide": 0,
-        "includeAll": false,
-        "multi": true,
-        "name": "WorkspaceID",
-        "options": [],
-        "query": "dimension_values(default,AWS/Prometheus,RuleEvaluations,Workspace)",
-        "refresh": 1,
-        "regex": "",
-        "skipUrlSync": false,
-        "sort": 0,
-        "type": "query"
-      },
-      {
-        "current": {
-          "selected": false,
-          "text": "Amazon CloudWatch us-west-2",
-          "value": "Amazon CloudWatch us-west-2"
-        },
-        "hide": 0,
-        "includeAll": false,
-        "multi": false,
-        "name": "datasource",
-        "options": [],
-        "query": "cloudwatch",
-        "refresh": 1,
-        "regex": "",
-        "skipUrlSync": false,
-        "type": "datasource"
-      }
-    ]
-  },
-  "time": {
-    "from": "now-6h",
-    "to": "now"
-  },
-  "timepicker": {},
-  "timezone": "",
-  "title": "AMP Accelerator Dashboard",
-  "uid": "",
-  "version": 1,
-  "weekStart": ""
-}

modules/managed-prometheus-monitoring/main.tf

@@ -14,17 +14,25 @@ resource "grafana_data_source" "cloudwatch" {
 
   # Giving priority to Managed Prometheus datasources
   is_default = false
-  json_data {
+  json_data_encoded = jsonencode({
     default_region  = var.aws_region
     sigv4_auth      = true
     sigv4_auth_type = "workspace-iam-role"
     sigv4_region    = var.aws_region
+  })
+}
+
+data "http" "dashboard" {
+  url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/a72787328e493c4628680487e3c885fc395d1c56/artifacts/grafana-dashboards/amp/amp-dashboard.json"
+
+  request_headers = {
+    Accept = "application/json"
   }
 }
 
 resource "grafana_dashboard" "this" {
   folder      = var.dashboards_folder_id
-  config_json = file("${path.module}/dashboards/amp-dashboard.json")
+  config_json = data.http.dashboard.response_body
 }
 
 module "billing" {

modules/managed-prometheus-monitoring/variables.tf

@@ -1,8 +1,3 @@
-variable "dashboards_folder_id" {
-  description = "Grafana folder ID for automatic dashboards"
-  type        = string
-}
-
 variable "aws_region" {
   description = "AWS Region"
   type        = string
@@ -24,3 +19,9 @@ variable "ingestion_rate_threshold" {
   type        = number
   default     = 70000
 }
+
+variable "dashboards_folder_id" {
+  description = "Grafana folder ID for automatic dashboards"
+  default     = "0"
+  type        = string
+}

modules/managed-prometheus-monitoring/versions.tf

@@ -6,9 +6,17 @@ terraform {
       source  = "hashicorp/aws"
       version = ">= 4.0.0"
     }
+    helm = {
+      source  = "hashicorp/helm"
+      version = ">= 2.4.1"
+    }
     grafana = {
       source  = "grafana/grafana"
       version = ">= 1.25.0"
     }
+    http = {
+      source  = "hashicorp/http"
+      version = ">= 3.3.0"
+    }
   }
 }

outputs.tf

@@ -22,28 +22,3 @@ output "managed_grafana_workspace_endpoint" {
   description = "Amazon Managed Grafana workspace endpoint"
   value       = local.amg_ws_endpoint
 }
-
-output "managed_grafana_workspace_id" {
-  description = "Amazon Managed Grafana workspace ID"
-  value       = local.amg_ws_id
-}
-
-output "grafana_dashboards_folder_id" {
-  description = "Grafana folder ID for automatic dashboards. Required by workload modules"
-  value       = var.create_dashboard_folder ? grafana_folder.this[0].id : ""
-}
-
-output "grafana_prometheus_datasource_test" {
-  description = "Grafana save & test URL for Amazon Managed Prometheus workspace"
-  value       = var.create_prometheus_data_source ? "${local.amg_ws_endpoint}/datasources/edit/${grafana_data_source.amp[0].uid}" : ""
-}
-
-output "grafana_dashboard_folder_created" {
-  description = "Boolean value indicating if the module created a dashboard folder in Amazon Managed Grafana"
-  value       = var.create_dashboard_folder
-}
-
-output "prometheus_data_source_created" {
-  description = "Boolean value indicating if the module created a prometheus data source in Amazon Managed Grafana"
-  value       = var.create_prometheus_data_source
-}

variables.tf

@@ -27,10 +27,10 @@ variable "enable_alertmanager" {
   default     = false
 }
 
-variable "managed_grafana_workspace_id" {
+variable "tags" {
-  description = "Amazon Managed Grafana Workspace ID"
+  description = "Additional tags (e.g. `map('BusinessUnit`,`XYZ`)"
-  type        = string
+  type        = map(string)
-  default     = ""
+  default     = {}
 }
 
 variable "grafana_api_key" {
@@ -38,20 +38,7 @@ variable "grafana_api_key" {
   type        = string
 }
 
-variable "create_prometheus_data_source" {
+variable "managed_grafana_workspace_id" {
-  description = "Boolean flag to enable Amazon Managed Grafana datasource"
+  description = "Amazon Managed Grafana Workspace ID"
-  type        = bool
+  type        = string
-  default     = true
-}
-
-variable "create_dashboard_folder" {
-  description = "Boolean flag to enable Amazon Managed Grafana folder and dashboards"
-  type        = bool
-  default     = true
-}
-
-variable "tags" {
-  description = "Additional tags (e.g. `map('BusinessUnit`,`XYZ`)"
-  type        = map(string)
-  default     = {}
 }

versions.tf

@@ -12,7 +12,7 @@ terraform {
     }
     grafana = {
       source  = "grafana/grafana"
-      version = "1.25.0"
+      version = ">= 1.25.0"
     }
   }
 }