Rodrigue Koffi
fa38a90efc
* Typo * Remove Grafana provider * Temp: move dashbaords to gitOps * Move external labels to resource attributes * Avoid DDoS with using 0.0.0.0 * Pre-commit * Transition in two steps Will need to remove provider in a separate version to provide a transition path as removing this will break terraform and leave orphans in the state * Move patterns' dashboards creation to gitOps Standardize config objects for patterns as well * Pre-commit * Create AMP dashboard from external source with Grafana provider * Fix deprecated option * Fix Flux requirements * Run pre-commit * Update example with operator * Cleanup examples * Update multicluster example * Update multicluster example * Drop dead variable * Update docs * Change GitOps branch name * Update docs * Replacing Secrets Manager to SSM to store Grafana API Key (#178) * Fixing SSM * Fixing SSM * Replacing Secrets Manager with SSM * Replacing Secrets Manager with SSM * Update architecture diagram * Update architecture diagram * Update README.md * Update index.md * Fixing Grafana Operator Version * Fix multicluster example * Update docs --------- Co-authored-by: Ela AWS <51791117+elamaran11@users.noreply.github.com> Co-authored-by: Elamaran Shanmugam <elamaran.shan@gmail.com>
3.6 KiB
AWS EKS Multicluster Observability
This example shows how to use the AWS Observability Accelerator, with more than one EKS cluster and verify the collected metrics from all the clusters in the dashboards of a common Amazon Managed Grafana workspace.
Prerequisites
1. EKS clusters
Using the example eks-cluster-with-vpc, create two EKS clusters with the names:
eks-cluster-1eks-cluster-2
2. Amazon Managed Service for Prometheus (AMP) workspace
We recommend that you create a new AMP workspace. To do that you can run the following command.
Ensure you have the following necessary IAM permissions
aps.CreateWorkspace
export TF_VAR_managed_prometheus_workspace_id=$(aws amp create-workspace --alias observability-accelerator --query='workspaceId' --output text)
3. Amazon Managed Grafana (AMG) workspace
To run this example you need an AMG workspace. If you have an existing workspace, create an environment variable as described below. To create a new workspace, visit our supporting example for managed Grafana.
!!! note
For the URL https://g-xyz.grafana-workspace.eu-central-1.amazonaws.com, the workspace ID would be g-xyz
export TF_VAR_managed_grafana_workspace_id=g-xxx
4. Grafana API Key
AMG provides a control plane API for generating Grafana API keys.
As a security best practice, we will provide to Terraform a short lived API key to
run the apply or destroy command.
Ensure you have the following necessary IAM permissions
grafana.CreateWorkspaceApiKeygrafana.DeleteWorkspaceApiKey
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 7200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
Setup
1. Download sources and initialize Terraform
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
cd terraform-aws-observability-accelerator/examples/eks-multicluster
terraform init
2. Deploy
Verify by looking at the file variables.tf that there are two EKS clusters targeted for deployment by the names/ids:
eks-cluster-1eks-cluster-2
The difference in deployment between these clusters is that Terraform, when setting up the EKS cluster behind variable eks_cluster_1_id for observability, also sets up:
- Dashboard folder and files in Amazon Managed Grafana
- Prometheus and Java, alerting and recording rules in Amazon Managed Service for Prometheus
!!! warning
To override the defaults, create a terraform.tfvars and change the default values of the variables.
Run the following command to deploy
terraform apply --auto-approve
Verifying Multicluster Observability
One you have successfully run the above setup, you should be able to see dashboards similar to the images shown below in Amazon Managed Grafana workspace.
Note how you are able to use the cluster dropdown to filter the dashboards to metrics collected from a specific EKS cluster.
Cleanup
To clean up entirely, run the following command:
terraform destroy --auto-approve