mirror of
https://github.com/storytold/terraform-aws-observability-accelerator.git
synced 2026-10-09 00:09:43 +00:00
Docs (#69)
* working first run * removing core module dependencies * adding CW datasource * alarms MVP * readmes * Adding Screenshot * Adding billing note * adding billing module * Revert "adding billing module" This reverts commit 40d667e37db1036cd71a471ef2fde83ec02aaa13. reverting * adding billing module * Updating Screenshot * resolving feedback * removing unused modules * fmt * Support for tf 1.3.x * removing unused variables * support alarms for multiple workspaces * Updating Readme * docs first draft * indigo nav fix * Simplify docs * RUM White Logo * amp docs * removing billing docs * Change docs structure, reword infrastructure monitoring doc * Pre-commit fixes * drop dead code * Add concepts page * Update contributors * Update java * Update docs * Docs for workloads * Update docs site * typos * pre-commit fixes * Update pre-commit * Update pre-commit Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
This commit is contained in:
Kevin Lewin
@@ -0,0 +1,88 @@
|
||||
# Concepts
|
||||
|
||||
## Prerequisites
|
||||
|
||||
All examples in this repository require the following tools installed
|
||||
|
||||
1. [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
|
||||
2. [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
|
||||
3. [Kubectl](https://Kubernetes.io/docs/tasks/tools/)
|
||||
|
||||
### Minimum IAM Policy
|
||||
|
||||
To run the examples, you need a set of AWS IAM permissions. You can find an example of minimum
|
||||
permissions required [in this file](https://github.com/aws-observability/terraform-aws-observability-accelerator/blob/docs/docs/iam/min-iam-policy.json).
|
||||
|
||||
> **Note**: The policy resource is set as `*` to allow all resources, this is not a recommended practice.
|
||||
You should restrict instead to the ARNs when applicable.
|
||||
|
||||
### Terraform states and variables
|
||||
|
||||
By default, our examples are using local Terraform states. If you need
|
||||
your Terraform states to be saved remotely, on Amazon S3, visit the
|
||||
[terraform remote states](https://www.terraform.io/language/state/remote) documentation.
|
||||
|
||||
For simplicity, we use Terraform supported environment variables.
|
||||
You can also edit the `terraform.tfvars` files directly and deploy
|
||||
with `terraform apply -var-file=terraform.tfvars`. Terraform tfvars file can be useful if
|
||||
you need to track changes as part of a Git repository or CI/CD pipeline.
|
||||
|
||||
> **Note:** When using `tfvars` files, always be careful to not store and commit any secrets (keys, passwords, ...)
|
||||
|
||||
## Base module
|
||||
|
||||
The base module allows you to configure the AWS Observability services for your cluster and the AWS Distro for OpenTelemetry (ADOT) Operator as the signals collection mechanism.
|
||||
|
||||
Here is the minimum configuration to have a new Managed Grafana Workspace, Amazon Managed Service for Prometheus Workspace, ADOT Operator deployed for you and ready to receive your data.
|
||||
|
||||
```hcl
|
||||
module "eks_observability_accelerator" {
|
||||
source = "aws-observability/terraform-aws-observability-accelerator"
|
||||
aws_region = "eu-west-1"
|
||||
eks_cluster_id = "my-eks-cluster"
|
||||
}
|
||||
```
|
||||
|
||||
You can optionally reuse existing Workspaces to dissociate their lifecycle from the
|
||||
Terraform state.
|
||||
|
||||
```hcl
|
||||
module "eks_observability_accelerator" {
|
||||
source = "aws-observability/terraform-aws-observability-accelerator"
|
||||
aws_region = "eu-west-1"
|
||||
eks_cluster_id = "my-eks-cluster"
|
||||
|
||||
# prevents creation of a new Amazon Managed Prometheus workspace
|
||||
enable_managed_prometheus = false
|
||||
|
||||
# reusing existing Amazon Managed Prometheus Workspace
|
||||
managed_prometheus_workspace_id = "ws-abcd123..."
|
||||
|
||||
# prevents creation of a new Amazon Managed Grafana workspace
|
||||
enable_managed_grafana = false
|
||||
|
||||
managed_grafana_workspace_id = "g-abcdef123"
|
||||
grafana_api_key = var.grafana_api_key
|
||||
}
|
||||
```
|
||||
|
||||
View all the configuration options in the [module's documentation](https://github.com/aws-observability/terraform-aws-observability-accelerator#requirements)
|
||||
|
||||
## Workload modules
|
||||
|
||||
Workloads modules are focused Terraform modules provided in this repository. They essentially provide curated metrics collection, alerts and Grafana dashboards according to the use case. Most of those modules require the base module.
|
||||
|
||||
You can check the full workload modules list and their documentation [here](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads).
|
||||
|
||||
All the modules come with end-to-end deployable examples.
|
||||
|
||||
## Examples
|
||||
|
||||
[Examples](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/examples) put modules together in a ready to deploy terraform configuration as a starting point. With little to no configuration, you can run `terraform apply` and use the deployed resources on your AWS Account.
|
||||
|
||||
You can find **workload** examples like [Amazon EKS infrstructure monitoring](/terraform-aws-observability-accelerator/workloads/eks/) or [monitoring your Amazon Managed Service for Prometheus workspace](terraform-aws-observability-accelerator/workloads/managed-prometheus/) and more.
|
||||
|
||||
|
||||
## Getting started with AWS Observability services
|
||||
|
||||
If you are new to AWS Observability services, or want to dive deeper into them, check our [One Observability Workshop](https://catalog.workshops.aws/observability/) for a hands-on experience in a self-paced environement or at an AWS venue.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Contributors
|
||||
|
||||
The content on this site is maintained by the Solutions Architects from the
|
||||
AWS observability team with support from the AWS service teams and other
|
||||
volunteers from across the organization.
|
||||
|
||||
Our goal is to make it easier to use AWS Open Source Observability Services.
|
||||
|
||||
The core team include the following people:
|
||||
|
||||
* Abhi Khanna
|
||||
* Imaya Kumar Jagannathan
|
||||
* Jerome DECQ
|
||||
* Kevin Lewin
|
||||
* Michael Hausenblas
|
||||
* Munish Dabra
|
||||
* Ramesh Kumar Venkatraman
|
||||
* Rodrigue Koffi
|
||||
* Toshal Dudhwhala
|
||||
* Vara Bonthu
|
||||
* Vikram Venkataraman
|
||||
|
||||
We welcome the wider open source community and thank [those who contribute](https://github.com/aws-observability/terraform-aws-observability-accelerator/graphs/contributors)
|
||||
to this project.
|
||||
|
||||
Note that all information published on this site is available via the
|
||||
Apache 2.0 license.
|
||||
+159
@@ -0,0 +1,159 @@
|
||||
# Amazon EKS cluster monitoring
|
||||
|
||||
This example demonstrates how to monitor your Amazon Elastic Kubernetes Service
|
||||
(Amazon EKS) cluster with the Observability Accelerator's EKS
|
||||
[infrastructure module](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/infra).
|
||||
|
||||
Monitoring Amazon Elastic Kubernetes Service (Amazon EKS) has two categories:
|
||||
the control plane and the Amazon EKS nodes (with Kubernetes objects).
|
||||
The Amazon EKS control plane consists of control plane nodes that run the Kubernetes software,
|
||||
such as etcd and the Kubernetes API server. To read more on the components of an Amazon EKS cluster,
|
||||
please read the [service documentation](https://docs.aws.amazon.com/eks/latest/userguide/clusters.html).
|
||||
|
||||
The Amazon EKS infrastructure Terraform modules focuses on metrics collection to Amazon
|
||||
Managed Service for Prometheus using the [AWS Distro for OpenTelemetry Operator](https://docs.aws.amazon.com/eks/latest/userguide/opentelemetry.html) for Amazon EKS.
|
||||
Additionally, it provides default dashboards to get a comprehensible visibility on the nodes,
|
||||
namespaces, pods, and kubelet operations health. Finally, you get curated Prometheus recording rules
|
||||
and alerts to operate your cluster.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
|
||||
before proceeding.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Download sources and initialize Terraform
|
||||
|
||||
```
|
||||
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
|
||||
cd examples/existing-cluster-with-base-and-infra
|
||||
terraform init
|
||||
```
|
||||
|
||||
### 2. AWS Region
|
||||
|
||||
Specify the AWS Region where the resources will be deployed:
|
||||
|
||||
```bash
|
||||
export TF_VAR_aws_region=xxx
|
||||
```
|
||||
|
||||
### 3. Amazon EKS Cluster
|
||||
|
||||
To run this example, you need to provide your EKS cluster name. If you don't
|
||||
have a cluster ready, visit [this example](/terraform-aws-observability-accelerator/helpers/new-eks-cluster.md)
|
||||
first to create a new one.
|
||||
|
||||
Specify your cluster name:
|
||||
|
||||
```bash
|
||||
export TF_VAR_eks_cluster_id=xxx
|
||||
```
|
||||
|
||||
### 4. Amazon Managed Service for Prometheus workspace (optional)
|
||||
|
||||
By default, we create an Amazon Managed Service for Prometheus workspace for you.
|
||||
However, if you have an existing workspace you want to reuse, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_prometheus_workspace_id=ws-xxx
|
||||
```
|
||||
|
||||
To create a workspace outside of Terraform's state, simply run:
|
||||
|
||||
```bash
|
||||
aws amp create-workspace --alias observability-accelerator --query '.workspaceId' --output text
|
||||
```
|
||||
|
||||
### 5. Amazon Managed Grafana workspace
|
||||
|
||||
To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_grafana_workspace_id=g-xxx
|
||||
```
|
||||
|
||||
To create a new one, within this example's Terraform state (sharing the same lifecycle with all the
|
||||
other resources created by Terraform):
|
||||
|
||||
- Edit main.tf and set `enable_managed_grafana = true`
|
||||
- Run
|
||||
|
||||
```bash
|
||||
terraform init
|
||||
terraform apply -target "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
export TF_VAR_managed_grafana_workspace_id=$(terraform output --raw managed_grafana_workspace_id)
|
||||
```
|
||||
|
||||
### 6. Grafana API Key
|
||||
|
||||
Amazon Managed Grafana provides a control plane API for generating Grafana API keys.
|
||||
As a security best practice, we will provide to Terraform a short lived API key to
|
||||
run the `apply` or `destroy` command.
|
||||
|
||||
Ensure you have necessary IAM permissions (`CreateWorkspaceApiKey, DeleteWorkspaceApiKey`)
|
||||
|
||||
```bash
|
||||
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Simply run this command to deploy the example
|
||||
|
||||
```bash
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Visualization
|
||||
|
||||
1. Prometheus datasource on Grafana
|
||||
|
||||
Open your Grafana workspace and under Configuration -> Data sources, you should see `aws-observability-accelerator`. Open and click `Save & test`. You should see a notification confirming that the Amazon Managed Service for Prometheus workspace is ready to be used on Grafana.
|
||||
|
||||
2. Grafana dashboards
|
||||
|
||||
Go to the Dashboards panel of your Grafana workspace. You should see a list of dashboards under the `Observability Accelerator Dashboards`
|
||||
|
||||
<img width="1540" alt="image" src="https://user-images.githubusercontent.com/10175027/190000716-29e16698-7c90-49d6-8c37-79ca1790e2cc.png">
|
||||
|
||||
Open a specific dashboard and you should be able to view its visualization
|
||||
|
||||
<img width="2056" alt="cluster headlines" src="https://user-images.githubusercontent.com/10175027/199110753-9bc7a9b7-1b45-4598-89d3-32980154080e.png">
|
||||
|
||||
2. Amazon Managed Service for Prometheus rules and alerts
|
||||
|
||||
Open the Amazon Managed Service for Prometheus console and view the details of your workspace. Under the `Rules management` tab, you should find new rules deployed.
|
||||
|
||||
<img width="1629" alt="image" src="https://user-images.githubusercontent.com/10175027/189301297-4865e75d-2d71-434f-b5d0-9750b3533632.png">
|
||||
|
||||
|
||||
To setup your alert receiver, with Amazon SNS, follow [this documentation](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-alertmanager-receiver.html)
|
||||
|
||||
|
||||
## Destroy resources
|
||||
|
||||
If you leave this stack running, you will continue to incur charges. To remove all resources
|
||||
created by Terraform, [refresh your Grafana API key](#6-grafana-api-key) and run the command below.
|
||||
|
||||
Be careful, this command will removing everything created by Terraform. If you wish
|
||||
to keep your Amazon Managed Grafana or Amazon Managed Service for Prometheus workspaces. Remove them
|
||||
from your terraform state before running the destroy command.
|
||||
|
||||
```bash
|
||||
terraform destroy
|
||||
```
|
||||
|
||||
To remove resources from your Terraform state, run
|
||||
|
||||
```bash
|
||||
# grafana workspace
|
||||
terraform state rm "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
|
||||
# prometheus workspace
|
||||
terraform state rm "module.eks_observability_accelerator.aws_prometheus_workspace.this[0]"
|
||||
```
|
||||
|
||||
|
||||
> **Note:** To view all the features proposed by this module, visit the [module documentation](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/infra).
|
||||
@@ -0,0 +1,78 @@
|
||||
# Creating a new Amazon EKS cluster with VPC
|
||||
|
||||
> Note: This example is a subset from [this EKS Blueprint example](https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main/examples/eks-cluster-with-new-vpc)
|
||||
|
||||
This example deploys the following:
|
||||
|
||||
- New sample VPC, 3 Private Subnets and 3 Public Subnets
|
||||
- Internet gateway for Public Subnets and NAT Gateway for Private Subnets
|
||||
- EKS Cluster Control plane with one managed node group
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
|
||||
before proceeding.
|
||||
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Download sources and initialize Terraform
|
||||
|
||||
```
|
||||
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
|
||||
cd examples/eks-cluster-with-vpc/
|
||||
terraform init
|
||||
```
|
||||
|
||||
### 2. AWS Region
|
||||
|
||||
Specify the AWS Region where the resources will be deployed:
|
||||
|
||||
```bash
|
||||
export TF_VAR_aws_region=xxx
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Simply run this command to deploy the example
|
||||
|
||||
```bash
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Login to your cluster
|
||||
|
||||
EKS Cluster details can be extracted from terraform output or from AWS Console to get the name of cluster.
|
||||
Use the following commands in your local machine where you want to interact with your EKS Cluster.
|
||||
|
||||
### 1. Run `update-kubeconfig` command
|
||||
|
||||
`~/.kube/config` file gets updated with cluster details and certificate from the below command
|
||||
|
||||
aws eks --region <enter-your-region> update-kubeconfig --name <cluster-name>
|
||||
|
||||
### 2. List all the worker nodes by running the command below
|
||||
|
||||
kubectl get nodes
|
||||
|
||||
### 3. List all the pods running in `kube-system` namespace
|
||||
|
||||
kubectl get pods -n kube-system
|
||||
|
||||
## Cleanup
|
||||
|
||||
To clean up your environment, destroy the Terraform modules in reverse order.
|
||||
|
||||
Destroy the Kubernetes Add-ons, EKS cluster with Node groups and VPC
|
||||
|
||||
```sh
|
||||
terraform destroy -target="module.eks_blueprints_kubernetes_addons" -auto-approve
|
||||
terraform destroy -target="module.eks_blueprints" -auto-approve
|
||||
terraform destroy -target="module.vpc" -auto-approve
|
||||
```
|
||||
|
||||
Finally, destroy any additional resources that are not in the above modules
|
||||
|
||||
```sh
|
||||
terraform destroy -auto-approve
|
||||
```
|
||||
@@ -0,0 +1,200 @@
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"aps:CreateAlertManagerDefinition",
|
||||
"aps:CreateWorkspace",
|
||||
"aps:DeleteAlertManagerDefinition",
|
||||
"aps:DeleteWorkspace",
|
||||
"aps:DescribeAlertManagerDefinition",
|
||||
"aps:DescribeWorkspace",
|
||||
"aps:ListTagsForResource",
|
||||
"autoscaling:CreateAutoScalingGroup",
|
||||
"autoscaling:CreateOrUpdateTags",
|
||||
"autoscaling:DeleteAutoScalingGroup",
|
||||
"autoscaling:DeleteLifecycleHook",
|
||||
"autoscaling:DeleteTags",
|
||||
"autoscaling:DescribeAutoScalingGroups",
|
||||
"autoscaling:DescribeLifecycleHooks",
|
||||
"autoscaling:DescribeTags",
|
||||
"autoscaling:PutLifecycleHook",
|
||||
"autoscaling:SetInstanceProtection",
|
||||
"autoscaling:UpdateAutoScalingGroup",
|
||||
"ec2:AllocateAddress",
|
||||
"ec2:AssociateRouteTable",
|
||||
"ec2:AttachInternetGateway",
|
||||
"ec2:AuthorizeSecurityGroupEgress",
|
||||
"ec2:AuthorizeSecurityGroupIngress",
|
||||
"ec2:CreateEgressOnlyInternetGateway",
|
||||
"ec2:CreateInternetGateway",
|
||||
"ec2:CreateLaunchTemplate",
|
||||
"ec2:CreateNatGateway",
|
||||
"ec2:CreateNetworkAclEntry",
|
||||
"ec2:CreateRoute",
|
||||
"ec2:CreateRouteTable",
|
||||
"ec2:CreateSecurityGroup",
|
||||
"ec2:CreateSubnet",
|
||||
"ec2:CreateTags",
|
||||
"ec2:CreateVpc",
|
||||
"ec2:DeleteEgressOnlyInternetGateway",
|
||||
"ec2:DeleteInternetGateway",
|
||||
"ec2:DeleteLaunchTemplate",
|
||||
"ec2:DeleteNatGateway",
|
||||
"ec2:DeleteNetworkAclEntry",
|
||||
"ec2:DeleteRoute",
|
||||
"ec2:DeleteRouteTable",
|
||||
"ec2:DeleteSecurityGroup",
|
||||
"ec2:DeleteSubnet",
|
||||
"ec2:DeleteTags",
|
||||
"ec2:DeleteVpc",
|
||||
"ec2:DescribeAccountAttributes",
|
||||
"ec2:DescribeAddresses",
|
||||
"ec2:DescribeAvailabilityZones",
|
||||
"ec2:DescribeEgressOnlyInternetGateways",
|
||||
"ec2:DescribeImages",
|
||||
"ec2:DescribeInternetGateways",
|
||||
"ec2:DescribeLaunchTemplateVersions",
|
||||
"ec2:DescribeLaunchTemplates",
|
||||
"ec2:DescribeNatGateways",
|
||||
"ec2:DescribeNetworkAcls",
|
||||
"ec2:DescribeNetworkInterfaces",
|
||||
"ec2:DescribeRouteTables",
|
||||
"ec2:DescribeSecurityGroups",
|
||||
"ec2:DescribeSecurityGroupRules",
|
||||
"ec2:DescribeSubnets",
|
||||
"ec2:DescribeTags",
|
||||
"ec2:DescribeVpcAttribute",
|
||||
"ec2:DescribeVpcClassicLink",
|
||||
"ec2:DescribeVpcClassicLinkDnsSupport",
|
||||
"ec2:DescribeVpcs",
|
||||
"ec2:DetachInternetGateway",
|
||||
"ec2:DisassociateRouteTable",
|
||||
"ec2:ModifySubnetAttribute",
|
||||
"ec2:ModifyVpcAttribute",
|
||||
"ec2:ReleaseAddress",
|
||||
"ec2:RevokeSecurityGroupEgress",
|
||||
"ec2:RevokeSecurityGroupIngress",
|
||||
"eks:CreateAddon",
|
||||
"eks:CreateCluster",
|
||||
"eks:CreateFargateProfile",
|
||||
"eks:CreateNodegroup",
|
||||
"eks:DeleteAddon",
|
||||
"eks:DeleteCluster",
|
||||
"eks:DeleteFargateProfile",
|
||||
"eks:DeleteNodegroup",
|
||||
"eks:DescribeAddon",
|
||||
"eks:DescribeAddonVersions",
|
||||
"eks:DescribeCluster",
|
||||
"eks:DescribeFargateProfile",
|
||||
"eks:DescribeNodegroup",
|
||||
"eks:TagResource",
|
||||
"elasticfilesystem:CreateFileSystem",
|
||||
"elasticfilesystem:CreateMountTarget",
|
||||
"elasticfilesystem:DeleteFileSystem",
|
||||
"elasticfilesystem:DeleteMountTarget",
|
||||
"elasticfilesystem:DescribeFileSystems",
|
||||
"elasticfilesystem:DescribeLifecycleConfiguration",
|
||||
"elasticfilesystem:DescribeMountTargetSecurityGroups",
|
||||
"elasticfilesystem:DescribeMountTargets",
|
||||
"emr-containers:CreateVirtualCluster",
|
||||
"emr-containers:DeleteVirtualCluster",
|
||||
"emr-containers:DescribeVirtualCluster",
|
||||
"events:DeleteRule",
|
||||
"events:DescribeRule",
|
||||
"events:ListTagsForResource",
|
||||
"events:ListTargetsByRule",
|
||||
"events:PutRule",
|
||||
"events:PutTargets",
|
||||
"events:RemoveTargets",
|
||||
"iam:AddRoleToInstanceProfile",
|
||||
"iam:AttachRolePolicy",
|
||||
"iam:CreateInstanceProfile",
|
||||
"iam:CreateOpenIDConnectProvider",
|
||||
"iam:CreatePolicy",
|
||||
"iam:CreateRole",
|
||||
"iam:CreateServiceLinkedRole",
|
||||
"iam:DeleteInstanceProfile",
|
||||
"iam:DeleteOpenIDConnectProvider",
|
||||
"iam:DeletePolicy",
|
||||
"iam:DeleteRole",
|
||||
"iam:DetachRolePolicy",
|
||||
"iam:GetInstanceProfile",
|
||||
"iam:GetOpenIDConnectProvider",
|
||||
"iam:GetPolicy",
|
||||
"iam:GetPolicyVersion",
|
||||
"iam:GetRole",
|
||||
"iam:ListAttachedRolePolicies",
|
||||
"iam:ListInstanceProfilesForRole",
|
||||
"iam:ListPolicyVersions",
|
||||
"iam:ListRolePolicies",
|
||||
"iam:PassRole",
|
||||
"iam:RemoveRoleFromInstanceProfile",
|
||||
"iam:TagOpenIDConnectProvider",
|
||||
"iam:TagInstanceProfile",
|
||||
"iam:TagPolicy",
|
||||
"iam:TagRole",
|
||||
"iam:UpdateAssumeRolePolicy",
|
||||
"kms:CreateAlias",
|
||||
"kms:CreateKey",
|
||||
"kms:DeleteAlias",
|
||||
"kms:DescribeKey",
|
||||
"kms:EnableKeyRotation",
|
||||
"kms:GetKeyPolicy",
|
||||
"kms:GetKeyRotationStatus",
|
||||
"kms:ListAliases",
|
||||
"kms:ListResourceTags",
|
||||
"kms:PutKeyPolicy",
|
||||
"kms:ScheduleKeyDeletion",
|
||||
"kms:TagResource",
|
||||
"logs:CreateLogGroup",
|
||||
"logs:DeleteLogGroup",
|
||||
"logs:DescribeLogGroups",
|
||||
"logs:ListTagsLogGroup",
|
||||
"logs:PutRetentionPolicy",
|
||||
"s3:CreateBucket",
|
||||
"s3:DeleteBucket",
|
||||
"s3:DeleteBucketOwnershipControls",
|
||||
"s3:DeleteBucketPolicy",
|
||||
"s3:DeleteObject",
|
||||
"s3:GetAccelerateConfiguration",
|
||||
"s3:GetBucketAcl",
|
||||
"s3:GetBucketCORS",
|
||||
"s3:GetBucketLogging",
|
||||
"s3:GetBucketObjectLockConfiguration",
|
||||
"s3:GetBucketOwnershipControls",
|
||||
"s3:GetBucketPolicy",
|
||||
"s3:GetBucketPublicAccessBlock",
|
||||
"s3:GetBucketRequestPayment",
|
||||
"s3:GetBucketTagging",
|
||||
"s3:GetBucketVersioning",
|
||||
"s3:GetBucketWebsite",
|
||||
"s3:GetEncryptionConfiguration",
|
||||
"s3:GetLifecycleConfiguration",
|
||||
"s3:GetObject",
|
||||
"s3:GetObjectTagging",
|
||||
"s3:GetObjectVersion",
|
||||
"s3:GetReplicationConfiguration",
|
||||
"s3:ListAllMyBuckets",
|
||||
"s3:ListBucket",
|
||||
"s3:PutBucketAcl",
|
||||
"s3:PutBucketOwnershipControls",
|
||||
"s3:PutBucketPolicy",
|
||||
"s3:PutBucketPublicAccessBlock",
|
||||
"s3:PutBucketTagging",
|
||||
"s3:PutBucketVersioning",
|
||||
"s3:PutEncryptionConfiguration",
|
||||
"s3:PutObject",
|
||||
"secretsmanager:CreateSecret",
|
||||
"secretsmanager:DeleteSecret",
|
||||
"secretsmanager:DescribeSecret",
|
||||
"secretsmanager:GetResourcePolicy",
|
||||
"secretsmanager:GetSecretValue",
|
||||
"secretsmanager:PutSecretValue",
|
||||
"sts:GetCallerIdentity"
|
||||
],
|
||||
"Resource": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 392 B |
Binary file not shown.
|
After Width: | Height: | Size: 3.6 KiB |
+43
-6
@@ -1,19 +1,56 @@
|
||||
# AWS Observability Accelerator for Terraform
|
||||
|
||||
|
||||
Welcome to the AWS Observability Accelerator for Terraform!
|
||||
|
||||
Welcome to AWS Observability Accelerator for Terraform!
|
||||
The AWS Observability accelerator is a set of Terraform modules to help you
|
||||
configure Observability for your workloads and environemnts with AWS
|
||||
Observability services. This project proposes a core module to bootstrap
|
||||
your cluster with the AWS Distro for OpenTelemetry (ADOT) Operator for EKS,
|
||||
Amazon Managed Service for Prometheus, Amazon Managed Grafana.
|
||||
Additionally we have a set of workload modules to leverage curated ADOT
|
||||
collector configurations, Grafana dashboards, Prometheus recording rules and alerts.
|
||||
|
||||
<img width="1501" alt="image" src="https://user-images.githubusercontent.com/10175027/193913383-94aaf4e2-58c6-4779-935b-e40528e86c03.png">
|
||||
|
||||
## What is AWS Observability Accelerator for Terraform
|
||||
## Getting started
|
||||
|
||||
This project provides a set of Terraform modules to enable metrics collection,
|
||||
dashboards and alerts for monitoring:
|
||||
|
||||
## Examples
|
||||
- Amazon EKS clusters infrastructure
|
||||
- NGINX workloads (running on Amazon EKS)
|
||||
- Java/JMX workloads (running on Amazon EKS)
|
||||
- Amazon Managed Service for Prometheus workspaces with Amazon CloudWatch
|
||||
|
||||
These modules can be directly configured in your exisiting Terraform configurations or ready
|
||||
to be deployed in our packaged
|
||||
[examples](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/examples)
|
||||
|
||||
## Workshop
|
||||
We have supporting examples for quick setup such as:
|
||||
|
||||
- Creating an empty Amazon EKS cluster and a VPC
|
||||
- Creating and configure an Amazon Managed Grafana workspace with SSO
|
||||
|
||||
## Motivation
|
||||
|
||||
## What can I do with this Solution?
|
||||
To gain deep visibility into your workloads and environements, AWS proposes a
|
||||
set of secure, scalabale, highly available, production-grade managed open
|
||||
source services such as Amazon Managed Service for Prometheus, Amazon Managed
|
||||
Grafana and Amazon OpenSearch.
|
||||
|
||||
AWS customers have asked for best-practices and guidance to collect metrics, logs
|
||||
and traces from their containerized applications and microservices with ease of
|
||||
deployment. Customers can use the AWS Observability Accelerator to configure their
|
||||
metrics collection, leveraging [AWS Distro for OpenTelemetry](https://aws-otel.github.io/),
|
||||
to have opinionated dashoards and alerts available in only minutes.
|
||||
|
||||
|
||||
## Support & Feedback
|
||||
|
||||
AWS Observability Accelerator for Terraform is maintained by AWS Solution Architects.
|
||||
It is not part of an AWS service and support is provided best-effort by the
|
||||
AWS Observability Accelerator community.
|
||||
|
||||
To post feedback, submit feature ideas, or report bugs, please use the [issues](https://github.com/aws-observability/terraform-aws-observability-accelerator/issues) section of this GitHub repo.
|
||||
|
||||
If you are interested in contributing, see the [contribution guide](https://github.com/aws-observability/terraform-aws-observability-accelerator/blob/main/CONTRIBUTING.md).
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block extrahead %}
|
||||
|
||||
<script>
|
||||
(function(n,i,v,r,s,c,x,z){x=window.AwsRumClient={q:[],n:n,i:i,v:v,r:r,c:c};window[n]=function(c,p){x.q.push({c:c,p:p});};z=document.createElement('script');z.async=true;z.src=s;document.head.insertBefore(z,document.head.getElementsByTagName('script')[0]);})(
|
||||
'cwr',
|
||||
'1244d427-de32-4423-b7fe-3d6903e95178',
|
||||
'1.0.0',
|
||||
'us-east-2',
|
||||
'https://client.rum.us-east-1.amazonaws.com/1.12.0/cwr.js',
|
||||
{
|
||||
sessionSampleRate: 1,
|
||||
guestRoleArn: "arn:aws:iam::147084596884:role/RUM-Monitor-us-east-2-147084596884-2189797490761-Unauth",
|
||||
identityPoolId: "us-east-2:4aa2665a-6b7f-4202-856c-333e1f4bdec6",
|
||||
endpoint: "https://dataplane.rum.us-east-2.amazonaws.com",
|
||||
telemetries: ["performance","errors","http"],
|
||||
allowCookies: true,
|
||||
enableXRay: false
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,197 @@
|
||||
# Monitor Java/JMX applications running on Amazon EKS
|
||||
|
||||
The current example deploys the [java workload module](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/java),
|
||||
to provide to an existing EKS cluster with an OpenTelemetry collector,
|
||||
curated Grafana dashboards, Prometheus alerting and recording rules with multiple
|
||||
configuration options on the cluster infrastructure.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
|
||||
before proceeding.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Download sources and initialize Terraform
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
|
||||
cd examples/existing-cluster-java
|
||||
terraform init
|
||||
```
|
||||
|
||||
### 2. AWS Region
|
||||
|
||||
Specify the AWS Region where the resources will be deployed:
|
||||
|
||||
```bash
|
||||
export TF_VAR_aws_region=xxx
|
||||
```
|
||||
|
||||
### 3. Amazon EKS Cluster
|
||||
|
||||
To run this example, you need to provide your EKS cluster name. If you don't
|
||||
have a cluster ready, visit [this example](/terraform-aws-observability-accelerator/helpers/new-eks-cluster.md)
|
||||
first to create a new one.
|
||||
|
||||
Specify your cluster name:
|
||||
|
||||
```bash
|
||||
export TF_VAR_eks_cluster_id=xxx
|
||||
```
|
||||
|
||||
### 4. Amazon Managed Service for Prometheus workspace (optional)
|
||||
|
||||
By default, we create an Amazon Managed Service for Prometheus workspace for you.
|
||||
However, if you have an existing workspace you want to reuse, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_prometheus_workspace_id=ws-xxx
|
||||
```
|
||||
|
||||
To create a workspace outside of Terraform's state, simply run:
|
||||
|
||||
```bash
|
||||
aws amp create-workspace --alias observability-accelerator --query '.workspaceId' --output text
|
||||
```
|
||||
|
||||
### 5. Amazon Managed Grafana workspace
|
||||
|
||||
To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_grafana_workspace_id=g-xxx
|
||||
```
|
||||
|
||||
To create a new one, within this example's Terraform state (sharing the same lifecycle with all the other resources):
|
||||
|
||||
- Edit main.tf and set `enable_managed_grafana = true`
|
||||
- Run
|
||||
|
||||
```bash
|
||||
terraform init
|
||||
terraform apply -target "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
export TF_VAR_managed_grafana_workspace_id=$(terraform output --raw managed_grafana_workspace_id)
|
||||
```
|
||||
|
||||
### 6. Grafana API Key
|
||||
|
||||
Amazon Managed Grafana provides a control plane API for generating Grafana API keys.
|
||||
As a security best practice, we will provide to Terraform a short lived API key to
|
||||
run the `apply` or `destroy` command.
|
||||
|
||||
Ensure you have necessary IAM permissions (`CreateWorkspaceApiKey, DeleteWorkspaceApiKey`)
|
||||
|
||||
```bash
|
||||
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Simply run this command to deploy.
|
||||
|
||||
```bash
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Visualization
|
||||
|
||||
1. Prometheus datasource on Grafana
|
||||
|
||||
Open your Grafana workspace and under Configuration -> Data sources, you will see `aws-observability-accelerator`. Open and click `Save & test`. You will then see a notification confirming that the Amazon Managed Service for Prometheus workspace is ready to be used on Grafana.
|
||||
|
||||
2. Grafana dashboards
|
||||
|
||||
Go to the Dashboards panel of your Grafana workspace. There will be a folder called `Observability Accelerator Dashboards`
|
||||
|
||||
<img width="832" alt="image" src="https://user-images.githubusercontent.com/97046295/194903648-57c55d30-6f90-4b03-9eb6-577aaba7dc22.png">
|
||||
|
||||
Open the "Java/JMX" dashboard to view its visualization
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
2. Amazon Managed Service for Prometheus rules and alerts
|
||||
|
||||
Open the Amazon Managed Service for Prometheus console and view the details of your workspace. Under the `Rules management` tab, you will find new rules deployed.
|
||||
|
||||
<img width="1314" alt="image" src="https://user-images.githubusercontent.com/97046295/194904104-09a28577-d149-478e-b0a1-dc21cb7effc1.png">
|
||||
|
||||
|
||||
To setup your alert receiver, with Amazon SNS, follow [this documentation](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-alertmanager-receiver.html)
|
||||
|
||||
|
||||
## Deploy an Example Java Application
|
||||
|
||||
In this section we will reuse an example from the AWS OpenTelemetry collector [repository](https://github.com/aws-observability/aws-otel-collector/blob/main/docs/developers/container-insights-eks-jmx.md). For convenience, the steps can be found below.
|
||||
|
||||
1. Clone [this repository](https://github.com/aws-observability/aws-otel-test-framework) and navigate to the `sample-apps/jmx/` directory.
|
||||
|
||||
2. Authenticate to Amazon ECR
|
||||
|
||||
```sh
|
||||
export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
|
||||
export AWS_REGION={region}
|
||||
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
|
||||
```
|
||||
|
||||
3. Create an Amazon ECR repository
|
||||
|
||||
```sh
|
||||
aws ecr create-repository --repository-name prometheus-sample-tomcat-jmx \
|
||||
--image-scanning-configuration scanOnPush=true \
|
||||
--region $AWS_REGION
|
||||
```
|
||||
|
||||
4. Build Docker image and push to ECR.
|
||||
|
||||
```sh
|
||||
docker build -t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest .
|
||||
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest
|
||||
```
|
||||
|
||||
5. Install sample application
|
||||
|
||||
```sh
|
||||
export SAMPLE_TRAFFIC_NAMESPACE=javajmx-sample
|
||||
curl https://raw.githubusercontent.com/aws-observability/aws-otel-test-framework/terraform/sample-apps/jmx/examples/prometheus-metrics-sample.yaml > metrics-sample.yaml
|
||||
sed -i "s/{{aws_account_id}}/$AWS_ACCOUNT_ID/g" metrics-sample.yaml
|
||||
sed -i "s/{{region}}/$AWS_REGION/g" metrics-sample.yaml
|
||||
sed -i "s/{{namespace}}/$SAMPLE_TRAFFIC_NAMESPACE/g" metrics-sample.yaml
|
||||
kubectl apply -f metrics-sample.yaml
|
||||
```
|
||||
|
||||
Verify that the sample application is running:
|
||||
|
||||
```sh
|
||||
kubectl get pods -n $SAMPLE_TRAFFIC_NAMESPACE
|
||||
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
tomcat-bad-traffic-generator 1/1 Running 0 11s
|
||||
tomcat-example-7958666589-2q755 0/1 ContainerCreating 0 11s
|
||||
tomcat-traffic-generator 1/1 Running 0 11s
|
||||
```
|
||||
|
||||
## Destroy resources
|
||||
|
||||
If you leave this stack running, you will continue to incur charges. To remove all resources
|
||||
created by Terraform, [refresh your Grafana API key](#6-grafana-api-key) and run the command below.
|
||||
|
||||
Be careful, this command will removing everything created by Terraform. If you wish
|
||||
to keep your Amazon Managed Grafana or Amazon Managed Service for Prometheus workspaces. Remove them
|
||||
from your terraform state before running the destroy command.
|
||||
|
||||
```bash
|
||||
terraform destroy
|
||||
```
|
||||
|
||||
To remove resources from your Terraform state, run
|
||||
|
||||
```bash
|
||||
# grafana workspace
|
||||
terraform state rm "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
|
||||
# prometheus workspace
|
||||
terraform state rm "module.eks_observability_accelerator.aws_prometheus_workspace.this[0]"
|
||||
```
|
||||
@@ -0,0 +1,98 @@
|
||||
# Monitoring Amazon Managed Service for Prometheus workspaces
|
||||
|
||||
This example allows you to monitor your Amazon Managed Service for Prometheus workspaces
|
||||
using Amazon CloudWatch vended metrics and logs. It also creates configurable CloudWatch
|
||||
alarms for service usage limits. Those informations are displayed in a Managed Grafana
|
||||
workspace dashboard.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
|
||||
before proceeding.
|
||||
|
||||
> This example doesn't require an Amazon EKS cluster and Kubernetes tools (ex. `kubectl`).
|
||||
|
||||
## Setup
|
||||
|
||||
|
||||
### 1. Download sources and initialize Terraform
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
|
||||
cd examples/managed-prometheus-monitoring
|
||||
terraform init
|
||||
```
|
||||
|
||||
### 2. AWS Region
|
||||
|
||||
Specify the AWS Region where the resources will be deployed:
|
||||
|
||||
```bash
|
||||
export TF_VAR_aws_region=xxx
|
||||
```
|
||||
|
||||
### 3. Amazon Managed Service for Prometheus workspace
|
||||
|
||||
Specify one or more workspaces in the same Region separated with a comma seperated string.
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_prometheus_workspace_id="ws-xxx"
|
||||
```
|
||||
|
||||
You can use the following command to create alarms for all of the workspaces in a region.
|
||||
|
||||
```sh
|
||||
export TF_VAR_managed_prometheus_workspace_id=$(aws amp list-workspaces --query 'workspaces[].workspaceId' --output text | sed -E 's/\t/,/g')
|
||||
```
|
||||
|
||||
### 4. Amazon Managed Grafana workspace
|
||||
|
||||
To run this example you need an Amazon Managed Grafana workspace.
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_grafana_workspace_id=g-xxx
|
||||
```
|
||||
|
||||
### 5. Grafana API Key
|
||||
|
||||
Amazon Managed Grafana provides a control plane API for generating Grafana API keys.
|
||||
As a security best practice, we will provide to Terraform a short lived API key to
|
||||
run the `apply` or `destroy` command.
|
||||
|
||||
Ensure you have necessary IAM permissions (`CreateWorkspaceApiKey, DeleteWorkspaceApiKey`)
|
||||
|
||||
```bash
|
||||
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Simply run this command to deploy the example
|
||||
|
||||
```sh
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Visualization
|
||||
|
||||
### 1. Cloudwatch datasource on Grafana
|
||||
|
||||
Open your Grafana workspace and under Configuration -> Data sources, you should see `aws-observability-accelerator-cloudwatch`. Open and click `Save & test`. You should see a notification confirming that the CloudWatch datasource is ready to be used on Grafana.
|
||||
|
||||
### 2. Grafana dashboards
|
||||
|
||||
Go to the Dashboards panel of your Grafana workspace. You should see a list of dashboards under the `AMP Monitoring Dashboards` folder.
|
||||
|
||||
Open the `AMP Accelerator Dashboard` to see a visualization of the AMP workspace.
|
||||
|
||||
<img width="1786" alt="Screen Shot 2022-10-11 at 2 16 17 PM" src="https://user-images.githubusercontent.com/97046295/196742772-fba1a5fb-dd38-445c-88a9-607f38994713.png">
|
||||
|
||||
### 3. Amazon Managed Service for Prometheus CloudWatch Alarms.
|
||||
|
||||
Open the CloudWatch console and click `Alarms` > `All Alarms` to review the service limit alarms.
|
||||
|
||||
<img width="1525" alt="image" src="https://user-images.githubusercontent.com/97046295/196742923-876e3b1c-6f2a-419d-ad39-9c057a0f7650.png">
|
||||
|
||||
In us-east-1 region an alarm is created for billing. This alarm utilizes anomaly detection to detect anomalies in the Estimated Charges billing metric.
|
||||
|
||||
<img width="1346" alt="image" src="https://user-images.githubusercontent.com/97046295/197042518-a98d69df-8f53-4a4a-afb8-f424d91da56f.png">
|
||||
@@ -0,0 +1,197 @@
|
||||
# Monitor Nginx applications running on Amazon EKS
|
||||
|
||||
The current example deploys the [nginx workload module](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/nginx),
|
||||
to provide an existing EKS cluster with an OpenTelemetry collector,
|
||||
curated Grafana dashboards, Prometheus alerting and recording rules with multiple
|
||||
configuration options on the cluster infrastructure.
|
||||
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
|
||||
before proceeding.
|
||||
|
||||
## Setup
|
||||
|
||||
|
||||
### 1. Download sources and initialize Terraform
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
|
||||
cd examples/existing-cluster-nginx
|
||||
terraform init
|
||||
```
|
||||
|
||||
### 2. AWS Region
|
||||
|
||||
Specify the AWS Region where the resources will be deployed:
|
||||
|
||||
```bash
|
||||
export TF_VAR_aws_region=xxx
|
||||
```
|
||||
|
||||
### 3. Amazon EKS Cluster
|
||||
|
||||
To run this example, you need to provide your EKS cluster name. If you don't
|
||||
have a cluster ready, visit [this example](/terraform-aws-observability-accelerator/helpers/new-eks-cluster.md)
|
||||
first to create a new one.
|
||||
|
||||
Specify your cluster name:
|
||||
|
||||
```bash
|
||||
export TF_VAR_eks_cluster_id=xxx
|
||||
```
|
||||
|
||||
### 4. Amazon Managed Service for Prometheus workspace (optional)
|
||||
|
||||
By default, we create an Amazon Managed Service for Prometheus workspace for you.
|
||||
However, if you have an existing workspace you want to reuse, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_prometheus_workspace_id=ws-xxx
|
||||
```
|
||||
|
||||
To create a workspace outside of Terraform's state, simply run:
|
||||
|
||||
```bash
|
||||
aws amp create-workspace --alias observability-accelerator --query '.workspaceId' --output text
|
||||
```
|
||||
|
||||
### 5. Amazon Managed Grafana workspace
|
||||
|
||||
To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, edit and run:
|
||||
|
||||
```bash
|
||||
export TF_VAR_managed_grafana_workspace_id=g-xxx
|
||||
```
|
||||
|
||||
To create a new one, within this example's Terraform state (sharing the same lifecycle with all the other resources):
|
||||
|
||||
- Edit main.tf and set `enable_managed_grafana = true`
|
||||
- Run
|
||||
|
||||
```bash
|
||||
terraform init
|
||||
terraform apply -target "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
export TF_VAR_managed_grafana_workspace_id=$(terraform output --raw managed_grafana_workspace_id)
|
||||
```
|
||||
|
||||
### 6. Grafana API Key
|
||||
|
||||
Amazon Managed Grafana provides a control plane API for generating Grafana API keys.
|
||||
As a security best practice, we will provide to Terraform a short lived API key to
|
||||
run the `apply` or `destroy` command.
|
||||
|
||||
Ensure you have necessary IAM permissions (`CreateWorkspaceApiKey, DeleteWorkspaceApiKey`)
|
||||
|
||||
```bash
|
||||
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Simply run this command to deploy.
|
||||
|
||||
```bash
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Visualization
|
||||
|
||||
### 1. Prometheus datasource on Grafana
|
||||
|
||||
Open your Grafana workspace and under Configuration -> Data sources, you will see `aws-observability-accelerator`. Open and click `Save & test`. You will see a notification confirming that the Amazon Managed Service for Prometheus workspace is ready to be used on Grafana.
|
||||
|
||||
### 2. Grafana dashboards
|
||||
|
||||
Go to the Dashboards panel of your Grafana workspace. You will see a list of dashboards under the `Observability Accelerator Dashboards`
|
||||
|
||||
<img width="1208" alt="image" src="https://user-images.githubusercontent.com/97046295/190665211-60faef71-d83d-4d59-ac80-bf4309d8c082.png">
|
||||
|
||||
Open the NGINX dashboard and you will be able to view its visualization
|
||||
|
||||
<img width="1850" alt="image" src="https://user-images.githubusercontent.com/97046295/196226043-e49afeb9-7828-467f-9199-5707cdc69aa9.png">
|
||||
|
||||
### 3. Amazon Managed Service for Prometheus rules and alerts
|
||||
|
||||
Open the Amazon Managed Service for Prometheus console and view the details of your workspace. Under the `Rules management` tab, you will find new rules deployed.
|
||||
|
||||
<img width="1054" alt="image" src="https://user-images.githubusercontent.com/97046295/190665728-ae8bb709-ad93-4629-b845-85c158dd1925.png">
|
||||
|
||||
|
||||
To setup your alert receiver, with Amazon SNS, follow [this documentation](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-alertmanager-receiver.html)
|
||||
|
||||
## Deploy an Example Application to Visualize
|
||||
|
||||
In this section we will deploy sample application and extract metrics using AWS OpenTelemetry collector
|
||||
|
||||
### 1. Add the helm incubator repo:
|
||||
|
||||
```sh
|
||||
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
|
||||
```
|
||||
|
||||
### 2. Enter the following command to create a new namespace:
|
||||
|
||||
```sh
|
||||
kubectl create namespace nginx-ingress-sample
|
||||
```
|
||||
|
||||
### 3. Enter the following commands to install NGINX:
|
||||
|
||||
```sh
|
||||
helm install my-nginx ingress-nginx/ingress-nginx \
|
||||
--namespace nginx-ingress-sample \
|
||||
--set controller.metrics.enabled=true \
|
||||
--set-string controller.metrics.service.annotations."prometheus\.io/port"="10254" \
|
||||
--set-string controller.metrics.service.annotations."prometheus\.io/scrape"="true"
|
||||
```
|
||||
|
||||
### 4. Set an EXTERNAL-IP variable to the value of the EXTERNAL-IP column in the row of the NGINX ingress controller.
|
||||
|
||||
```sh
|
||||
EXTERNAL_IP=your-nginx-controller-external-ip
|
||||
```
|
||||
|
||||
### 5. Start some sample NGINX traffic by entering the following command.
|
||||
|
||||
```sh
|
||||
SAMPLE_TRAFFIC_NAMESPACE=nginx-sample-traffic
|
||||
curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/master/k8s-deployment-manifest-templates/deployment-mode/service/cwagent-prometheus/sample_traffic/nginx-traffic/nginx-traffic-sample.yaml |
|
||||
sed "s/{{external_ip}}/$EXTERNAL_IP/g" |
|
||||
sed "s/{{namespace}}/$SAMPLE_TRAFFIC_NAMESPACE/g" |
|
||||
kubectl apply -f -
|
||||
```
|
||||
|
||||
### 6. Verify if the application is running
|
||||
|
||||
```sh
|
||||
kubectl get pods -n nginx-ingress-sample
|
||||
```
|
||||
|
||||
### 7. Visualize the Application's dashboard
|
||||
|
||||
Log back into your Managed Grafana Workspace and navigate to the dashboard side panel, click on `Observability Accelerator Dashboards` Folder and open the `NGINX` Dashboard.
|
||||
|
||||
## Destroy resources
|
||||
|
||||
If you leave this stack running, you will continue to incur charges. To remove all resources
|
||||
created by Terraform, [refresh your Grafana API key](#6-grafana-api-key) and run the command below.
|
||||
|
||||
Be careful, this command will removing everything created by Terraform. If you wish
|
||||
to keep your Amazon Managed Grafana or Amazon Managed Service for Prometheus workspaces. Remove them
|
||||
from your terraform state before running the destroy command.
|
||||
|
||||
```bash
|
||||
terraform destroy
|
||||
```
|
||||
|
||||
To remove resources from your Terraform state, run
|
||||
|
||||
```bash
|
||||
# grafana workspace
|
||||
terraform state rm "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
|
||||
|
||||
# prometheus workspace
|
||||
terraform state rm "module.eks_observability_accelerator.aws_prometheus_workspace.this[0]"
|
||||
```
|
||||
@@ -109,7 +109,7 @@ In us-east-1 region an alarm is created for billing. This alarm utilizes anomaly
|
||||
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0, < 1.3.0 |
|
||||
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
|
||||
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
|
||||
| <a name="requirement_grafana"></a> [grafana](#requirement\_grafana) | >= 1.25.0 |
|
||||
|
||||
@@ -117,15 +117,14 @@ In us-east-1 region an alarm is created for billing. This alarm utilizes anomaly
|
||||
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.36.1 |
|
||||
| <a name="provider_grafana"></a> [grafana](#provider\_grafana) | 1.30.0 |
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.46.0 |
|
||||
| <a name="provider_grafana"></a> [grafana](#provider\_grafana) | 1.31.1 |
|
||||
|
||||
## Modules
|
||||
|
||||
| Name | Source | Version |
|
||||
|------|--------|---------|
|
||||
| <a name="module_amp_monitor"></a> [amp\_monitor](#module\_amp\_monitor) | ../../modules/workloads/amp-monitoring | n/a |
|
||||
| <a name="module_billing"></a> [billing](#module\_billing) | ../../modules/Billing | n/a |
|
||||
| <a name="module_managed_prometheus_monitoring"></a> [managed\_prometheus\_monitoring](#module\_managed\_prometheus\_monitoring) | ../../modules/workloads/managed-prometheus-monitoring | n/a |
|
||||
|
||||
## Resources
|
||||
|
||||
@@ -140,12 +139,12 @@ In us-east-1 region an alarm is created for billing. This alarm utilizes anomaly
|
||||
|------|-------------|------|---------|:--------:|
|
||||
| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
|
||||
| <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | API key for authorizing the Grafana provider to make changes to Amazon Managed Grafana | `string` | n/a | yes |
|
||||
| <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana (AMG) workspace ID | `string` | n/a | yes |
|
||||
| <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Service for Prometheus Workspace ID to create Alarms for | `string` | n/a | yes |
|
||||
| <a name="input_managed_grafana_workspace_id"></a> [managed\_grafana\_workspace\_id](#input\_managed\_grafana\_workspace\_id) | Amazon Managed Grafana workspace ID | `string` | n/a | yes |
|
||||
| <a name="input_managed_prometheus_workspace_ids"></a> [managed\_prometheus\_workspace\_ids](#input\_managed\_prometheus\_workspace\_ids) | Amazon Managed Service for Prometheus Workspace IDs to create Alarms for | `string` | n/a | yes |
|
||||
|
||||
## Outputs
|
||||
|
||||
| Name | Description |
|
||||
|------|-------------|
|
||||
| <a name="output_grafana_dashboards_folder_id"></a> [grafana\_dashboards\_folder\_id](#output\_grafana\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards. Required by workload modules |
|
||||
| <a name="output_grafana_dashboard_urls"></a> [grafana\_dashboard\_urls](#output\_grafana\_dashboard\_urls) | URLs for dashboards created |
|
||||
<!-- END_TF_DOCS -->
|
||||
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
site_name: AWS Observability Accelerator for Terraform
|
||||
docs_dir: "docs"
|
||||
copyright: "Copyright © Amazon 2022"
|
||||
site_author: "AWS"
|
||||
site_url: "https://aws-observability.github.io/terraform-aws-observability-accelerator/"
|
||||
repo_name: "aws-observability/terraform-aws-observability-accelerator"
|
||||
repo_url: "https://github.com/aws-observability/terraform-aws-observability-accelerator"
|
||||
|
||||
theme:
|
||||
logo: ../images/aws-logo.png
|
||||
favicon: ../images/aws-favicon.png
|
||||
name: material
|
||||
font:
|
||||
text: ember
|
||||
custom_dir: docs/overrides
|
||||
icon:
|
||||
repo: fontawesome/brands/github
|
||||
features:
|
||||
- navigation.tabs.sticky
|
||||
|
||||
palette:
|
||||
primary: indigo
|
||||
accent: grey
|
||||
|
||||
nav:
|
||||
- Home: index.md
|
||||
- Concepts: concepts.md
|
||||
- Amazon EKS Cluster Monitoring: eks.md
|
||||
- Workload Monitoring:
|
||||
- Java/JMX: workloads/java.md
|
||||
- Nginx: workloads/nginx.md
|
||||
- Amazon Managed Service for Prometheus Workspaces: workloads/managed-prometheus.md
|
||||
- Supporting Examples:
|
||||
- EKS Cluster with VPC: helpers/new-eks-cluster.md
|
||||
# - Amazon Managed Grafana setup: helpers/managed-grafana.md
|
||||
- Contributors: contributors.md
|
||||
|
||||
markdown_extensions:
|
||||
- toc:
|
||||
permalink: true
|
||||
- admonition
|
||||
- codehilite
|
||||
- footnotes
|
||||
- pymdownx.critic
|
||||
|
||||
plugins:
|
||||
- search
|
||||
Reference in New Issue
Block a user