storytold/terraform-aws-observability-accelerator
2
0
Fork 0
mirror of https://github.com/storytold/terraform-aws-observability-accelerator.git synced 2026-10-09 00:09:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
f29113f4da08d2d36e5ce2401eedbe94146da50f
terraform-aws-observability…/modules/eks-monitoring/README.md
T
Rodrigue Koffi 08b4be1490 ADOT collector scrape improvements (#188) 
* Bump default scrape interval to 120s

* fixup! Bump default scrape interval to 120s
2023-07-07 09:44:45 -04:00

19 KiB
Raw Blame History

Infrastructure monitoring

This module provides EKS cluster monitoring with the following resources:

  • AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
  • Logs with AWS for FluentBit
  • Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
  • Installs FluxCD to perform GitOps sync of a Git Repo to EKS Cluster. We will use this later for creating Grafana Dashboards and AWS datasources to Amazon Managed Grafana.
  • Installs External Secrets Operator to retrieve and Sync the Grafana API keys from AWS SSM Parameter Store.
  • Amazon Managed Grafana Dashboard and data source
  • Alerts and recording rules with AWS Managed Service for Prometheus

This module makes use of the open source kube-prometheus-stack

See examples using this Terraform modules in the Amazon EKS section of this documentation

Requirements

Name Version
terraform >= 1.1.0
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 1.14
kubernetes >= 2.10

Providers

Name Version
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 1.14

Modules

Name Source Version
external_secrets ./add-ons/external-secrets n/a
fluentbit_logs ./add-ons/aws-for-fluentbit n/a
helm_addon github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon v4.32.0
java_monitoring ./patterns/java n/a
nginx_monitoring ./patterns/nginx n/a
operator ./add-ons/adot-operator n/a

Resources

Name Type
aws_prometheus_rule_group_namespace.alerting_rules resource
aws_prometheus_rule_group_namespace.recording_rules resource
helm_release.fluxcd resource
helm_release.grafana_operator resource
helm_release.kube_state_metrics resource
helm_release.prometheus_node_exporter resource
kubectl_manifest.flux_gitrepository resource
kubectl_manifest.flux_kustomization resource
aws_caller_identity.current data source
aws_eks_cluster.eks_cluster data source
aws_partition.current data source
aws_region.current data source

Inputs

Name Description Type Default Required
adot_loglevel Verbosity level for ADOT collector logs string "warn" no
custom_metrics_config Configuration object to enable custom metrics collection 
object({
    ports = list(number)
    # paths = optional(list(string), ["/metrics"])
    # list of samples to be dropped by label prefix, ex: go_ -> discards go_.*
    dropped_series_prefixes = list(string)
  })
 
{
  "dropped_series_prefixes": [
    "unspecified"
  ],
  "ports": []
}
 no
eks_cluster_id EKS Cluster Id string n/a yes
enable_alerting_rules Enables or disables Managed Prometheus alerting rules bool true no
enable_amazon_eks_adot Enables the ADOT Operator on the EKS Cluster bool true no
enable_cert_manager Allow reusing an existing installation of cert-manager bool true no
enable_custom_metrics Allows additional metrics collection for config elements in the custom_metrics_config config object. Automatic dashboards are not included bool false no
enable_dashboards Enables or disables curated dashboards bool true no
enable_external_secrets Installs External Secrets to EKS Cluster bool true no
enable_fluxcd Enables or disables FluxCD. Disabling this might affect some data in the dashboards bool true no
enable_grafana_operator Deploys Grafana Operator to EKS Cluster bool true no
enable_java Enable Java workloads monitoring, alerting and default dashboards bool false no
enable_kube_state_metrics Enables or disables Kube State metrics exporter. Disabling this might affect some data in the dashboards bool true no
enable_logs Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch bool true no
enable_nginx Enable NGINX workloads monitoring, alerting and default dashboards bool false no
enable_node_exporter Enables or disables Node exporter. Disabling this might affect some data in the dashboards bool true no
enable_recording_rules Enables or disables Managed Prometheus recording rules bool true no
enable_tracing (Experimental) Enables tracing with AWS X-Ray. This changes the deploy mode of the collector to daemon set. Requirement: adot add-on <= 0.58-build.0 bool false no
flux_config FluxCD configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "flux2",
  "helm_chart_version": "2.7.0",
  "helm_release_name": "observability-fluxcd-addon",
  "helm_repo_url": "https://fluxcd-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "flux-system"
}
 no
flux_gitrepository_branch Flux GitRepository Branch string "main" no
flux_gitrepository_name Flux GitRepository name string "aws-observability-accelerator" no
flux_gitrepository_url Flux GitRepository URL string "https://github.com/aws-observability/aws-observability-accelerator" no
flux_kustomization_name Flux Kustomization name string "grafana-dashboards-infrastructure" no
flux_kustomization_path Flux Kustomization Path string "./artifacts/grafana-operator-manifests/eks/infrastructure" no
go_config Grafana Operator configuration 
object({
    create_namespace   = bool
    helm_chart         = string
    helm_name          = string
    k8s_namespace      = string
    helm_release_name  = string
    helm_chart_version = string
  })
 
{
  "create_namespace": true,
  "helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",
  "helm_chart_version": "v5.0.0-rc3",
  "helm_name": "grafana-operator",
  "helm_release_name": "grafana-operator",
  "k8s_namespace": "grafana-operator"
}
 no
grafana_api_key Grafana API key for the Amazon Managed Grafana workspace. Required if enable_external_secrets = true string "" no
grafana_cluster_dashboard_url Dashboard URL for Cluster Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json" no
grafana_kubelet_dashboard_url Dashboard URL for Kubelet Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json" no
grafana_namespace_workloads_dashboard_url Dashboard URL for Namespace Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json" no
grafana_node_exporter_dashboard_url Dashboard URL for Node Exporter Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json" no
grafana_nodes_dashboard_url Dashboard URL for Nodes Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json" no
grafana_url Endpoint URL of Amazon Managed Grafana workspace. Required if enable_grafana_operator = true string "" no
grafana_workloads_dashboard_url Dashboard URL for Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json" no
helm_config Helm Config for Prometheus any {} no
irsa_iam_permissions_boundary IAM permissions boundary for IRSA roles string null no
irsa_iam_role_path IAM role path for IRSA roles string "/" no
java_config Configuration object for Java/JMX monitoring 
object({
    enable_alerting_rules  = bool
    enable_recording_rules = bool
    enable_dashboards      = bool
    scrape_sample_limit    = number


    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    grafana_dashboard_url = string

    prometheus_metrics_endpoint = string
  })
 null no
ksm_config Kube State metrics configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)

    scrape_interval = string
    scrape_timeout  = string
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "kube-state-metrics",
  "helm_chart_version": "4.24.0",
  "helm_release_name": "kube-state-metrics",
  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "kube-system",
  "scrape_interval": "60s",
  "scrape_timeout": "15s"
}
 no
logs_config Configuration object for logs collection 
object({
    cw_log_retention_days = number
  })
 
{
  "cw_log_retention_days": 90
}
 no
managed_prometheus_workspace_endpoint Amazon Managed Prometheus Workspace Endpoint string "" no
managed_prometheus_workspace_id Amazon Managed Prometheus Workspace ID string null no
managed_prometheus_workspace_region Amazon Managed Prometheus Workspace's Region string null no
ne_config Node exporter configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)

    scrape_interval = string
    scrape_timeout  = string
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "prometheus-node-exporter",
  "helm_chart_version": "4.14.0",
  "helm_release_name": "prometheus-node-exporter",
  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "prometheus-node-exporter",
  "scrape_interval": "60s",
  "scrape_timeout": "60s"
}
 no
nginx_config Configuration object for NGINX monitoring 
object({
    enable_alerting_rules  = bool
    enable_recording_rules = bool
    enable_dashboards      = bool
    scrape_sample_limit    = number

    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    grafana_dashboard_url = string

    prometheus_metrics_endpoint = string
  })
 null no
prometheus_config Controls default values such as scrape interval, timeouts and ports globally 
object({
    global_scrape_interval = string
    global_scrape_timeout  = string
  })
 
{
  "global_scrape_interval": "120s",
  "global_scrape_timeout": "15s"
}
 no
tags Additional tags (e.g. map('BusinessUnit,XYZ) map(string) {} no
target_secret_name Target secret in Kubernetes to store the Grafana API Key Secret string "grafana-admin-credentials" no
target_secret_namespace Target namespace of secret in Kubernetes to store the Grafana API Key Secret string "grafana-operator" no
tracing_config Configuration object for traces collection to AWS X-Ray 
object({
    otlp_grpc_endpoint = string
    otlp_http_endpoint = string
    send_batch_size    = number
    timeout            = string
  })
 
{
  "otlp_grpc_endpoint": "0.0.0.0:4317",
  "otlp_http_endpoint": "0.0.0.0:4318",
  "send_batch_size": 50,
  "timeout": "30s"
}
 no

Outputs

Name Description
eks_cluster_id EKS Cluster Id
eks_cluster_version EKS Cluster version
Reference in New Issue View Git Blame Copy Permalink