mirror of
https://github.com/storytold/terraform-aws-observability-accelerator.git
synced 2026-10-09 00:09:43 +00:00
Support Fluentbit to CloudWatch logs (#140)
* Import and customize fluenbit add-on * Enable fluent bit logs * Bump helm addon version * Dropping account id as it seems to create scraping errors * Create separate log groups per namespace * Apply pre-commit * Remove conflicting global label * Add config object for logs * Enable logs in examples * Add logs docs * Fix broken link * Add screenshots * Update docs * Typos
This commit is contained in:
Rodrigue Koffi
@@ -2,11 +2,12 @@
|
||||
|
||||
This module provides EKS cluster monitoring with the following resources:
|
||||
|
||||
- AWS Distro For OpenTelemetry Operator and Collector
|
||||
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
|
||||
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
|
||||
- AWS Managed Grafana Dashboard and data source
|
||||
- Alerts and recording rules with AWS Managed Service for Prometheus
|
||||
|
||||
This module is inspired from the open source [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack)
|
||||
This module makes use of the open source [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack)
|
||||
|
||||
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
|
||||
## Requirements
|
||||
@@ -32,7 +33,8 @@ This module is inspired from the open source [kube-prometheus-stack](https://git
|
||||
|
||||
| Name | Source | Version |
|
||||
|------|--------|---------|
|
||||
| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.13.1 |
|
||||
| <a name="module_fluentbit_logs"></a> [fluentbit\_logs](#module\_fluentbit\_logs) | ./add-ons/aws-for-fluentbit | n/a |
|
||||
| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.26.0 |
|
||||
| <a name="module_java_monitoring"></a> [java\_monitoring](#module\_java\_monitoring) | ./patterns/java | n/a |
|
||||
| <a name="module_nginx_monitoring"></a> [nginx\_monitoring](#module\_nginx\_monitoring) | ./patterns/nginx | n/a |
|
||||
| <a name="module_operator"></a> [operator](#module\_operator) | ./add-ons/adot-operator | n/a |
|
||||
@@ -70,6 +72,7 @@ This module is inspired from the open source [kube-prometheus-stack](https://git
|
||||
| <a name="input_enable_dashboards"></a> [enable\_dashboards](#input\_enable\_dashboards) | Enables or disables curated dashboards | `bool` | `true` | no |
|
||||
| <a name="input_enable_java"></a> [enable\_java](#input\_enable\_java) | Enable Java workloads monitoring, alerting and default dashboards | `bool` | `false` | no |
|
||||
| <a name="input_enable_kube_state_metrics"></a> [enable\_kube\_state\_metrics](#input\_enable\_kube\_state\_metrics) | Enables or disables Kube State metrics exporter. Disabling this might affect some data in the dashboards | `bool` | `true` | no |
|
||||
| <a name="input_enable_logs"></a> [enable\_logs](#input\_enable\_logs) | Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch | `bool` | `true` | no |
|
||||
| <a name="input_enable_nginx"></a> [enable\_nginx](#input\_enable\_nginx) | Enable NGINX workloads monitoring, alerting and default dashboards | `bool` | `false` | no |
|
||||
| <a name="input_enable_node_exporter"></a> [enable\_node\_exporter](#input\_enable\_node\_exporter) | Enables or disables Node exporter. Disabling this might affect some data in the dashboards | `bool` | `true` | no |
|
||||
| <a name="input_enable_tracing"></a> [enable\_tracing](#input\_enable\_tracing) | (Experimental) Enables tracing with AWS X-Ray. This changes the deploy mode of the collector to daemon set. Requirement: adot add-on <= 0.58-build.0 | `bool` | `false` | no |
|
||||
@@ -78,6 +81,7 @@ This module is inspired from the open source [kube-prometheus-stack](https://git
|
||||
| <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
|
||||
| <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br> enable_alerting_rules = bool<br> scrape_sample_limit = number<br> })</pre> | <pre>{<br> "enable_alerting_rules": true,<br> "scrape_sample_limit": 1000<br>}</pre> | no |
|
||||
| <a name="input_ksm_config"></a> [ksm\_config](#input\_ksm\_config) | Kube State metrics configuration | <pre>object({<br> create_namespace = bool<br> k8s_namespace = string<br> helm_chart_name = string<br> helm_chart_version = string<br> helm_release_name = string<br> helm_repo_url = string<br> helm_settings = map(string)<br> helm_values = map(any)<br><br> scrape_interval = string<br> scrape_timeout = string<br> })</pre> | <pre>{<br> "create_namespace": true,<br> "helm_chart_name": "kube-state-metrics",<br> "helm_chart_version": "4.24.0",<br> "helm_release_name": "kube-state-metrics",<br> "helm_repo_url": "https://prometheus-community.github.io/helm-charts",<br> "helm_settings": {},<br> "helm_values": {},<br> "k8s_namespace": "kube-system",<br> "scrape_interval": "60s",<br> "scrape_timeout": "15s"<br>}</pre> | no |
|
||||
| <a name="input_logs_config"></a> [logs\_config](#input\_logs\_config) | Configuration object for logs collection | <pre>object({<br> cw_log_retention_days = number<br> })</pre> | <pre>{<br> "cw_log_retention_days": 90<br>}</pre> | no |
|
||||
| <a name="input_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#input\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus Workspace Endpoint | `string` | `""` | no |
|
||||
| <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |
|
||||
| <a name="input_managed_prometheus_workspace_region"></a> [managed\_prometheus\_workspace\_region](#input\_managed\_prometheus\_workspace\_region) | Amazon Managed Prometheus Workspace's Region | `string` | `null` | no |
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# AWS for Fluent Bit
|
||||
|
||||
Fluent Bit is an open source Log Processor and Forwarder which allows you to collect any data like metrics and logs from different sources, enrich them with filters and send them to multiple destinations.
|
||||
AWS provides a Fluent Bit image with plugins for CloudWatch Logs, Kinesis Data Firehose, Kinesis Data Stream and Amazon OpenSearch Service.
|
||||
|
||||
This add-on is configured to stream the worker node logs to CloudWatch Logs by default. It can be configured to stream the logs to additional destinations like Kinesis Data Firehose, Kinesis Data Streams and Amazon OpenSearch Service by passing the custom `values.yaml`.
|
||||
See this [Helm Chart](https://github.com/aws/eks-charts/tree/master/stable/aws-for-fluent-bit) for more details.
|
||||
|
||||
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
|
||||
## Requirements
|
||||
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0.0 |
|
||||
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.72 |
|
||||
|
||||
## Providers
|
||||
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.72 |
|
||||
|
||||
## Modules
|
||||
|
||||
| Name | Source | Version |
|
||||
|------|--------|---------|
|
||||
| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.26.0 |
|
||||
|
||||
## Resources
|
||||
|
||||
| Name | Type |
|
||||
|------|------|
|
||||
| [aws_iam_policy.aws_for_fluent_bit](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
|
||||
| [aws_iam_policy_document.irsa](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
|
||||
## Inputs
|
||||
|
||||
| Name | Description | Type | Default | Required |
|
||||
|------|-------------|------|---------|:--------:|
|
||||
| <a name="input_addon_context"></a> [addon\_context](#input\_addon\_context) | Input configuration for the addon | <pre>object({<br> aws_caller_identity_account_id = string<br> aws_caller_identity_arn = string<br> aws_eks_cluster_endpoint = string<br> aws_partition_id = string<br> aws_region_name = string<br> eks_cluster_id = string<br> eks_oidc_issuer_url = string<br> eks_oidc_provider_arn = string<br> tags = map(string)<br> irsa_iam_role_path = string<br> irsa_iam_permissions_boundary = string<br> })</pre> | n/a | yes |
|
||||
| <a name="input_cw_log_retention_days"></a> [cw\_log\_retention\_days](#input\_cw\_log\_retention\_days) | FluentBit CloudWatch Log group retention period | `number` | `90` | no |
|
||||
| <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm provider config aws\_for\_fluent\_bit. | `any` | `{}` | no |
|
||||
| <a name="input_irsa_policies"></a> [irsa\_policies](#input\_irsa\_policies) | Additional IAM policies for a IAM role for service accounts | `list(string)` | `[]` | no |
|
||||
| <a name="input_manage_via_gitops"></a> [manage\_via\_gitops](#input\_manage\_via\_gitops) | Determines if the add-on should be managed via GitOps. | `bool` | `false` | no |
|
||||
|
||||
## Outputs
|
||||
|
||||
| Name | Description |
|
||||
|------|-------------|
|
||||
| <a name="output_irsa_arn"></a> [irsa\_arn](#output\_irsa\_arn) | IAM role ARN for the service account |
|
||||
| <a name="output_irsa_name"></a> [irsa\_name](#output\_irsa\_name) | IAM role name for the service account |
|
||||
| <a name="output_release_metadata"></a> [release\_metadata](#output\_release\_metadata) | Map of attributes of the Helm release metadata |
|
||||
| <a name="output_service_account"></a> [service\_account](#output\_service\_account) | Name of Kubernetes service account |
|
||||
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
|
||||
@@ -0,0 +1,21 @@
|
||||
data "aws_iam_policy_document" "irsa" {
|
||||
statement {
|
||||
sid = "PutLogEvents"
|
||||
effect = "Allow"
|
||||
resources = ["arn:${var.addon_context.aws_partition_id}:logs:${var.addon_context.aws_region_name}:${var.addon_context.aws_caller_identity_account_id}:log-group:*:log-stream:*"]
|
||||
actions = ["logs:PutLogEvents"]
|
||||
}
|
||||
|
||||
statement {
|
||||
sid = "CreateCWLogs"
|
||||
effect = "Allow"
|
||||
resources = ["arn:${var.addon_context.aws_partition_id}:logs:${var.addon_context.aws_region_name}:${var.addon_context.aws_caller_identity_account_id}:log-group:*"]
|
||||
|
||||
actions = [
|
||||
"logs:CreateLogGroup",
|
||||
"logs:CreateLogStream",
|
||||
"logs:DescribeLogGroups",
|
||||
"logs:DescribeLogStreams",
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
locals {
|
||||
name = "aws-for-fluent-bit"
|
||||
service_account = try(var.helm_config.service_account, "${local.name}-sa")
|
||||
|
||||
set_values = [
|
||||
{
|
||||
name = "serviceAccount.name"
|
||||
value = local.service_account
|
||||
},
|
||||
{
|
||||
name = "serviceAccount.create"
|
||||
value = false
|
||||
}
|
||||
]
|
||||
|
||||
# https://github.com/aws/eks-charts/blob/master/stable/aws-for-fluent-bit/Chart.yaml
|
||||
default_helm_config = {
|
||||
name = local.name
|
||||
chart = local.name
|
||||
repository = "https://aws.github.io/eks-charts"
|
||||
version = "0.1.24"
|
||||
namespace = local.name
|
||||
values = local.default_helm_values
|
||||
description = "aws-for-fluentbit Helm Chart deployment configuration"
|
||||
}
|
||||
|
||||
helm_config = merge(
|
||||
local.default_helm_config,
|
||||
var.helm_config
|
||||
)
|
||||
|
||||
default_helm_values = [templatefile("${path.module}/values.yaml", {
|
||||
aws_region = var.addon_context.aws_region_name
|
||||
cluster_name = var.addon_context.eks_cluster_id
|
||||
log_retention_days = var.cw_log_retention_days
|
||||
service_account = local.service_account
|
||||
})]
|
||||
|
||||
irsa_config = {
|
||||
kubernetes_namespace = local.helm_config["namespace"]
|
||||
kubernetes_service_account = local.service_account
|
||||
create_kubernetes_namespace = try(local.helm_config["create_namespace"], true)
|
||||
create_kubernetes_service_account = true
|
||||
create_service_account_secret_token = try(local.helm_config["create_service_account_secret_token"], false)
|
||||
irsa_iam_policies = concat([aws_iam_policy.aws_for_fluent_bit.arn], var.irsa_policies)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
module "helm_addon" {
|
||||
source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon?ref=v4.26.0"
|
||||
manage_via_gitops = var.manage_via_gitops
|
||||
set_values = local.set_values
|
||||
helm_config = local.helm_config
|
||||
irsa_config = local.irsa_config
|
||||
addon_context = var.addon_context
|
||||
}
|
||||
|
||||
resource "aws_iam_policy" "aws_for_fluent_bit" {
|
||||
name = "${var.addon_context.eks_cluster_id}-fluentbit"
|
||||
description = "IAM Policy for AWS for FluentBit"
|
||||
policy = data.aws_iam_policy_document.irsa.json
|
||||
tags = var.addon_context.tags
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
output "release_metadata" {
|
||||
description = "Map of attributes of the Helm release metadata"
|
||||
value = module.helm_addon.release_metadata
|
||||
}
|
||||
|
||||
output "irsa_arn" {
|
||||
description = "IAM role ARN for the service account"
|
||||
value = module.helm_addon.irsa_arn
|
||||
}
|
||||
|
||||
output "irsa_name" {
|
||||
description = "IAM role name for the service account"
|
||||
value = module.helm_addon.irsa_name
|
||||
}
|
||||
|
||||
output "service_account" {
|
||||
description = "Name of Kubernetes service account"
|
||||
value = module.helm_addon.service_account
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
serviceAccount:
|
||||
create: false
|
||||
name: ${service_account}
|
||||
|
||||
cloudWatch:
|
||||
enabled: false
|
||||
|
||||
cloudWatchLogs:
|
||||
enabled: true
|
||||
region: ${aws_region}
|
||||
# logGroupName is a fallback to failed parsing
|
||||
logGroupName: /aws/eks/observability-accelerator/workloads
|
||||
logGroupTemplate: /aws/eks/observability-accelerator/${cluster_name}/$kubernetes['namespace_name']
|
||||
logStreamTemplate: $kubernetes['container_name'].$kubernetes['pod_name']
|
||||
log_key: log
|
||||
log_retention_days: ${log_retention_days}
|
||||
@@ -0,0 +1,40 @@
|
||||
variable "helm_config" {
|
||||
description = "Helm provider config aws_for_fluent_bit."
|
||||
type = any
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "cw_log_retention_days" {
|
||||
description = "FluentBit CloudWatch Log group retention period"
|
||||
type = number
|
||||
default = 90
|
||||
}
|
||||
|
||||
variable "manage_via_gitops" {
|
||||
type = bool
|
||||
description = "Determines if the add-on should be managed via GitOps."
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "irsa_policies" {
|
||||
description = "Additional IAM policies for a IAM role for service accounts"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "addon_context" {
|
||||
description = "Input configuration for the addon"
|
||||
type = object({
|
||||
aws_caller_identity_account_id = string
|
||||
aws_caller_identity_arn = string
|
||||
aws_eks_cluster_endpoint = string
|
||||
aws_partition_id = string
|
||||
aws_region_name = string
|
||||
eks_cluster_id = string
|
||||
eks_oidc_issuer_url = string
|
||||
eks_oidc_provider_arn = string
|
||||
tags = map(string)
|
||||
irsa_iam_role_path = string
|
||||
irsa_iam_permissions_boundary = string
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
terraform {
|
||||
required_version = ">= 1.0.0"
|
||||
|
||||
required_providers {
|
||||
aws = {
|
||||
source = "hashicorp/aws"
|
||||
version = ">= 3.72"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -44,7 +44,7 @@ resource "helm_release" "prometheus_node_exporter" {
|
||||
}
|
||||
|
||||
module "helm_addon" {
|
||||
source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon?ref=v4.13.1"
|
||||
source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon?ref=v4.26.0"
|
||||
|
||||
helm_config = merge(
|
||||
{
|
||||
@@ -169,3 +169,11 @@ module "nginx_monitoring" {
|
||||
enable_alerting_rules = var.nginx_config.enable_alerting_rules
|
||||
dashboards_folder_id = var.dashboards_folder_id
|
||||
}
|
||||
|
||||
module "fluentbit_logs" {
|
||||
source = "./add-ons/aws-for-fluentbit"
|
||||
count = var.enable_logs ? 1 : 0
|
||||
|
||||
cw_log_retention_days = var.logs_config.cw_log_retention_days
|
||||
addon_context = local.context
|
||||
}
|
||||
|
||||
@@ -40,7 +40,6 @@ spec:
|
||||
scrape_timeout: {{ .Values.globalScrapeTimeout }}
|
||||
external_labels:
|
||||
cluster: {{ .Values.ekscluster }}
|
||||
account_id: {{ .Values.accountId }}
|
||||
region: {{ .Values.region }}
|
||||
scrape_configs:
|
||||
- job_name: 'kubernetes-kubelet'
|
||||
|
||||
@@ -247,3 +247,21 @@ variable "nginx_config" {
|
||||
prometheus_metrics_endpoint = "metrics"
|
||||
}
|
||||
}
|
||||
|
||||
variable "enable_logs" {
|
||||
description = "Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch"
|
||||
type = bool
|
||||
default = true
|
||||
}
|
||||
|
||||
variable "logs_config" {
|
||||
description = "Configuration object for logs collection"
|
||||
type = object({
|
||||
cw_log_retention_days = number
|
||||
})
|
||||
|
||||
default = {
|
||||
# Valid values are [1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]
|
||||
cw_log_retention_days = 90
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user