fix(eks-monitoring): kube-proxy scrape config and alert rule (#221)

* changes to just OTEL kube-proxy scrape job

* agree with koffir@ to implement kube-proxy by default

* re-added KubeProxyDown Prometheus alert rule

* added kube-proxy URL related stuff

* corrected kube-proxy stuff according to new structure

* Use SEMVER for flux now

* Run pre-commit

* Bump OTEL chart version

* Substitute URL path with SEMVER

* Run pre-commit

* Added kube-proxy config object

* Run pre-commit

* Use kube-proxy config object for dashboards

---------

Co-authored-by: Jens-Uwe Walther <waltju@amazon.com>

modules/eks-monitoring/README.md

@@ -59,6 +59,7 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | [kubectl_manifest.api_server_dashboards](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 | [kubectl_manifest.flux_gitrepository](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 | [kubectl_manifest.flux_kustomization](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
+| [kubectl_manifest.kubeproxy_monitoring_dashboard](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
 | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
 | [aws_eks_cluster.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
 | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
@@ -69,7 +70,7 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_adot_loglevel"></a> [adot\_loglevel](#input\_adot\_loglevel) | Verbosity level for ADOT collector logs. This accepts (detailed\|normal\|basic), see https://aws-otel.github.io/docs/components/misc-exporters for mor infos. | `string` | `"normal"` | no |
-| <a name="input_adothealth_monitoring_config"></a> [adothealth\_monitoring\_config](#input\_adothealth\_monitoring\_config) | Config object for API server monitoring | <pre>object({<br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    dashboards = object({<br>      grafana_adothealth_dashboard_url = string<br>    })<br>  })</pre> | `null` | no |
+| <a name="input_adothealth_monitoring_config"></a> [adothealth\_monitoring\_config](#input\_adothealth\_monitoring\_config) | Config object for ADOT health monitoring | <pre>object({<br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    dashboards = object({<br>      grafana_adothealth_dashboard_url = string<br>    })<br>  })</pre> | `null` | no |
 | <a name="input_apiserver_monitoring_config"></a> [apiserver\_monitoring\_config](#input\_apiserver\_monitoring\_config) | Config object for API server monitoring | <pre>object({<br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    dashboards = object({<br>      basic           = string<br>      advanced        = string<br>      troubleshooting = string<br>    })<br>  })</pre> | `null` | no |
 | <a name="input_custom_metrics_config"></a> [custom\_metrics\_config](#input\_custom\_metrics\_config) | Configuration object to enable custom metrics collection | <pre>map(object({<br>    enableBasicAuth       = bool<br>    path                  = string<br>    basicAuthUsername     = string<br>    basicAuthPassword     = string<br>    ports                 = string<br>    droppedSeriesPrefixes = string<br>  }))</pre> | `null` | no |
 | <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
@@ -92,26 +93,28 @@ See examples using this Terraform modules in the **Amazon EKS** section of [this
 | <a name="input_enable_recording_rules"></a> [enable\_recording\_rules](#input\_enable\_recording\_rules) | Enables or disables Managed Prometheus recording rules | `bool` | `true` | no |
 | <a name="input_enable_tracing"></a> [enable\_tracing](#input\_enable\_tracing) | Enables tracing with OTLP traces receiver to X-Ray | `bool` | `true` | no |
 | <a name="input_flux_config"></a> [flux\_config](#input\_flux\_config) | FluxCD configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "flux2",<br>  "helm_chart_version": "2.7.0",<br>  "helm_release_name": "observability-fluxcd-addon",<br>  "helm_repo_url": "https://fluxcd-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "flux-system"<br>}</pre> | no |
-| <a name="input_flux_gitrepository_branch"></a> [flux\_gitrepository\_branch](#input\_flux\_gitrepository\_branch) | Flux GitRepository Branch | `string` | `"main"` | no |
+| <a name="input_flux_gitrepository_branch"></a> [flux\_gitrepository\_branch](#input\_flux\_gitrepository\_branch) | Flux GitRepository Branch | `string` | `"v0.2.0"` | no |
 | <a name="input_flux_gitrepository_name"></a> [flux\_gitrepository\_name](#input\_flux\_gitrepository\_name) | Flux GitRepository name | `string` | `"aws-observability-accelerator"` | no |
 | <a name="input_flux_gitrepository_url"></a> [flux\_gitrepository\_url](#input\_flux\_gitrepository\_url) | Flux GitRepository URL | `string` | `"https://github.com/aws-observability/aws-observability-accelerator"` | no |
 | <a name="input_flux_kustomization_name"></a> [flux\_kustomization\_name](#input\_flux\_kustomization\_name) | Flux Kustomization name | `string` | `"grafana-dashboards-infrastructure"` | no |
 | <a name="input_flux_kustomization_path"></a> [flux\_kustomization\_path](#input\_flux\_kustomization\_path) | Flux Kustomization Path | `string` | `"./artifacts/grafana-operator-manifests/eks/infrastructure"` | no |
 | <a name="input_go_config"></a> [go\_config](#input\_go\_config) | Grafana Operator configuration | <pre>object({<br>    create_namespace   = bool<br>    helm_chart         = string<br>    helm_name          = string<br>    k8s_namespace      = string<br>    helm_release_name  = string<br>    helm_chart_version = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",<br>  "helm_chart_version": "v5.0.0-rc3",<br>  "helm_name": "grafana-operator",<br>  "helm_release_name": "grafana-operator",<br>  "k8s_namespace": "grafana-operator"<br>}</pre> | no |
 | <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace. Required if `enable_external_secrets = true` | `string` | `""` | no |
-| <a name="input_grafana_cluster_dashboard_url"></a> [grafana\_cluster\_dashboard\_url](#input\_grafana\_cluster\_dashboard\_url) | Dashboard URL for Cluster Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json"` | no |
+| <a name="input_grafana_cluster_dashboard_url"></a> [grafana\_cluster\_dashboard\_url](#input\_grafana\_cluster\_dashboard\_url) | Dashboard URL for Cluster Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/cluster.json"` | no |
-| <a name="input_grafana_kubelet_dashboard_url"></a> [grafana\_kubelet\_dashboard\_url](#input\_grafana\_kubelet\_dashboard\_url) | Dashboard URL for Kubelet Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json"` | no |
+| <a name="input_grafana_kubelet_dashboard_url"></a> [grafana\_kubelet\_dashboard\_url](#input\_grafana\_kubelet\_dashboard\_url) | Dashboard URL for Kubelet Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json"` | no |
-| <a name="input_grafana_namespace_workloads_dashboard_url"></a> [grafana\_namespace\_workloads\_dashboard\_url](#input\_grafana\_namespace\_workloads\_dashboard\_url) | Dashboard URL for Namespace Workloads Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json"` | no |
+| <a name="input_grafana_kubeproxy_dashboard_url"></a> [grafana\_kubeproxy\_dashboard\_url](#input\_grafana\_kubeproxy\_dashboard\_url) | Dashboard URL for kube-proxy Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/kube-proxy/kube-proxy.json"` | no |
-| <a name="input_grafana_node_exporter_dashboard_url"></a> [grafana\_node\_exporter\_dashboard\_url](#input\_grafana\_node\_exporter\_dashboard\_url) | Dashboard URL for Node Exporter Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json"` | no |
+| <a name="input_grafana_namespace_workloads_dashboard_url"></a> [grafana\_namespace\_workloads\_dashboard\_url](#input\_grafana\_namespace\_workloads\_dashboard\_url) | Dashboard URL for Namespace Workloads Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json"` | no |
-| <a name="input_grafana_nodes_dashboard_url"></a> [grafana\_nodes\_dashboard\_url](#input\_grafana\_nodes\_dashboard\_url) | Dashboard URL for Nodes Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json"` | no |
+| <a name="input_grafana_node_exporter_dashboard_url"></a> [grafana\_node\_exporter\_dashboard\_url](#input\_grafana\_node\_exporter\_dashboard\_url) | Dashboard URL for Node Exporter Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json"` | no |
+| <a name="input_grafana_nodes_dashboard_url"></a> [grafana\_nodes\_dashboard\_url](#input\_grafana\_nodes\_dashboard\_url) | Dashboard URL for Nodes Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodes.json"` | no |
 | <a name="input_grafana_url"></a> [grafana\_url](#input\_grafana\_url) | Endpoint URL of Amazon Managed Grafana workspace. Required if `enable_grafana_operator = true` | `string` | `""` | no |
-| <a name="input_grafana_workloads_dashboard_url"></a> [grafana\_workloads\_dashboard\_url](#input\_grafana\_workloads\_dashboard\_url) | Dashboard URL for Workloads Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"` | no |
+| <a name="input_grafana_workloads_dashboard_url"></a> [grafana\_workloads\_dashboard\_url](#input\_grafana\_workloads\_dashboard\_url) | Dashboard URL for Workloads Grafana Dashboard JSON | `string` | `"https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"` | no |
 | <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm Config for Prometheus | `any` | `{}` | no |
 | <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
 | <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
 | <a name="input_istio_config"></a> [istio\_config](#input\_istio\_config) | Configuration object for ISTIO monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    enable_dashboards      = bool<br>    scrape_sample_limit    = number<br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    managed_prometheus_workspace_id = string<br>    prometheus_metrics_endpoint     = string<br><br>    dashboards = object({<br>      cp          = string<br>      mesh        = string<br>      performance = string<br>      service     = string<br>    })<br>  })</pre> | `null` | no |
 | <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    enable_dashboards      = bool<br>    scrape_sample_limit    = number<br><br><br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    grafana_dashboard_url = string<br><br>    prometheus_metrics_endpoint = string<br>  })</pre> | `null` | no |
 | <a name="input_ksm_config"></a> [ksm\_config](#input\_ksm\_config) | Kube State metrics configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br><br>    scrape_interval = string<br>    scrape_timeout  = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "kube-state-metrics",<br>  "helm_chart_version": "4.24.0",<br>  "helm_release_name": "kube-state-metrics",<br>  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "kube-system",<br>  "scrape_interval": "60s",<br>  "scrape_timeout": "15s"<br>}</pre> | no |
+| <a name="input_kubeproxy_monitoring_config"></a> [kubeproxy\_monitoring\_config](#input\_kubeproxy\_monitoring\_config) | Config object for kube-proxy monitoring | <pre>object({<br>    flux_gitrepository_name   = string<br>    flux_gitrepository_url    = string<br>    flux_gitrepository_branch = string<br>    flux_kustomization_name   = string<br>    flux_kustomization_path   = string<br><br>    dashboards = object({<br>      grafana_kubeproxy_dashboard_url = string<br>    })<br>  })</pre> | `null` | no |
 | <a name="input_logs_config"></a> [logs\_config](#input\_logs\_config) | Configuration object for logs collection | <pre>object({<br>    cw_log_retention_days = number<br>  })</pre> | <pre>{<br>  "cw_log_retention_days": 90<br>}</pre> | no |
 | <a name="input_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#input\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus Workspace Endpoint | `string` | `""` | no |
 | <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |

modules/eks-monitoring/alerts.tf

@@ -262,6 +262,14 @@ groups:
         annotations:
           description: Kubelet has disappeared from Prometheus target discovery.
           summary: Target disappeared from Prometheus target discovery.
+      - alert: KubeProxyDown
+        expr: absent(up{job="kube-proxy"} == 1)
+        for: 15m
+        labels:
+          severity: critical
+        annotations:
+          description: KubeProxy has disappeared from Prometheus target discovery.
+          summary: Target disappeared from Prometheus target discovery.
       - alert: KubeVersionMismatch
         expr: count by(cluster) (count by(git_version, cluster) (label_replace(kubernetes_build_info{job!~"kube-dns|coredns"}, "git_version", "$1", "git_version", "(v[0-9]*.[0-9]*).*"))) > 1
         for: 15m

modules/eks-monitoring/dashboards.tf

@@ -94,3 +94,25 @@ YAML
   count      = var.enable_adotcollector_metrics ? 1 : 0
   depends_on = [module.external_secrets]
 }
+
+resource "kubectl_manifest" "kubeproxy_monitoring_dashboard" {
+  yaml_body  = <<YAML
+apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
+kind: Kustomization
+metadata:
+  name: ${local.kubeproxy_monitoring_config.flux_kustomization_name}
+  namespace: flux-system
+spec:
+  interval: 1m0s
+  path: ${local.kubeproxy_monitoring_config.flux_kustomization_path}
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: ${local.kubeproxy_monitoring_config.flux_gitrepository_name}
+  postBuild:
+    substitute:
+      GRAFANA_KUBEPROXY_DASH_URL: ${local.kubeproxy_monitoring_config.dashboards.default}
+YAML
+  count      = var.enable_dashboards ? 1 : 0
+  depends_on = [module.external_secrets]
+}

modules/eks-monitoring/locals.tf

@@ -41,14 +41,14 @@ locals {
 
     flux_gitrepository_name   = "aws-observability-accelerator"
     flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
-    flux_gitrepository_branch = "main"
+    flux_gitrepository_branch = "v0.2.0"
     flux_kustomization_name   = "grafana-dashboards-java"
     flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/java"
 
     managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
     prometheus_metrics_endpoint     = "/metrics"
 
-    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/java/default.json"
+    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/java/default.json"
   }
 
   nginx_pattern_config = {
@@ -62,14 +62,14 @@ locals {
 
     flux_gitrepository_name   = "aws-observability-accelerator"
     flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
-    flux_gitrepository_branch = "main"
+    flux_gitrepository_branch = "v0.2.0"
     flux_kustomization_name   = "grafana-dashboards-nginx"
     flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/nginx"
 
     managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
     prometheus_metrics_endpoint     = "/metrics"
 
-    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/nginx/nginx.json"
+    grafana_dashboard_url = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/nginx/nginx.json"
   }
 
   istio_pattern_config = {
@@ -83,7 +83,7 @@ locals {
 
     flux_gitrepository_name   = "aws-observability-accelerator"
     flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
-    flux_gitrepository_branch = "main"
+    flux_gitrepository_branch = "v0.2.0"
     flux_kustomization_name   = "grafana-dashboards-istio"
     flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/istio"
 
@@ -91,10 +91,10 @@ locals {
     prometheus_metrics_endpoint     = "/metrics"
 
     dashboards = {
-      cp          = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/istio/istio-control-plane-dashboard.json"
+      cp          = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/istio/istio-control-plane-dashboard.json"
-      mesh        = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/istio/istio-mesh-dashboard.json"
+      mesh        = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/istio/istio-mesh-dashboard.json"
-      performance = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/istio/istio-performance-dashboard.json"
+      performance = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/istio/istio-performance-dashboard.json"
-      service     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/istio/istio-service-dashboard.json"
+      service     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/istio/istio-service-dashboard.json"
     }
   }
 
@@ -102,14 +102,14 @@ locals {
     # can be overriden by providing a config
     flux_gitrepository_name   = "aws-observability-accelerator"
     flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
-    flux_gitrepository_branch = "main"
+    flux_gitrepository_branch = "v0.2.0"
     flux_kustomization_name   = "grafana-dashboards-apiserver"
     flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/apiserver"
 
     dashboards = {
-      basic           = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/apiserver/apiserver-basic.json"
+      basic           = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/apiserver/apiserver-basic.json"
-      advanced        = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/apiserver/apiserver-advanced.json"
+      advanced        = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/apiserver/apiserver-advanced.json"
-      troubleshooting = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/apiserver/apiserver-troubleshooting.json"
+      troubleshooting = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/apiserver/apiserver-troubleshooting.json"
     }
   }
 
@@ -117,12 +117,25 @@ locals {
     # can be overriden by providing a config
     flux_gitrepository_name   = "aws-observability-accelerator"
     flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
-    flux_gitrepository_branch = "main"
+    flux_gitrepository_branch = "v0.2.0"
     flux_kustomization_name   = "grafana-dashboards-adothealth"
     flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/adot"
 
     dashboards = {
-      health = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/adot/adothealth.json"
+      health = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/adot/adothealth.json"
+    }
+  }
+
+  kubeproxy_monitoring_config = {
+    # can be overriden by providing a config
+    flux_gitrepository_name   = "aws-observability-accelerator"
+    flux_gitrepository_url    = "https://github.com/aws-observability/aws-observability-accelerator"
+    flux_gitrepository_branch = "v0.2.0"
+    flux_kustomization_name   = "grafana-dashboards-kubeproxy"
+    flux_kustomization_path   = "./artifacts/grafana-operator-manifests/eks/kube-proxy"
+
+    dashboards = {
+      default = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/kube-proxy/kube-proxy.json"
     }
   }
 }

modules/eks-monitoring/otel-config/Chart.yaml

@@ -2,5 +2,5 @@ apiVersion: v2
 name: opentelemetry
 description: A Helm chart to install otel operator
 type: application
-version: 0.7.0
+version: 0.8.0
-appVersion: 0.7.0
+appVersion: 0.8.0

modules/eks-monitoring/otel-config/templates/opentelemetrycollector.yaml

@@ -972,112 +972,25 @@ spec:
                     own_namespace: false
                     names:
                     - kube-system
-            - job_name: serviceMonitor/default/kube-prometheus-stack-kube-proxy/0
+
-              honor_timestamps: true
+            - job_name: 'kube-proxy'
-              scrape_interval: {{ .Values.globalScrapeInterval }}
+              honor_labels: true
-              scrape_timeout: {{ .Values.globalScrapeTimeout }}
-              scheme: http
-              authorization:
-                type: Bearer
-                credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
-              follow_redirects: true
-              enable_http2: true
-              relabel_configs:
-              - source_labels: [job]
-                separator: ;
-                regex: (.*)
-                target_label: __tmp_prometheus_job_name
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_service_label_app, __meta_kubernetes_service_labelpresent_app]
-                separator: ;
-                regex: (kube-prometheus-stack-kube-proxy);true
-                replacement: $$1
-                action: keep
-              - source_labels: [__meta_kubernetes_service_label_release, __meta_kubernetes_service_labelpresent_release]
-                separator: ;
-                regex: (kube-prometheus-stack);true
-                replacement: $$1
-                action: keep
-              - source_labels: [__meta_kubernetes_endpoint_port_name]
-                separator: ;
-                regex: http-metrics
-                replacement: $$1
-                action: keep
-              - source_labels: [__meta_kubernetes_endpoint_address_target_kind, __meta_kubernetes_endpoint_address_target_name]
-                separator: ;
-                regex: Node;(.*)
-                target_label: node
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_endpoint_address_target_kind, __meta_kubernetes_endpoint_address_target_name]
-                separator: ;
-                regex: Pod;(.*)
-                target_label: pod
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_namespace]
-                separator: ;
-                regex: (.*)
-                target_label: namespace
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_service_name]
-                separator: ;
-                regex: (.*)
-                target_label: service
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_pod_name]
-                separator: ;
-                regex: (.*)
-                target_label: pod
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_pod_container_name]
-                separator: ;
-                regex: (.*)
-                target_label: container
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_service_name]
-                separator: ;
-                regex: (.*)
-                target_label: job
-                replacement: $$1
-                action: replace
-              - source_labels: [__meta_kubernetes_service_label_jobLabel]
-                separator: ;
-                regex: (.+)
-                target_label: job
-                replacement: $$1
-                action: replace
-              - separator: ;
-                regex: (.*)
-                target_label: endpoint
-                replacement: http-metrics
-                action: replace
-              - source_labels: [__address__]
-                separator: ;
-                regex: (.*)
-                modulus: 1
-                target_label: __tmp_hash
-                replacement: $$1
-                action: hashmod
-              - source_labels: [__tmp_hash]
-                separator: ;
-                regex: "0"
-                replacement: $$1
-                action: keep
               kubernetes_sd_configs:
-                - role: endpoints
+              - role: pod
-                  kubeconfig_file: ""
+              relabel_configs:
-                  follow_redirects: true
+              - action: keep
-                  enable_http2: true
+                source_labels:
-                  namespaces:
+                - __meta_kubernetes_namespace
-                    own_namespace: false
+                - __meta_kubernetes_pod_name
-                    names:
+                separator: '/'
-                    - kube-system
+                regex: 'kube-system/kube-proxy.+'
+              - source_labels:
+                - __address__
+                action: replace
+                target_label: __address__
+                regex: (.+?)(\\:\\d+)?
+                replacement: $$1:10249
+
             - job_name: serviceMonitor/default/kube-prometheus-stack-kube-controller-manager/0
               honor_timestamps: true
               scrape_interval: {{ .Values.globalScrapeInterval }}

modules/eks-monitoring/variables.tf

@@ -96,7 +96,7 @@ variable "flux_gitrepository_url" {
 variable "flux_gitrepository_branch" {
   description = "Flux GitRepository Branch"
   type        = string
-  default     = "main"
+  default     = "v0.2.0"
 }
 
 variable "flux_kustomization_path" {
@@ -465,37 +465,43 @@ variable "grafana_url" {
 variable "grafana_cluster_dashboard_url" {
   description = "Dashboard URL for Cluster Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/cluster.json"
 }
 
 variable "grafana_kubelet_dashboard_url" {
   description = "Dashboard URL for Kubelet Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json"
+}
+
+variable "grafana_kubeproxy_dashboard_url" {
+  description = "Dashboard URL for kube-proxy Grafana Dashboard JSON"
+  type        = string
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/kube-proxy/kube-proxy.json"
 }
 
 variable "grafana_namespace_workloads_dashboard_url" {
   description = "Dashboard URL for Namespace Workloads Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json"
 }
 
 variable "grafana_node_exporter_dashboard_url" {
   description = "Dashboard URL for Node Exporter Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json"
 }
 
 variable "grafana_nodes_dashboard_url" {
   description = "Dashboard URL for Nodes Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodes.json"
 }
 
 variable "grafana_workloads_dashboard_url" {
   description = "Dashboard URL for Workloads Grafana Dashboard JSON"
   type        = string
-  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
+  default     = "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
 }
 
 variable "target_secret_name" {
@@ -517,7 +523,7 @@ variable "enable_adotcollector_metrics" {
 }
 
 variable "adothealth_monitoring_config" {
-  description = "Config object for API server monitoring"
+  description = "Config object for ADOT health monitoring"
   type = object({
     flux_gitrepository_name   = string
     flux_gitrepository_url    = string
@@ -533,3 +539,21 @@ variable "adothealth_monitoring_config" {
   # defaults are pre-computed in locals.tf, provide a full definition to override
   default = null
 }
+
+variable "kubeproxy_monitoring_config" {
+  description = "Config object for kube-proxy monitoring"
+  type = object({
+    flux_gitrepository_name   = string
+    flux_gitrepository_url    = string
+    flux_gitrepository_branch = string
+    flux_kustomization_name   = string
+    flux_kustomization_path   = string
+
+    dashboards = object({
+      grafana_kubeproxy_dashboard_url = string
+    })
+  })
+
+  # defaults are pre-computed in locals.tf, provide a full definition to override
+  default = null
+}