Commit Graph

110 Commits

Author SHA1 Message Date
Rodrigue Koffi 79559d0b10 chore: Make dashboard configurations overridable (#232)
Make dashboard configurations overridable
2023-09-26 11:22:12 -04:00
Venkat Penmetsa d7daeb8c22 EKS Cross Account Observability using central AMP (#213)
* Added example for multi-cluster eks-monitoring and made changes to eks-monitoring module to allow cross-cluster IRSA

* Updated eks-monitoring module's README to add the variable description

* Hard-coded grafana license type in eks-cross-cluster-with-amp/main.tf

* Added README page for eks-cross-account-with-central-amp

* Extracted out the eks/amg creation and modified it to use existing resources

* Update README.md to add multiaccount dashboard png

* Updated README.md and multiaccount.md to add eks multiaccount png

* Updated eks-cross-cluster-with-amp example to disable dashboard creation for cluster 2

* Pre commit changed committed

* Fixed cross-account-observability docs and README.md and added variable for amp_workpace_alias

* Removed extra spacing in multiaccount.md and added precommit suggested changes

* Modified cross-account-observability example to change cross-account-amp-role to snake_case

* Capitalized Terraform string in multiaccount.md and converted iam-role-attach to snake_case

---------

Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
2023-09-01 12:55:44 -04:00
Rodrigue Koffi a05e82b220 Use tags for flux repository (#226) 2023-08-29 10:42:34 +02:00
Jens-Uwe Walther 86cc9489d2 fix(eks-monitoring): kube-proxy scrape config and alert rule (#221)
* changes to just OTEL kube-proxy scrape job

* agree with koffir@ to implement kube-proxy by default

* re-added KubeProxyDown Prometheus alert rule

* added kube-proxy URL related stuff

* corrected kube-proxy stuff according to new structure

* Use SEMVER for flux now

* Run pre-commit

* Bump OTEL chart version

* Substitute URL path with SEMVER

* Run pre-commit

* Added kube-proxy config object

* Run pre-commit

* Use kube-proxy config object for dashboards

---------

Co-authored-by: Jens-Uwe Walther <waltju@amazon.com>
2023-08-25 10:36:35 +02:00
Kevin Tibi 72e0d87fb4 Fix customMetrics issue (#222) 2023-08-23 16:54:52 +02:00
Rodrigue Koffi 4517913549 🧹 Cleanup flux manifests (#220)
* Drop dead variables for kustomization post build

* Cleanup istio config variables

* Cleanup java and nginx variables

* Bump dependencies

* Fix tf bug

* Pre-commit checks
2023-08-18 10:04:53 +02:00
Reshma Mathews 6ca939d923 Adot health monitoring (#212)
* Added adot health metric collection

* corrected alignment related error in Opentelemetry config

* reverted the links

* pre-commit

* Update dashboards.tf

split adothealth dashboard

* split dashboards

* changed dashboard link

* corrected flux_kustomization_path

* Pre-commit
2023-08-17 16:50:34 +02:00
Rodrigue Koffi 0e4c2249c7 Hotfix apiserver links (#219) 2023-08-11 20:25:54 +02:00
Rodrigue Koffi 5bbb4afc41 Split api server dashboards (#216)
* Split api server dashboards

* Pre commit
2023-08-11 18:25:21 +02:00
Rodrigue Koffi 43f1d5865a Temporarily disable lint rule (#218)
* Lint: Skip flaky module

* Force test on the module

* Temporary disable unused_declaration
2023-08-11 17:24:51 +02:00
Rajat Omar 97177eea04 EKS container insights (#204)
* adot-container-insight-tf-code by Rajat Omar

* Documentation and naming convention added

* PR review fixes

* PR CI pipeline fixes

* pr ci run fixes

* added the variables mentioned in ci build

* ci variable fixes

* changed the module name

---------

Co-authored-by: Omar <merajat@3c0630162a5a.ant.amazon.com>
2023-08-09 18:23:54 +02:00
Jens-Uwe Walther 4d88d16cb6 Add Kubernetes admin dashboards for api-server (#208)
* added workqueue related and apiserver_storage_db_total_size_in_bytes (available since K8s/EKS v1.26+) metrics into kube-admin scrape job

* chnanged OTEL scrape config and recording rules file to make apiserver Grafana dashboards working

* added APISERVER Grafana dashboards into variables, Flux kustomization

* added original kube-prom-stack kube-apiserver scrape config into OTEL

* Revert "added original kube-prom-stack kube-apiserver scrape config into OTEL"
because this scrape config does not work :-(

This reverts commit 715db895c656af09430e3d6ada2087bd1828413f.

* added API server troubleshoting dashboard

* removed empty line for clarity

* make API serve rmonitoring default to true but can be disabled as well

* Update eks-apiserver.md

* updated eks-monitoring/README.md according to pre-commit

---------

Co-authored-by: Jens-Uwe Walther <waltju@amazon.com>
2023-08-09 16:53:16 +02:00
Rodrigue Koffi 26371d5210 Improve tracing (#195)
* Dropping daemonset option

* Update adot loglevel config

* Enable tracing by default

* Init tracing docs

* Add tracing docs

* Update tracing.md

* Update collector endpoint

* Fix tracing instructions
2023-08-07 20:59:18 +02:00
Kevin Tibi e6083b163f Extend configuration for customMetrics (#202) 2023-08-07 12:42:16 +02:00
Dave Shimko 1ceecb3041 Add EKS Istio Dashboards (#194)
* Update readme

* checking in new readme example

* new istio readme

* Updated mkdocs.yml

* checking in readme files

* checking in all changes

* Test

* Test

* Test

* Testing

* Testing

* test

* Update main.tf

* Update outputs.tf

* Adding Istio dashboards

* Updating URL's

* Removing dashboards

* Updates from pre-commit

* Update istio.md

* Removing empty file

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update README.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update istio.md

* Update README.md

* Update README.md

* Istio Documentation Updates

Added istioctl prereq, updated dashboards image

* Updated setup and cleanup of Istio Bookinfo

Updated Istio Bookinfo sample app instructions, included clean up of Bookinfo resources

* Updating image and removing advanced configuration

-Switching image to Github generated link
-Removing advanced configuration section

---------

Co-authored-by: Prithvi Reddy <ypreddy@amazon.com>
Co-authored-by: Prithvi Reddy <47993564+preddy727@users.noreply.github.com>
2023-07-27 18:44:52 +02:00
Rodrigue Koffi 7f60bd72d6 Cleans up alert rules (#192) 2023-07-15 15:55:31 +02:00
Rodrigue Koffi f29113f4da Fluentbit improvements (#189)
* Bump chart version

* Take only log message

* Add multiline parsing

* Add variable config for refresh interval
2023-07-13 18:58:14 +02:00
Rodrigue Koffi 08b4be1490 ADOT collector scrape improvements (#188)
* Bump default scrape interval to 120s

* fixup! Bump default scrape interval to 120s
2023-07-07 09:44:45 -04:00
Rodrigue Koffi 739b581988 K8s operator metrics (#185)
* Add kube admin metrics

* Update chart version

* Variable verbosity level for adot collector
2023-06-29 08:52:41 -04:00
Rodrigue Koffi a7633d1548 fix(doc): EKS custom metrics collection (#184)
* Quickfix eks custom metrics collection

* Fix ssm overwrite required
2023-06-21 18:17:52 +01:00
Ela AWS c705238f0c Fixing Documentation issues with Secrets Manager (#183)
* Fixing Documentation issues with Secrets Manager

* Fixing Documentation issues with Secrets Manager
2023-06-16 17:25:32 +02:00
Rodrigue Koffi de8c1233ee eks-monitoring minor fixes (#182)
* Grafana URL should not be required at all times

* Enable ADOT OTLP receiver on 0.0.0.0

* Fix variable name

* Update helm release version
2023-06-14 17:23:34 +02:00
Ela AWS 905400a4bb Fixing SSM Parameter Issue (#180) 2023-06-12 20:46:18 +02:00
Rodrigue Koffi 86d0831248 Remove Grafana provider for EKS (#179)
* Remove Grafana provider for EKS

* Drop provider

* Add provider constraint config
2023-06-12 20:40:54 +02:00
Rodrigue Koffi fa38a90efc Move all dashboards to GitOps (#175)
* Typo

* Remove Grafana provider

* Temp: move dashbaords to gitOps

* Move external labels to resource attributes

* Avoid DDoS with using 0.0.0.0

* Pre-commit

* Transition in two steps

Will need to remove provider in a separate version to provide a transition path as removing this will break terraform and leave orphans in the state

* Move patterns' dashboards creation to gitOps

Standardize config objects for patterns as well

* Pre-commit

* Create AMP dashboard from external source with Grafana provider

* Fix deprecated option

* Fix Flux requirements

* Run pre-commit

* Update example with operator

* Cleanup examples

* Update multicluster example

* Update multicluster example

* Drop dead variable

* Update docs

* Change GitOps branch name

* Update docs

* Replacing Secrets Manager to SSM to store Grafana API Key (#178)

* Fixing SSM

* Fixing SSM

* Replacing Secrets Manager with SSM

* Replacing Secrets Manager with SSM

* Update architecture diagram

* Update architecture diagram

* Update README.md

* Update index.md

* Fixing Grafana Operator Version

* Fix multicluster example

* Update docs

---------

Co-authored-by: Ela AWS <51791117+elamaran11@users.noreply.github.com>
Co-authored-by: Elamaran Shanmugam <elamaran.shan@gmail.com>
2023-06-12 18:00:42 +02:00
Rodrigue Koffi 3979910e50 Fix: collector failing to scrape targets (#174)
* Move external labels to resource attributes

* Avoid DDoS with using 0.0.0.0

* Pre-commit
2023-06-07 09:10:35 -04:00
Ela AWS 9b6b234922 Fixing breaking changes for path and dash urls (#173)
* Fixing breaking changes for path and dash urls

* Fixing breaking changes for path and dash urls

* Fixing breaking changes for path and dash urls

* Fixing breaking changes for path and dash urls
2023-06-06 09:59:51 +02:00
Rodrigue Koffi 1f16bec27f Update docs (#172) 2023-06-06 08:19:22 +02:00
Ela AWS 0392f5bda0 Using GitOps with Grafana Operator to create Grafana Dashboard with Flux. (#164)
* Grafana With GitOps Feature

* Grafana With GitOps Feature

* Grafana With GitOps Feature

* Fix setup logs retention policy (#169)

* Fixing GitOps Repo

* Commenting out the NodeExp Dash

* Commenting out the NodeExp Dash

* Adding all Grafana Dashboards

* Adding all Grafana Dashboards

* Fixing Grafana Operator Version and cleaning full boards

* Fixing Grafana Operator Version and cleaning full boards

* Fixing Grafana Operator Version and cleaning full boards

* Fixing Grafana Operator Version and cleaning full boards

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

* Fixing Grafana Operator Version and cleaning full boards and PR Issues

---------

Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
2023-06-05 11:11:30 +02:00
Rodrigue Koffi 0d1c2ef8dd Add OTLP receiver for custom metrics (#170) 2023-06-02 13:26:03 -04:00
Rodrigue Koffi 9085af9285 Fix setup logs retention policy (#169) 2023-06-02 11:54:42 +02:00
Rodrigue Koffi 0c4407638a Update dependencies (#167)
* Lint file

* Update dependencies

* Update badges

* Bump terraform versions

* Temp allow run on current branch

* Run tests on push
2023-06-01 10:55:35 +02:00
Ela AWS 5b52c7b8e1 GitOps support with Grafana Operator, External Secrets and FluxCD (#151)
* Adding FluxCD and Grafana Operator Addons

* Adding Helm release for Grafana Operator

* Adding External Secrets

* removing spot options

* Fixing Documentation
---------
2023-04-14 10:20:24 +02:00
Vijay Chintalapati caeebe0e1a Make creation of Grafana Data Source and Folder configurable (#145)
* Made the cluster variable visible in dashboards

* Made the repo reusable with multiple EKS clusters

* Relabled the input variable from create_grafana_data_source to create_prometheus_data_source
2023-04-06 11:04:29 +02:00
Soto Sugita cd28d9975a Bump node-exporter chart to 4.14.0 (#143)
* Bump node-exporter chart to 4.14.0

* terraform docs
2023-04-05 11:19:18 +02:00
Rodrigue Koffi 4019db6cd7 Support Fluentbit to CloudWatch logs (#140)
* Import and customize fluenbit add-on

* Enable fluent bit logs

* Bump helm addon version

* Dropping account id as it seems to create scraping errors

* Create separate log groups per namespace

* Apply pre-commit

* Remove conflicting global label

* Add config object for logs

* Enable logs in examples

* Add logs docs

* Fix broken link

* Add screenshots

* Update docs

* Typos
2023-03-30 15:48:40 +02:00
Luis Amaral 1ee09d2c1a Lsa/resource name length i#118 (#135)
* Validation added to cluster_name variable

* Update ref to eks-cluster-with-vpc on v4.13.1 (#121)

* Added cluster_name validation with message (#118)

* Error message typo fixed to follow the standards.

* Adding requested new line at the end
2023-03-28 09:48:09 +02:00
Rodrigue Koffi cdd0eb7beb Fix drill down links in dashboards (#137) 2023-03-24 22:12:50 +01:00
Rodrigue Koffi daed34db80 Compose EKS monitoring modules (#115)
* Move modules around

* Update amp billing source

* Merge Java monitoring to EKS

* Update docs

* Merge nginx pattern

* Pre-commit

* Add save and test URL output

* Move EKS dependencies to EKS monitoring module

* update docs

* Update examples and docs

* Add java doc

* Add NGINX doc

* Update nginx doc

* Fix amp monitoring example path

* Fix pre-commit

* Todo: move to main after merge

* Update docs, fix tags
2023-02-20 18:36:08 +01:00
Rodrigue Koffi b617f8d04e Enhance Java dashboard (#114)
* Enhance Java dashboards visuals

- migrate panels versions
- apply rate functions on counters
- dynamic datasource

* Update screenshot

* Update dashboard screenshot

* fixup! Update dashboard screenshot
2023-02-09 15:18:45 +01:00
Rodrigue Koffi 0c0ac15496 impr: Tracing to AWS X-Ray config options (#110)
* Remove duplicate exposed ports in ADOT 0.61+

* Sync ADOT collector to ADOT addon version

* Bump helm version

* Provide configuration options for tracing
2023-02-06 20:31:41 +01:00
Mark Beacom 2e3011da1c Adjust Terraform defaults and usage for tflint & validation checks (#104)
* Fix #103 Adjust Terraform defaults and usage for tflint and validation checks

* Remove temporarily unused validation
2023-02-01 11:19:31 +01:00
Soto Sugita 802c82951e Bump KSM chart to 4.24.0 (#80)
* Update default helm chart version

Update default helm chart version for node-exporter and kube-state-metrics

* update README

* revert node-exporter chart version
2023-01-24 09:28:55 +01:00
Rodrigue Koffi c726f508b5 Tracing support with custom metrics collection (#67)
* Init tracing support

* Update xray exporter

* Remove duplicate metrics only on daemon set

* Add support for custom metrics collection

* Remove experimental feature

* Rename config items

* Update requirements for tracing support
2023-01-19 21:24:39 +01:00
Soto Sugita db0cb5d7aa fix: variable description for cert manager (#73)
Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
2022-12-14 18:12:20 +01:00
Soto Sugita 8086b3f9d7 fix: typo (#72)
Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
2022-12-14 17:23:19 +01:00
Rodrigue Koffi f6a4c31785 fix: eks permissions (#71)
* Update bug report template

* Fix net permissions
2022-12-14 17:10:52 +01:00
Kevin Lewin 07545aca27 AMP Observability Pattern (#55)
* working first run

* removing core module dependencies

* adding CW datasource

* alarms MVP

* readmes

* Adding Screenshot

* Adding billing note

* adding billing module

* Revert "adding billing module"

This reverts commit 40d667e37db1036cd71a471ef2fde83ec02aaa13.

reverting

* adding billing module

* Updating Screenshot

* resolving feedback

* removing unused modules

* fmt

* Support for tf 1.3.x

* removing unused variables

* support alarms for multiple workspaces

* Updating Readme

* amp to managed prometheus

* sub-module

* Fix pre-commit

Co-authored-by: Rodrigue Koffi <bonclay7@users.noreply.github.com>
2022-12-09 10:49:55 -05:00
Rodrigue Koffi 0f772e9e34 💅🏾 Improvements: Infrastructure dashboards (#60)
* Improve dashboards panels

* Change lines overlay and migrate table panels

* Update README.md
2022-11-02 09:39:05 +01:00
Rodrigue Koffi 54c810e1a9 Add support for tf 1.3+ (#58)
* Add support for tf 1.3+

* Bump min version

* Bump blueprint version
2022-10-25 15:57:37 +02:00