public-ds-cluster-env
aws 환경 구축 시 사용하는 파일을 정리한다.
directory 구조
- __docs: 문서파일 저장.
- architecture : aws 구성도
- kops: aws kubernetes cluster 구성
- packer: aws ami 생성
- script: lambda 실행, key 배포 등 스크립트
- terraform: terraform code
노드 목록 [24]
| name | ip | taints_key | instance_group | cpu | memory | zone | instance_type | os | k8s_ver | runtime_ver |
|---|---|---|---|---|---|---|---|---|---|---|
| i-01da83c436c0fb834 | 172.24.13.47/23 | - | k8s-prod-process-c2 | 4 | 16 | ap-northeast-2c | m6i.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-04abf3bdccadaf05c | 172.24.13.138/23 | - | k8s-prod-tmp | 8 | 31 | ap-northeast-2c | m5a.2xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0628e1521b484fde1 | 172.24.11.188/23 | control-plane | - | 2 | 4 | ap-northeast-2b | c5a.large | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-05a497c27d7f65d78 | 172.24.12.22/23 | control-plane | - | 2 | 4 | ap-northeast-2c | c5a.large | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0446ae551deac0b98 | 172.24.8.218/23 | control-plane | - | 2 | 4 | ap-northeast-2a | c5a.large | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-07aebcfff479c0331 | 172.24.12.210/23 | prod/data-druid | k8s-prod-data-druid-small | 4 | 16 | ap-northeast-2c | m6i.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-07046cde77753a2d4 | 172.24.12.252/23 | prod/data-druid | k8s-prod-data-druid-c | 16 | 62 | ap-northeast-2c | m5a.4xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0c370e3211468e6e5 | 172.24.12.96/23 | prod/data-druid | k8s-prod-data-druid-middle | 8 | 31 | ap-northeast-2c | m5a.2xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0f2de40153ccd640e | 172.24.13.2/23 | prod/data-druid | k8s-prod-data-druid-middle | 8 | 31 | ap-northeast-2c | m5a.2xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0a36b91bce0b86b6f | 172.24.12.142/23 | prod/data-druid | k8s-prod-data-druid-c | 16 | 62 | ap-northeast-2c | m5a.4xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-06e3ca9b0a909e7bd | 172.24.13.62/23 | prod/data-es | k8s-prod-data-es-c | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0fd0e6d60f02a7b14 | 172.24.13.29/23 | prod/data-es | k8s-prod-data-es-c2 | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0df4bd76188c35590 | 172.24.13.100/23 | prod/data-es | k8s-prod-data-es-c3 | 4 | 16 | ap-northeast-2c | m6i.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0a6a549b5bf51efbc | 172.24.12.207/23 | prod/data-kafka | k8s-prod-data-kafka-c | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0097ed484d8513a71 | 172.24.12.66/23 | prod/data-kafka | k8s-prod-data-kafka-c3 | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-092354f3890ecf207 | 172.24.13.215/23 | prod/data-kafka | k8s-prod-data-kafka-c2 | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0d3e74b8884948aa3 | 172.24.13.32/23 | prod/process | k8s-prod-process-c | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0abc091cf1fc5084f | 172.24.12.23/23 | prod/process | k8s-prod-process-c4 | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-08cea9eddc5260b4d | 172.24.13.119/23 | prod/process | k8s-prod-process-c2 | 4 | 16 | ap-northeast-2c | m6i.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-00fdae3a253e720bf | 172.24.12.227/23 | prod/process | k8s-prod-process-c | 4 | 16 | ap-northeast-2c | m5a.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0ed4e23a4e2da73a4 | 172.24.9.103/23 | prod/rel-process | k8s-rel-process-a | 4 | 16 | ap-northeast-2a | m4.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0ccf664103c7f172b | 172.24.8.119/23 | prod/rel-process | k8s-rel-process-a | 4 | 16 | ap-northeast-2a | m4.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0a012ed882df30c1e | 172.24.9.57/23 | prod/rel-process | k8s-rel-process-a | 4 | 16 | ap-northeast-2a | m4.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
| i-0a71a915a2664b7b7 | 172.24.9.246/23 | prod/rel-process | k8s-rel-process-a | 4 | 16 | ap-northeast-2a | m4.xlarge | Ubuntu 20.04.4 LTS | v1.25.2 | containerd://1.6.8 |
리소스 목록 [93]
| name | kind | count | reqeust_cpu | reqeust_mem | limit_cpu | limit_mem |
|---|---|---|---|---|---|---|
| promtail | DaemonSet | 24 | ||||
| dsk-log-agent | DaemonSet | 20 | 250m | 512Mi | 500m | 1G |
| prometheus-prometheus-node-exporter | DaemonSet | 24 | ||||
| aws-cloud-controller-manager | DaemonSet | 3 | 200m | |||
| dsk-trace-agent | DaemonSet | 24 | ||||
| calico-node | DaemonSet | 24 | 100m | |||
| kops-controller | DaemonSet | 3 | 50m | 50Mi | ||
| ebs-csi-node | DaemonSet | 24 | ||||
| ingress-nginx-controller | DaemonSet | 5 | 100m | 90Mi | ||
| dsk-base-agent | DaemonSet | 24 | 50m | 128Mi | 100m | 250Mi |
| druid-exporter-prometheus-druid-exporter | Deployment | 0 | ||||
| druid-router | Deployment | 1 | ||||
| dsk-kubernetes-agent | Deployment | 1 | 100m | 512Mi | 1 | 1000Mi |
| dsk-otel | Deployment | 1 | 250m | 512M | 1 | 2048M |
| dsk-postgres-agents-pg-1 | Deployment | 1 | 100m | 512Mi | 1 | 1000Mi |
| dsk-vault-agent | Deployment | 1 | ||||
| ebs-csi-controller | Deployment | 2 | ||||
| es-exporter-prometheus-elasticsearch-exporter | Deployment | 1 | ||||
| prod-dsk-datagate-dsk-datagate-plan | Deployment | 1 | 100m | 100M | 250m | 250M |
| calico-kube-controllers | Deployment | 1 | ||||
| cert-manager | Deployment | 1 | ||||
| cert-manager-cainjector | Deployment | 1 | ||||
| cert-manager-webhook | Deployment | 1 | ||||
| coredns | Deployment | 2 | 100m | 70Mi | 170Mi | |
| coredns-autoscaler | Deployment | 1 | 20m | 10Mi | ||
| dns-controller | Deployment | 1 | 50m | 50Mi | ||
| druid-broker | Deployment | 2 | 26Gi | 26Gi | ||
| druid-coordinator | Deployment | 1 | 1Gi | 8Gi | ||
| prod-dsk-metric-base-agg-stream-1m | Deployment | 3 | 500m | 1Gi | 1 | 2Gi |
| prod-dsk-metric-base-agg-stream-5m | Deployment | 0 | 500m | 1Gi | 1 | 2Gi |
| prod-dsk-metric-base-flat-stream | Deployment | 4 | 500m | 500Mi | 1 | 1Gi |
| prod-dsk-metric-custom-flat-stream | Deployment | 3 | ||||
| prod-dsk-notification-api | Deployment | 1 | ||||
| prod-dsk-tagging-connector-group1 | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-tagging-connector-group2 | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-tagging-connector-group3 | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-tagging-connector-group4 | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-tagging-connector-jaeger-process | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-tagging-connector-manifest | Deployment | 1 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-trace-stream | Deployment | 3 | ||||
| prod-dsk-ui | Deployment | 1 | ||||
| prod-dsk-usergate-api | Deployment | 1 | ||||
| prometheus-kube-state-metrics | Deployment | 1 | 10m | 32Mi | 100m | 64Mi |
| prometheus-prometheus-pushgateway | Deployment | 1 | 100m | 30Mi | 200m | 50Mi |
| prometheus-server | Deployment | 1 | ||||
| strimzi-cluster-operator | Deployment | 1 | 200m | 384Mi | 1 | 384Mi |
| vault-agent-injector | Deployment | 1 | ||||
| kafka-akhq | Deployment | 1 | ||||
| kafka-kafka-ui | Deployment | 1 | ||||
| metrics-server | Deployment | 1 | 100m | 200Mi | ||
| mongo-dsk-mongodb-sharded-mongos | Deployment | 1 | ||||
| mongo-manifest-mongodb-sharded-mongos | Deployment | 1 | ||||
| nfs-provider-prod-nfs-subdir-external-provisioner | Deployment | 1 | ||||
| prod-dsk-agent-api | Deployment | 1 | ||||
| prod-dsk-agentmanager-api | Deployment | 1 | ||||
| prod-dsk-alert-api | Deployment | 1 | ||||
| prod-dsk-apm-api | Deployment | 1 | 100m | 256Mi | 250m | 512Mi |
| prod-dsk-app-sender | Deployment | 1 | ||||
| prod-dsk-backoffice | Deployment | 1 | ||||
| prod-dsk-chart-api | Deployment | 7 | 500m | 512Mi | 1 | 1Gi |
| prod-dsk-dashboard-api | Deployment | 1 | ||||
| prod-dsk-database-api | Deployment | 3 | 100m | 256Mi | 250m | 512Mi |
| prod-dsk-database-plan-stream | Deployment | 1 | ||||
| prod-dsk-datagate-dsk-datagate-manifest | Deployment | 1 | 100m | 100M | 200m | 200M |
| prod-dsk-datagate-dsk-datagate-metric | Deployment | 5 | 250m | 250M | 500m | 500M |
| prod-dsk-datagate-dsk-datagate-trace | Deployment | 2 | 100m | 250M | 200m | 500M |
| prod-dsk-infrastructure-api | Deployment | 1 | 100m | 256Mi | 250m | 512Mi |
| prod-dsk-kafka2storage-database-plan-stream | Deployment | 1 | 300m | 500Mi | 500m | 1G |
| prod-dsk-kafka2storage-database-stat-query | Deployment | 1 | 300m | 500Mi | 500m | 1G |
| prod-dsk-kafka2storage-manifest-original | Deployment | 1 | 300m | 500Mi | 500m | 1G |
| prod-dsk-kafka2storage-trace-span-stream | Deployment | 1 | 300m | 500Mi | 500m | 1G |
| prod-dsk-krakend | Deployment | 3 | 200m | 256Mi | 500m | 512Mi |
| prod-dsk-log-api | Deployment | 1 | ||||
| prod-dsk-loggate | Deployment | 4 | 100m | 500Mi | 200m | 1500Mi |
| elasticsearch-master | Stateful_Set | 3 | 2 | 12Gi | 2 | 12Gi |
| kafka | Stateful_Set | 3 | 2 | 6000Mi | 3 | 10000Mi |
| zookeeper | Stateful_Set | 2 | 100m | 512Mi | 200m | 1000Mi |
| vault | Stateful_Set | 1 | ||||
| keycloak | Stateful_Set | 1 | ||||
| druid-zookeeper | Stateful_Set | 3 | 250m | 256Mi | ||
| druid-postgresql | Stateful_Set | 1 | 250m | 256Mi | ||
| mongo-dsk-mongodb-sharded-configsvr | Stateful_Set | 1 | ||||
| druid-historical | Stateful_Set | 2 | 26Gi | 26Gi | ||
| mongo-dsk-mongodb-sharded-shard0-data | Stateful_Set | 1 | ||||
| mongo-dsk-mongodb-sharded-shard1-data | Stateful_Set | 1 | ||||
| mongo-manifest-mongodb-sharded-configsvr | Stateful_Set | 1 | ||||
| rabbitmq | Stateful_Set | 1 | ||||
| mongo-manifest-mongodb-sharded-shard0-data | Stateful_Set | 1 | ||||
| mongo-manifest-mongodb-sharded-shard1-data | Stateful_Set | 1 | ||||
| prometheus-alertmanager | Stateful_Set | 1 | 10m | 32Mi | 100m | 128Mi |
| postgresql | Stateful_Set | 1 | 250m | 256Mi | ||
| redis-master | Stateful_Set | 1 | ||||
| druid-middle-manager | Stateful_Set | 2 | 1Gi | 30Gi |
서비스 목록 (NodePort) [37]
| name | namespace | type | Node Port | age |
|---|---|---|---|---|
| rabbitmq | dsk-middle | NodePort | 30038 | 5 mons 9 days 17:47:44 |
| druid-router | dsk-middle | NodePort | 30076 | 3 mons 22 days 17:46:40 |
| kafka-broker | dsk-middle | NodePort | 30094 | 3 mons 1 day 15:18:07 |
| kafka-broker-global | dsk-middle | NodePort | 30095 | 3 mons 1 day 15:18:07 |
| keycloak | dsk-middle | NodePort | 30100 | 5 mons 7 days 17:49:52 |
| keycloak | dsk-middle | NodePort | 30101 | 5 mons 7 days 17:49:52 |
| mongo-manifest-mongodb-sharded | dsk-middle | NodePort | 30111 | 23 days 08:43:06 |
| mongo-dsk-mongodb-sharded | dsk-middle | NodePort | 30112 | 23 days 07:30:26 |
| redis-master | dsk-middle | NodePort | 30229 | 5 mons 9 days 17:47:31 |
| elasticsearch-master | dsk-middle | NodePort | 30433 | 19 days 19:28:41 |
| rabbitmq | dsk-middle | NodePort | 30523 | 5 mons 9 days 17:47:44 |
| rabbitmq | dsk-middle | NodePort | 30565 | 5 mons 9 days 17:47:44 |
| rabbitmq | dsk-middle | NodePort | 30655 | 5 mons 9 days 17:47:44 |
| druid-router | dsk-middle | NodePort | 30907 | 3 mons 22 days 17:46:40 |
| rabbitmq | dsk-middle | NodePort | 31021 | 5 mons 9 days 17:47:44 |
| kafka-akhq | dsk-middle | NodePort | 31100 | 3 mons 1 day 15:18:07 |
| kafka-kafka-ui | dsk-middle | NodePort | 31101 | 3 mons 1 day 15:18:07 |
| elasticsearch-master | dsk-middle | NodePort | 31110 | 19 days 19:28:41 |
| prometheus-server | prometheus | NodePort | 31132 | 10 days 16:32:32 |
| druid-exporter-prometheus-druid-exporter | dsk-middle | NodePort | 31282 | 13 days 17:10:30 |
| prod-dsk-datagate-dsk-datagate-trace | dsk-datagate-prod | NodePort | 31300 | 4 mons 4 days 14:42:53 |
| prod-dsk-datagate-dsk-datagate-manifest | dsk-datagate-prod | NodePort | 31301 | 4 mons 16 days 15:58:38 |
| prod-dsk-datagate-dsk-datagate-metric | dsk-datagate-prod | NodePort | 31302 | 4 mons 16 days 15:58:38 |
| prod-dsk-datagate-dsk-datagate-plan | dsk-datagate-prod | NodePort | 31303 | 4 mons 16 days 15:58:38 |
| prod-dsk-loggate | dsk-datagate-prod | NodePort | 31304 | 4 mons 10 days 16:12:27 |
| prod-dsk-loggate | dsk-datagate-prod | NodePort | 31305 | 4 mons 10 days 16:12:27 |
| prod-dsk-backoffice | dsk-api-prod | NodePort | 31523 | 1 mon 16 days 21:54:23 |
| rabbitmq | dsk-middle | NodePort | 31987 | 5 mons 9 days 17:47:44 |
| postgresql | dsk-middle | NodePort | 32098 | 5 mons 9 days 17:48:07 |
| druid-broker | dsk-middle | NodePort | 32200 | 3 mons 22 days 17:46:40 |
| druid-broker | dsk-middle | NodePort | 32316 | 3 mons 22 days 17:46:40 |
| kafka-outside-0 | dsk-middle | NodePort | 32400 | 3 mons 1 day 15:18:07 |
| kafka-outside-1 | dsk-middle | NodePort | 32401 | 3 mons 1 day 15:18:07 |
| kafka-outside-2 | dsk-middle | NodePort | 32402 | 3 mons 1 day 15:18:07 |
| kafka-global-0 | dsk-middle | NodePort | 32500 | 3 mons 1 day 15:18:07 |
| kafka-global-1 | dsk-middle | NodePort | 32501 | 3 mons 1 day 15:18:07 |
| kafka-global-2 | dsk-middle | NodePort | 32502 | 3 mons 1 day 15:18:07 |