[AWS] External Secrets

kubernetes-external-secrets

github :  

GitHub - external-secrets/kubernetes-external-secrets: Integrate external secret management systems with Kubernetes

Example Architecture)

 

 

• NodeJS 기반의 ExternalSecrets CRD를 통해 Backend의 Secret 저장소들과 연동
• AWS Secrets Manager, AWS Parameter Store, Hashicorp Vault, AKS, GKS, Alibaba 등의 Secret Manager와 연동
  • ex) 

    

    kubernetes-external-secrets/examples at master · external-secrets/kubernetes-external-secrets
• 사용이 간편하며 기존 Secret Object를 사용하는 Pod들의 변경없이 사용 가능
• IAM Integration
• AWS 내부 EKS Slack Channel 및 Discussion 사이트 확인 결과 가장 많이 사용되고 있으며 가장 많이추천하고 있었음

with AWS Secretes Manager

Check AWS Secrets Manager

# Check Sample AWS SecretsManager $ aws secretsmanager list-secrets {     "SecretList": [         {             "ARN": "arn:aws:secretsmanager:ap-northeast-2:373394553405:secret:db-credential-SSstNC",             "Name": "db-credential",             "Description": "RDS Credential",             "LastChangedDate": "2021-04-04T09:00:01.123000+09:00",             "LastAccessedDate": "2021-04-04T09:00:00+09:00",             "Tags": [],             "SecretVersionsToStages": {                 "48ca88d2-badf-4a6a-bd2a-4ddaab0cc52a": [                     "AWSCURRENT"                 ]             }         }     ] } $ aws secretsmanager get-secret-value --secret-id db-credential {     "ARN": "arn:aws:secretsmanager:ap-northeast-2:373394553405:secret:db-credential-SSstNC",     "Name": "db-credential",     "VersionId": "48ca88d2-badf-4a6a-bd2a-4ddaab0cc52a",     "SecretString": "{\"username\":\"admin\",\"password\":\"woowaP@$$w0rd\",\"engine\":\"mysql\",\"host\":\"database-1.cducwci7wujy.ap-northeast-2.rds.amazonaws.com\",\"port\":3306,\"dbInstanceIdentifier\":\"database-1\"}",     "VersionStages": [         "AWSCURRENT"     ],     "CreatedDate": "2021-03-24T15:54:49.662000+09:00" }

 

IAM Policy for ServiceAccount

{   "Version": "2012-10-17",   "Statement": [     {       "Effect": "Allow",       "Action": [         "secretsmanager:GetResourcePolicy",         "secretsmanager:GetSecretValue",         "secretsmanager:DescribeSecret",         "secretsmanager:ListSecretVersionIds",         "secretsmanager:GetRandomPassword",         "secretsmanager:ListSecrets"              ],       "Resource": "*"     },     {           "Effect": "Allow",       "Action": "ssm:GetParameter",       "Resource": "*"         }   ] }

 

Create ExternalSecret Custom Resource

external-secrets-rds-sample.yaml

# Create ExternalSecret with Secrets Manager Backend apiVersion: "kubernetes-client.io/v1" kind: ExternalSecret metadata:   name: db-credential spec:   backendType: secretsManager   # optional: specify role to assume when retrieving the data   dataFrom:     - db-credential   # optional: specify a template with any additional markup you would like added to the downstream Secret resource.   # This template will be deep merged without mutating any existing fields. For example: you cannot override metadata.name.   template:     metadata:       annotations:         type: rds       labels:         backend: secretsManager

 

 

Check Secrets

# Check Kubernetes Secrets $ kubectl get secrets db-credential -o yaml apiVersion: v1 data:   dbInstanceIdentifier: ZGF0YWJhc2UtMQ==   engine: bXlzcWw=   host: ZGF0YWJhc2UtMS5jZHVjd2NpN3d1ankuYXAtbm9ydGhlYXN0LTIucmRzLmFtYXpvbmF3cy5jb20=   password: d29vd2FQQCQkdzByZA==   port: MzMwNg==   username: YWRtaW4= kind: Secret metadata:   annotations:     type: rds   creationTimestamp: "2021-04-03T00:24:42Z"   labels:     backend: secretsManager . . .   name: db-credential   namespace: default   ownerReferences:   - apiVersion: kubernetes-client.io/v1     controller: true     kind: ExternalSecret     name: db-credential type: Opaque

 

CloudTrail

• KMS Decrypt
• GetSecretValue

{     "eventVersion": "1.08",     "userIdentity": {         "type": "AssumedRole",         "principalId": "AROAVN4AMXI6V7ZXVFPWM:token-file-web-identity",         "arn": "arn:aws:sts::{{ .aws.accountId }}:assumed-role/external-secrets/token-file-web-identity",         "accountId": "{{ .aws.accountId }}",         "accessKeyId": "ASIAVN4AMXI66634QBOS",         "sessionContext": {             "sessionIssuer": {                 "type": "Role",                 "principalId": "AROAVN4AMXI6V7ZXVFPWM",                 "arn": "arn:aws:iam::{{ .aws.accountId }}:role/external-secrets",                 "accountId": "{{ .aws.accountId }}",                 "userName": "external-secrets"             },             "webIdFederationData": {                 "federatedProvider": "arn:aws:iam::{{ .aws.accountId }}:oidc-provider/oidc.eks.ap-northeast-2.amazonaws.com/id/BDEF1599366F79B51AAA16E6B6C0D958",                 "attributes": {}             },             "attributes": {                 "mfaAuthenticated": "false",                 "creationDate": "2021-04-05T04:16:29Z"             }         }     },     "eventTime": "2021-04-05T04:28:47Z",     "eventSource": "secretsmanager.amazonaws.com",     "eventName": "GetSecretValue",     "awsRegion": "ap-northeast-2",     "sourceIPAddress": "13.209.199.215",     "userAgent": "aws-sdk-nodejs/2.628.0 linux/v12.19.0 promise",     "requestParameters": {         "secretId": "db-credential",         "versionStage": "AWSCURRENT"     },     "responseElements": null,     "requestID": "12f8c024-208a-423d-a96e-01b5db1d49c1",     "eventID": "bce980c3-5fdc-4dfb-a601-ff2bb6a7a908",     "readOnly": true,     "eventType": "AwsApiCall",     "managementEvent": true,     "eventCategory": "Management",     "recipientAccountId": "{{ .aws.accountId }}" }

 

• CSI Driver 를 통해 Secrets Store를 제공하는 방식으로 (21.4.26 Launching)
  • https://aws.amazon.com/ko/about-aws/whats-new/2021/04/aws-secrets-manager-delivers-provider-kubernetes-secrets-store-csi-driver/
  • https://aws.amazon.com/blogs/security/how-to-use-aws-secrets-configuration-provider-with-kubernetes-secrets-store-csi-driver/
•  

  

  containers-roadmap · aws/containers-roadmap

• 

  [EKS] [request]: AWS Secrets Manager / SSM Parameter Store · Issue #168 · aws/containers-roadmap

• 

  Introduction - Secrets Store CSI Driver

• 

  GitHub - kubernetes-sigs/secrets-store-csi-driver: Secrets Store CSI driver for Kubernetes secrets - Integrates secrets stores with Kubernetes via a CSI volume.

기타 관련 프로젝트)

aws-ssm

• github : 

  

  GitHub - cmattoon/aws-ssm: Populates Kubernetes Secrets from AWS Parameter Store
• secret에 anootation을 통해 AWS ssm parameter store 값 사용

kubesec

• github: 

  

  GitHub - shyiko/kubesec: Secure Secret management for Kubernetes (with gpg, Google Cloud KMS and AWS KMS backends)
• 데이터를 Secret Object 또는 manifest에 저장하기 전에 encrypt (gpg, AWS/GCP KMS와 연동)

# Encrypt kubesec encrypt secret.yml# same as above but output is written back to secret.yml (instead of stdout) kubesec encrypt -i secret.yml # encrypt with AWS KMS key ("aws:" prefix is optional)## NOTE: you might need to `aws configure` (if you don't have ~/.aws/credentials already)## http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html kubesec encrypt --key=aws:<ARN of AWS KMS key> secret.yml kubesec encrypt \   --key=aws:arn:aws:kms:us-west-1:000000000000:key/00000000-0000-0000-0000-000000000000 secret.yml    # Decrypt # decrypt a Secret # (usually combined with kubectl (`kubesec decrypt secret.enc.yml | kubectl apply -f -`)) kubesec decrypt secret.enc.yml  apiVersion: v1 kind: Secret metadata:   name: myapp-default-0 type: Opaque data:   KEY: TUFkWD1iuKs=.O....D...=   ANOTHER_KEY: iOy1nf90+M6FrrEIoymN6cOSUYM=.E...=.q...=

• 런타임에 Application에서 데이터를 decrypt해서 사용해야함

secret-sync-operator

• github: https://github.com/Gamesight/secret-sync-operator

GitHub - Gamesight/secret-sync-operator: A Kubernetes operator enabling cross-cluster secret syncing

 

참고: https://hyperconnect.github.io/2020/11/02/cluster-migration-in-5-days.html