Skip to content

Custom Domain in Kubernetes with AWS Route 53, Gardener, Istio & Let's encrypt

Overview

In this article we are going to setup custom domain for Dirigible application in Kubernes cluster with Gardener, AWS Route 53, Istio, Let's encrypt.

The target Kubernetes deployment is shown bellow: Gardener - AWS - Istio - Dirigible

Overview

Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications in a cluster environment. You can read more about Kubernetes here.

Overview

Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. here.

Overview

Deliver fully-managed clusters at scale everywhere with your own Kubernetes-as-a-Service. Kubernetes-native system managing the full lifecycle of conformant Kubernetes clusters as a service on Alicloud, AWS, Azure, GCP, OpenStack, EquinixMetal, vSphere, MetalStack, and Kubevirt with minimal TCO.here.

Overview

Istio is an open source service mesh that layers transparently onto existing distributed applications. Istio’s powerful features provide a uniform and more efficient way to secure, connect, and monitor services. here.

Prerequisites

In this article we assume that you have already running productive Kubernetes Cluster on Gardener and configured kubectl for it. If you don't have such, you can create one by using the the open-source Gardener project. Also you will need AWS account or AWS Free Tier, you need to install Istio and Dirigible

AWS Route 53 Configuration

  1. Create hosted zone - when you create hosted zone choose type Public hosted zone see the image below.

    AWS - Hosted Zone

After you create your hosted zone you can delegate your subdomain to AWS, if you don't host your parent domain in AWS. You can take the name servers which you can see in the image bellow and add ns records to your domain. But if you host your domain in AWS you don't need to delegate.
AWS - Nameservers

  1. Create new user - which will provide to Gardener dns provider.

    • When you create user select credential type to be Access key - Programmatic access ( you see the image below).

    AWS - Create User

  2. Create group - add user to group, but for this scenario we need to create new group which will be using only for this purpose. That's why, click on Create group and it will open new tab to create the group.

    AWS - Create Group

  3. Create Policy - before you create the new group click on Create policy it will open new tab to create the policy.

    AWS - Create Group Name

    • On the first step click on the JSON see the image below:

    AWS - Create Policy - Step 1

    • Leave this tab open and find your Hosted zone ID see the image below:

    AWS - Take Your Hosted Zone Id

    • Add your hosted zone id to the JSON you can use this example json
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": "route53:ListResourceRecordSets",
                "Resource": "arn:aws:route53:::hostedzone/*"
            },
            {
                "Sid": "VisualEditor1",
                "Effect": "Allow",
                "Action": "route53:GetHostedZone",
                "Resource": "arn:aws:route53:::hostedzone/Z2XXXXXXXXXXXX"
            },
            {
                "Sid": "VisualEditor2",
                "Effect": "Allow",
                "Action": "route53:ListHostedZones",
                "Resource": "*"
            },
            {
                "Sid": "VisualEditor3",
                "Effect": "Allow",
                "Action": "route53:ChangeResourceRecordSets",
                "Resource": "arn:aws:route53:::hostedzone/Z2XXXXXXXXXXXX"
            }
        ]
    }
    

    AWS - Add Your Hosted Zone Id to Policy

    Note

    It's very important to add your correct hosted zone id

    • We don't need tags, see image below:

    AWS - Create Policy - Step 2

    • Тype policy name, see image below:

    AWS - Create Policy - Step 3

    • Add new policy to the group, see image below:

    AWS - Add New Policy to the Group

  4. Add new group to the user.

    AWS - Assign New Group to the New User

    • We don't need tags:

    AWS - Add User Step - 3

    Note

    Before you click on create user check Permissions summary that consist your new group.

    AWS - Add User Step - 4

  5. Download your access key.

    AWS - Download New Access Key Id

Gardener Configuration

  1. Provide AWS Route 53 credentials - we need to provide our AWS Route 53 credentials from the previous step:

    Gardener - Add New Amazone Route53 Secret

  2. Add gardener extensions to the Shoot cluster - configure dns and Let's Encrypt certificate.

    • Open Gardener Shoot yaml file and add DNS providers - shoot-dns-service, shoot-cert-service:
    spec:  
      dns:
        providers:
          - secretName: my-aws-route53-secret
            type: aws-route53
      extensions:
        - type: shoot-dns-service
          providerConfig:
            apiVersion: service.dns.extensions.gardener.cloud/v1alpha1
            kind: DNSConfig
            dnsProviderReplication:
              enabled: true
        - type: shoot-cert-service
          providerConfig:
            apiVersion: service.cert.extensions.gardener.cloud/v1alpha1
            issuers:
              - email: <your-email-here>
                name: <type-name-for-the-issue>
                server: 'https://acme-v02.api.letsencrypt.org/directory'
    
  3. Create subdomain for your application - we can use DNSEntry for Gardener, because in Gardener Shoot yaml file we configured:

    dnsProviderReplication:
              enabled: true
    
    • Find Istio ingress gateway external ip:
    kubectl get services istio-ingressgateway -n istio-system \ 
    --output jsonpath='{.status.loadBalancer.ingress[0].hostname}'
    
    apiVersion: dns.gardener.cloud/v1alpha1
    kind: DNSEntry
    metadata:
      annotations:
        dns.gardener.cloud/class: garden
      name: dns-entry
      namespace: default
    spec:
      dnsName: "app.demo.dirigible.io"
      ttl: 600
      targets: 
      - <type-here-the-result-from-previous-command>
    

Istio Configuration

We need to configure our istio ingress gateway to accept our new sub domain app.demo.dirigible.io and the certificate.

  1. Apply the dns configuration and certificate - in this article we will configure istio ingress gateway to accept wildcard certificate.

    kubectl edit svc istio-ingressgateway -n istio-system
    
    # Please edit the object below. Lines beginning with a '#' will be ignored,
    # and an empty file will abort the edit. If an error occurs while saving this file will be
    # reopened with the relevant failures.
    #
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        cert.gardener.cloud/issuer: app.demo.dirigible.io
        cert.gardener.cloud/secretname: wildcard-tls
        dns.gardener.cloud/class: garden
        dns.gardener.cloud/dnsnames: '*.demo.dirigible.io'
        dns.gardener.cloud/ttl: "120"
    
  2. Gateway configuration - configure Gateway and VirtualService for Dirigible application or any.

    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: dirigible
    spec:
      selector:
        istio: ingressgateway 
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        tls:
          httpsRedirect: true      
        hosts:
        - "app.demo.dirigible.io"
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: wildcard-tls
        hosts:
        - "app.demo.dirigible.io"    
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: dirigible
    spec:
      hosts:
      - "app.demo.dirigible.io"
      gateways:
      - dirigible
      http:
      - match:
        - uri:
            regex: /.*
        route:
        - destination:
            host: dirigible
            port:
              number: 8080        
    
Back to top