A10 Ingress Controller

A10 Kubernetes Ingress Controller 2.0 is an application that looks for Kubernetes Ingress resources (CRUD) and provides automation to translate them to A10 Harmony Controller Lightning ADC configuration.

It also looks for scale up or down or healing of Kubernetes service endpoints and communicates the changes to A10 Harmony Controller to keep the Lightning ADC configuration in sync.

For providing access through a Load Balancer for applications deployed in Kubernetes cluster, Ingress resource is required. There may be multiple Ingress resources based on the requirements of the application deployed in the cluster. These Ingress resources mainly provide the configuration for load balancing and content-based traffic switching functionality. One Ingress Controller is needed in a cluster for Ingress resources to work. Ingress controller executes rules provided through Ingress resources.

Deployment Architecture

For using A10 Lightning ADC with Kubernetes Ingress Controller, it is recommended to install Lightning ADC as Kubernetes daemon-set. Running ADC as demon-set ensures that every node of the Kubernetes cluster automatically runs an instance of Lightning ADC. A10 provides the docker image of Lightning ADC as well as its configuration template YAML file that is required to connect Lightning ADC to the Harmony Controller.

_images/ingress_architecture.png

Ingress resources from any namespace can work with the Ingress Controller and configure Lightning ADC accordingly. Additionally, Kubernetes Master service restarts the Ingress Controller if for any reason the Ingress Controller goes down.

In addition to the basic configuration, A10 supports configuration of various policies directly from the Ingress resource. This is done using the annotations in the Ingress resource.

Deploying the Kubernetes Headless Service

When a Kubernetes service is created, by default, Kube-proxy plays the role of load balancer. When Lightning ADC is added in the path, Kube-proxy becomes redundant. Deploying the application service as headless service eliminates Kube-proxy from the path and traffic will be routed to Lightning ADC.

Refer to Kubernetes documentation Kubernetes documentation for additional information.

Configuring Virtual IP Address

Virtual IP (VIP) address can be configured in three ways:

  1. By passing a configuration parameter - It can be a IPv4 or IPv6 address and you can specify one or more VIPs for each of the ports. In case more than one IP address is provided it should be comma separated. Lightning ADC configures these VIPs on local interface.
  2. At start-up - You can specify VIPs to Lightning ADC as an environment variable at start-up time.
  3. By passing an interface name to Lightning ADC - A10 Lightning ADC reads the IP address from the interface specified and uses them as VIPs.

Handling Scale with Ingress Controller

Scaling of Nodes

Lightning ADC is deployed as demon-set, an instance of Lightning ADC is automatically created on the newly created node in case of a scale-up event. The new Lightning ADC instance comes up with the information to connect Harmony Controller and get the latest ADC configuration.

In case of scale down event, the Lightning ADC is removed from the cluster and traffic is handled by other members of the cluster.

Scaling of Application Services

As a Kubernetes service configued with Ingress resource scales up or down, a trigger is received by the Ingress Controller and ADC configuration is updated using the Harmony APIs.

Deploy the Lightning ADC as a Daemon-set

For using A10 Lightning ADC with A10 Kubernetes Ingress Controller, it is recommended to install Lightning ADC as Kubernetes daemon-set.

  1. Deploy the Lightning ADC Daemon-set by downloading the below sample file.

    Note 1: If the hostNetwork is set to true and this is needed if Lightning ADC shares the network with the host rather than work over an overlay network. This is a recommended approach but, not mandatory.

    Note 2: If the privileged is set to true and it is not mandatory for Lightning ADC to be started in the privileged mode. If the administrator has set-up the Kubernetes cluster to allow ports that Lightning ADC listens to, privileged mode is not needed.

  2. Edit the following fields if required and fill in the appropriate values from your environment.

    • app Label - Name of the daemon-set.
    • Name - Name of the daemon-set.
    • Image - Lightning ADC docker image and this can be downloaded from docker hub repository.
    • Environment Name - The environment name can be fetched from Harmony Controller and refer to Deploying in Docker Environment section.
    • Environment Value - The environment value can be fetched from Harmony Controller and refer to Deploying in Docker Environment section.
    • Container Port -
    • Host Port -
  3. Run the following command to deploy the Lightning ADC Daemon-set:

    kubectl create -f hc-ladc-daemonset.yaml
    
  4. To update the daemon-set in default namespace, use the following command:

    kubectl edit daemonset hc-ladc-daemonset
    
  5. To delete the daemon-set in default namespace, use the following command:

    kubectl delete daemonset hc-ladc-daemonset
    

Place Harmony Controller access Credentials in Kubernetes Secret

For configuring certificate and private key can be added as Kubernetes secret and can be used in Ingress resource.

  1. Download the below sample file to place Harmony Controller access credentials in Kubernetes secret.

  2. Create the tenant credential as a Kubernetes secret using the command:

    kubectl create -f hc-creds-secret.yaml

  3. Edit the following fields if required and fill in the appropriate values from your environment.

    • Metadata Name - Name of the Kubernetes secret
    • Username - Harmony Controller tenant User Name (base 64 encoded)
    • Password - Harmony Controller tenant password (base 64 encoded)

Place Certificate and Private Key Credentials in Kubernetes Secret for SSL Applications

For configuring SSL termination at Lightning ADC, certificate can be added as Kubernetes secret and can be used in Ingress resource.

  1. Download the below sample file to place certificate and private key credentials in Kubernetes secret.

  2. Create TLS (ECC or RSA) certificate and key as Kubernetes secret for SSL applications using the command:

    kubectl create -f hc-rsaserver-secret.yaml

  3. Edit the following fields if required and fill in the appropriate values from your environment.

    • Metadata Name - Name of the Kubernetes secret
    • Certificate - Base 64 encoded SSL certificate
    • Key - Base 64 encoded SSL key

Deploy the A10 Ingress Controller

A10 provides a configuration template YAML file for creating the A10 Ingress Controller. Only single instance of the Ingress Controller is required to run in the entire cluster.

  1. Download the below sample file to deploy the A10 Ingress Controller.

    Note: Lightning ADC cluster and TLS secret is optional and if this option is available in Ingress resource, then Ingress resource takes precedence.

  2. Deploy the Ingress controller using the command:

    kubectl create -f hc-ingress-controller.yaml
    
  3. Edit the following fields if required and fill in the appropriate values from your environment.

    • app Label - Name of the Ingress Controller.
    • Name - Name of the Ingress Controller.
    • Image - Ingress Controller image and this can be downloaded from docker hub repository.
    • Environment Values - The environment values are Harmony Controller URL, Harmony Controller credentials, provider and tenant values.
  4. Set-up Role-based Access Control (RBAC) to allow API access for Ingress Controller and refer to the documentation for additional information.

  5. To update the Ingress controller in default namespace, use the following command:

    kubectl edit deployment hc-ingress-controller
    
  6. To delete the deployment in default namespace, use the following command:

    kubectl delete deployment hc-ingress-controller
    

Create an Ingress Resource

Ingress resource is the object that allows users to define load balancing and content switching rules. A10 provides a configuration template YAML file for creating the Ingress resources in the respective namespace.

  1. Download the below sample file to create an Ingress resource.

  2. Create an Ingress resource using the command:

    kubectl create -f hc-ingress-resource.yaml
    
  3. Edit the following fields if required and fill in the appropriate values from your environment.

    • Name - Name of the Ingress Resource.
    • Host - Front-end domain name
    • Secret Name - ECC or RSA Kubernetes secret file name.
    • Path - service path
    • Service Name - Kubernetes service name
    • Service Port - Kubernetes service port
  4. To update host, TLS secret (for SSL application), path, back-end service information (service name, service port) in Ingress resource, use the following command:

    kubectl edit ingress hc-ingress-resource
    
  5. To delete the Ingress resource in default namespace, use the following command:

    kubectl delete ingress hc-ingress-resource
    

Setting-up Custom Policies with Ingress Controller

You can add A10 Kubernetes annotations to specific Ingress objects to customize their behavior. To define an annotation, you need to define a domain, a unique namespace and specify the domain or annotation name.

Annotations are defined as key-value pair separated by colon in the annotations section of metadata in Ingress resource YAML file.

In the first annotation which is Kubernetes specific annotation as specified in the above sample kubernetes.io is the Fully Qualified Domain Name (FQDN) and ingress-class is name of the parameter. Here we have defined ‘ladc’ as a class.

kubernetes.io/ingress.class: "ladc"

The following table describes the sample application at domain end-point level for which the following policies can be configured.

Policies
Description
* hc-ladc-cluster
* listen-ports
* tcp-application
* udp-application
* tls-protocols
* tls-ciphers
* vip
* Lightning ADC cluster for which the application is associated.
* Harmony Controller listens for application traffic on the listening ports. You can even add multiple listening ports if required.
* To create TCP application.
* To create TDP application.
* Transport Layer Security (TLS) is a protocol that provides data encryption and authentication between applications and servers in scenarios where the data is sent across an insecure network and to enable TLS protocols TLS1, TLS 1.1, TLS 1.2, TLS 1.3 in front-end application.
* A cipher is an algorithm used to encrypt and decrypt data. When a client initiates an SSL connection with a server, the client and server must agree on a cipher to use to encrypt the information and to enable TLS ciphers in front-end application.

The following table describes the sample service at service end-point level for which the following policies can be configured.

Policies
Description
* service-monitor
* service-ssl-enabled
* body-rewrites
* When adding a new service in Harmony Controller, you can configure out-of-band monitoring of application servers where Harmony Controller probes actively whether the application servers are active or not. You need to specify the monitoring protocol (TCP/HTTP, or secure TCP/HTTP connections), monitoring interval, and timeout. Refer to Server Monitoring section for additional information.
* Each Application Domain with Fully Qualified Domain Names (FQDNs) requires its SSL settings if SSL is enabled on Harmony Controller. To enable SSL, you need to have a valid SSL Certificate that identifies you and install it on the application server. Refer to Configuring SSL for a Domain section for additional information.
* You can control the display of text, headers and error code to web page visitors by using Body Rewrites function. Refer to Response Body Rewrite section for additional information.

The following table describes the sample smart flow for which the following policies can be configured.

Policies
Description
* add-default-header-rewrites
* header-rewrites
* url-rewrites
* body-rewrites
* access
* compression
* websocket
* HTTP rewriting is the technique which allows the proxy to change content on the fly while, we can Add/Delete/rewrite request and response headers. Refer to Header Rewrite section for additional information.
* The URL Rewrite policy helps you to rewrite complex URLs into user-friendly and search-friendly URLs without changing the page structure. Refer to URL Rewrite section for additional information.
* You can control the display of text, headers and error code to web page visitors by using Body Rewrites function. Refer to Response Body Rewrite section for additional information.
* Access Policies allows to define policies by specifying allow or deny rules for traffic from IP addresses. Refer to Access Control section for additional information.
* The compression policy is used to deliver content or data faster by reducing the amount of data that is transferred. The speed of data transfer increases with data compression. While defining the compression policy, you just need to provide the minimum size you want to compress and the type of content to be compressed. The minimum compression size is an integer value measured in bytes, and the type of content that to be compressed can be plain text/HTML or just plain text. Refer to Compression section for additional information.

Policy Configuration Used in the Sample Application Annotation

  • hc-ladc-cluster

    Syntax - “hc-ingress-demo-cluster”

  • Listen-ports

    Syntax - “443|true|true|false”

    It is a comma separated string, each portion refers to one port. It is further separated by pipes. Port number|If true, SSL is enabled|If true, HTTP2 is enabled|If true, proxy protocol is enabled.

  • Compression

    Syntax - “1024|text/html,text/plain,text/css,application/json,application/xml,application/javascript”

    It is a comma separated string and further separated by pipes. Minimum byte size at which compression begins.

  • Add-default-header-rewrites

    Syntax - “true”

    Add default header re-write rules will be added.

  • Header-rewrites

    Syntax - “true|reqHdrAdd|X-Orig-Host|$http_host,true|reqHdrAdd|Host|54.67.36.118,true|resHdrAdd|Strict-Transport-Security|max-age=31536000;includeSubDomains,true|reqHdrDel|Accept-Encoding|”

    It is a comma separated string and further separated by pipes.

  • Service-secure-redirect

    Syntax - “true|https://hc-ingress-demo.dev.a10networks.com$request_uri”

Policy Configuration Used in the Sample Service Annotation

  • Service-monitor

    Syntax - “tcp|/|10|5”

    It is a pipe separated string. You can set the protocol over which Harmony Controller provides application server monitoring. The protocol can be TCP or Http. For secure monitoring, use SSL over TCP or HTTP options. When you select the monitor protocol such as Http or Https, you must also specify the Monitor URL. In this case, Harmony Controller probes the monitor URL specified. Note that this field is visible only when you select HTTP or HTTPs. The time (in seconds) for which the application server is probed and monitored. The time (in seconds) after which the monitoring probe should timeout, within the monitoring interval.

  • Service-ssl-enabled

    Syntax - “true”

    Service SSL is enabled.

  • Body-rewrites

    Syntax - “true|54.67.36.118:8443|$http_x_orig_host|true”

    Enter the Regex or String value in the field named Match. Enter a new string value or Regex in the Replace With field, and click Enable button. Enable Case Insensitive button, this is optional.