In a traditional ADC environment, there is little to no visibility into the flowing traffic especially at application layer. In that case, troubleshooting application layer issues like slowness of application becomes next to impossible as it is about coming up with various hypothesis and collecting data to validate them. In many cases, this data collection is either not possible or impacts the performance significantly. Even after collecting data, good amount of correlation and analysis effort is needed to make out actionable information.
In the A10’s ADC architecture, ADCs keep collecting metrics and push it to A10 HarmonyTM Controller in real-time without impacting the ADC performance. A10 HarmonyTM Controller has a big-data style analytics engine that collects all the metrics, correlate them and come up with meaningful and actionable insights. These insights are displayed on the A10 Harmony Portal for users to consume and fine tune their application and infrastructure.
Additionally, A10 Harmony Portal allows to filter, drill-down, slice and dice the information as per the requirements. The troubleshooting can be started from the high-level indices drilling down into the problem area. Filtered logs and details of individual transactions allows users to double-confirm the issue. All this not only significantly cuts the troubleshooting effort (~80%) but also raises the confidence in the troubleshooting results.
A10 Lightning ADC Analytics¶
Various metrics about the traffic, clients, ADCs, servers, application and the infrastructure are collected at A10 Lightning ADC and pushed to A10 HarmonyTM Controller for analysis. The analytics engine at A10 HarmonyTM Controller enriches the data, correlates various metrics, finds trends and anomalies, builds actionable insights and presents it via A10 Harmony Portal and A10 Harmony API.
Application Traffic Analytics and Insights¶
A10 HarmonyTM Controller provides detailed insights into the application traffic. Since the HarmonyTM Controller is front-ending the application servers, insights are available even for traffic that does not reach the servers due to any security or other traffic management policies applied by HarmonyTM Controller. Additionally, if all the micro-services of the application are modelled properly, these insights will provide application performance management (APM) capabilities as well. Analytics and Insights are summarized based on a time interval selection. Time Selection is available in the user interface (UI) at the top-right corner. A few predefined intervals are provided for convenience, but the custom selection is always available where the time range may be chosen at the granularity of a minute.
The main body of the dashboard has a series of charts that give a quick statistical overview of the behaviour of the chosen application traffic. The screen is broadly divided into three sections to provide this overview at a glance. Abnormal behaviors can be quickly picked up visually:
- The best and the worst view (or Top-Ten) in term of traffic performances
- The App server and service facing statistical indicators
- The Client facing statistical indicators
The Client tab displays the following charts:
- End-to-end latency
Displays the end-to-end response time at client for a http transaction and the time taken in different portions of transaction.
- Total Requests
Displays the number of http requests in different time intervals or time series distribution of requests.
Displays the number of requests from different countries.
Displays the types of operating system used by the client.
Displays the types of device used by the client.
Displays the browser types used by clients to access the applications.
- Most active clients
Displays the IP address of client sending maximum number of requests. Table representation is click-able and automatically filters are applied and logs are visible for the particular filter.
- Response Codes
Displays the http response code. If a user sees more errors with error code (4xx and 5xx), then a user can debug these errors using per request analysis and fix them. s - Displays the http response code. If a user sees more errors with error code (4xx and 5xx), then a user can debug these errors using per request analysis and fix them.
Displays the incoming and outgoing bytes in different time intervals.
The Lightning ADC tab displays the following charts:
Details of threats are available on the security dashboard that can be reached by clicking on the Blocked Requests.
- DDOS Attack Mitigation
If the user notices any deviation in performance, say if the SSL parameter crosses beyond the expected level; then the user needs to check for SSL protocol, ciphers, and SSL certificates. It may also be an indicator of a potential DoS attack.
The application tab displays the following charts:
- Response time
Displays the application response time. Clicking on Total Response Time at top-left of the dashboard takes you to Response-time Drill-down. Data shown is in the context of a Service of the Application that can be selected from a drop-down at the top-left of this page.
Charts on this page help the user visualize the breakup of all the clients response time providing a quick summary to quickly figure out if there are any issues one needs to investigate. The diagram on the top right on this page shows the various portions of an HTTP request-response process and provides the average time taken in each of those portions.
The chart below displays the various time duration of how much time is consumed on the average in various portions of the request to response period.
Identifies areas of issues in the request-response path. Optimize the portion of a period where on the average maximum time is being spent.
The other chart below the diagram on the top right focuses on the server response time for each of the server in the pool which is under the control of the user and this information can be used to improve performance, optimize resources, or troubleshoot servers by taking subjective measures.
Next, there are a few tables showing segmentation of the traffic response time based on various parameters: by type of response, by the client, by URL, by the server, by client geo-locations. Rows of these tables are sorted such that the ones having highest response time show up on the top row. In case if the list is long, then only top few entries are shown.
Clicking on any row takes the user to the next level of drill-down where individual logs are shown filtered by criterion selected by the particular row clicked.
This filtering and ability to look into individual transaction details help you figure out if there is a general problem with the system and you need to debug it further, or it was an anomaly that you can ignore.
- Top URLs
Displays the traffic distribution on different URL types.
- Top Domains
Displays the traffic distribution based on different domains.
Top Domain End Points
- Top Services
Provides information about the areas of the application that receive the maximum amount of traffic.
- Worst Transactions
Provides information about areas of application for which the average response time is maximum. Indicates potential sources of application latency seen by clients.
The Internet tab displays the following charts:
WAN Latency, requests - Displays the client latency based on country, number of http/https requests based on the selected region, incoming and outgoing bytes in different time intervals and total http/https requests included.
The Lightning ADC clusters tab displays the following charts:
Displays the overall CPU usage of the cluster.
Displays the memory usage of cluster.
Displays the incoming and outgoing bytes in different time intervals.
The AppServer tab displays the following charts:
- Server response
Displays the server latency time for the transactions.
Displays the CPU usage of the app server.
Displays the incoming and outgoing bytes in different time intervals on the server side.
Shows the TCP connection to a particular server.
- App Server health index
Displays the server health for the app server.
The A10 Analytics Dashboard for Lightning ADC is segmented into three parts: Traffic, Security, and Health dashboard, each catering analytics aggregated at an application level as shown. It also provides analytics for Metrics, Blue/Green, Alerts, Events, and Logs. The application selection can be made from the left panel. However, the A10 Analytics Dashboard for Thunder ADC has Traffic Dashboard along with analytics for Metrics, Blue/Green, Alerts, Events, and Logs.
Allows the user to access logs for each request along with details of request size, response size, source and referrer information, information of the server served the request along with the time taken in each portion of the transaction. Also, the user can access the WAF logs and WAF acknowledged logs. Helps user to get to the root of the problem and points the user to the problem area.
Analytics > Logs displays the Per Request logs, the WAF logs, and the WAF acknowledged logs. You can use the filters at the top to narrow down the scope of your analysis.
To access WAF logs from the dropdown select Only WAF Logs as shown.
To filter a specific WAF log user can create an exception by opting the check box under Create exception and save it as shown.
To access the WAF logs with an exception, the user can use the Acknowledged WAF Logs option from the filter as shown.
Logs can be filtered on any field by entering the specific values to be filtered as shown below.
Clicking on individual log row opens a pop-up providing detailed information about the request and response.
Per Request Analysis¶
The Analytics Dashboard is mechanized to provide Per Request Analysis logs, the user can now customize the per request log collection policies according to their needs. The per request log collection policies offer different options for the user to choose from such as OnErrors Logs, OnEvents Logs, and Manual Control Logs. OnError logs are collected in the event when traffic hits the server and the server throws an error response due to various reasons. OnEvent logs are collected after initial application onboarding for the time in minutes, or after every application change for the time in minutes. Manual control logs are collected between the defined time frame by the user.
The figure below is Per Request Analysis Log chart, in this, the Absent logs are the logs which are not collected.
You can create the Per Request Log policies from this screen. To reach here click the Per Request Log policies from the above screen.