Share on facebook
Share on twitter
Share on linkedin

Building a modern/serverless application is the most talked-about trend in the cloud engineering space as it enhances business agility and lowers the total cost of ownership. It frees your developer from the heavy lifting of server management with scalability & high availability so that the team can focus on application development.

However, debugging is the most significant problem of serverless applications and existing solutions.

This blog talks about a use case where MIND helped a client debug and troubleshoot their serverless application.

Use Case Overview

One of MIND’s clients built a serverless application on AWS using API Gateway, Lambda and DocumentDB as the backend for payment processing gateway (represented in the image below), which is critical and demands very low latency and offers high availability.

As a result, high availability was achieved using AWS Managed Services, including API Gateway, Lambda and DocumentDB; however, the latency and timeout errors were causing the client to have hard times.

These latency issues and timeouts were random and didn’t have any pattern. Therefore, the MIND team started to analyze each layer of the serverless application to find the actual component facing the issue.

The Approach

It started with the analysis of the API Gateway logs. We examined the IntegrationLatency metrics to measure the responsiveness of the backend and Latency metrics to calculate the overall responsiveness of the API calls.

Subsequently, we studied the Lambda logs and turned on the X-Ray, a great tool to visualize the application parts, troubleshoot performance and inspect the Lambda cold start issues.

To handle the Lambda cold start issue, which occurs if there are no reusable containers to serve the request, AWS needs to provision a new container, download the code and dependencies from S3, and initialize by executing the code outside the handler function.

This cold start introduces a latency of about 1-4 seconds, which is highly critical for financial applications.

The MIND team scheduled CloudWatch events to call each Lambda function every 5 minutes in order to keep the Lambda functions warm. In each Lambda function, a code existed that needed to be examined if it received a request from the CloudWatch event (passing a dummy parameter). After receiving the same, it returned the request without carrying out any processes.

Upon analyzing the Lambda X-Ray logs, we were able to identify that it was due to a cold start issue.

Findings

Based on the logs and error message research, the MIND team determined the main reason for the ongoing issue- the MongoDB Node.js Driver. Therefore, we advised the client to upgrade the driver, following which the timeout issue was resolved.

Moreover, about the latency issues, we found problems in the client approach of keeping the Lambda containers too warm to handle the Lambda cold start issues. Resultingly, we suggested a new method of using Provisioned Concurrency and helped the client implement the same, which solved the latency trouble.

To learn more about Serverless Web Applications, request a demo or contact us here.

About the Author:

Shailendra Singh Tomar – Principal Cloud Consultant

Shailendra is having 17+ years of experience in Java Technologies & AWS Cloud. He loves working with customers to build secure, high-performing, cost-optimal and resilient cloud solutions using the AWS Well-Architected Framework. He leads the AWS Immersion Days program at MIND to help customers learn how to best leverage the AWS platform to unlock business potential and meet key objectives.