AWS SQS: What is it, Industrial Usecase

Among the various cloud resources or services provided by Amazon, SQS is an extraordinary addition.

Let’s start by describing what SQS is and how it works, we will be covering the benefits of using SQS and then move to the case study, which discusses how companies are using Amazon’s SQS service to solve their issues. In this blog we have also discussed about how Amazon SQS solved a developer’s issue while creating an app on an individual basis.

How SQS Work ?

A Real World Example of SQS :

Ok, let’s imagine we’re running the latest and greatest video streaming service. Orange Tube.

Well Orange Tube has tons of users who are constantly uploading videos to it’s servers hosted on AWS.

Those videos are stored on S3 and a message is created each time that is added to the SQS queue. These messages are then picked up by worker nodes that are LONG polling the queue for jobs.

The messages have a 30 minute visibility timeout setting and this gives a worker node that amount of time to encode the uploaded video in the format the Orange Tube understands and then deletes the message from the queue.

To deal with sudden spikes in upload traffic, we can place the worker nodes behind a auto scaling group and then monitor the SQS queue for if the number of tasks grows beyond a certain value then we can simply increase (or decrease) the number of nodes available to process videos.

The messages that SQS handles can be unformatted strings, XML or JSON. Because SQS guarantees “exactly once” delivery, and because you can concurrently submit messages to and read messages from a given queue, SQS is a good option for integrating multiple independent systems.

You might well be asking: why use SQS if you can have an internal HTTP API for each service? While HTTP APIs are an accessible way to expose software systems to external users, it’s not the most efficient mechanism when it comes to integrating purely internal systems. A messaging queue is more lightweight. In particular, SQS also handles things like automated retries, preserving queue state across multiple availability zones in AWS, and keeping track of expiration timeouts on all messages.

BENEFITS OF USING SQS

For Serverless developers, using SQS generally provides a wealth of benefits, which you can read about below.

Scalability

Your SQS queues scale to the volume of messages you’re writing and reading. You don’t need to scale the queues; all the scaling and performance-at-scale aspects are taken care of by AWS.

Pay for what you use

When using SQS, you only get charged for the messages you read and write (see the details in the Pricing section). There aren’t any recurring or base fees.

Ease of setup

Since SQS is a managed service, so you don’t need to set up any infrastructure to start using SQS. You can simply use the API to read and write messages, or use the SQS <-> Lambda integration.

Options for Standard and FIFO queues

When creating an SQS queue, you can choose between a standard queue and a FIFO queue out of the box. Both of these queue types can be useful for different purposes.

Scheduling batch jobs.

SQS is a great option for scheduling batch jobs for two reasons. First, it maintains a durable queue of all the scheduled jobs, which means you don’t need to keep track of the job status — you can rely on SQS to pass the jobs through and to handle any retries, should an execution fail and your batch system returns the message to the queue. Second, it integrates with AWS Lambda; if you’re using AWS Lambda to process the batch jobs, SQS automatically launches your Lambda functions once the data is available for them to process.

UseCase:

RedBus is an Indian online bus ticketing platform providing ticket booking facility through its website, iOS and Android mobile apps. Headquartered in Bengaluru, India, it connects bus travellers with a network of over 2500 bus operators, across India, countries in South East Asia and Latin America. The company also offers software, on a Software as a Service (SaaS) basis, which gives bus operators the option of handling their own ticketing and managing their own inventories. To date, the company says they have sold over 30 million bus tickets and has more than 1750 bus operators using the software to manage their operations.

Few challenges faced by Redbus:

• The company previously ran its operations from a traditional data center by purchasing and renting its systems and infrastructure.
• In addition to the expense, several logistical problems evolved from this arrangement.
• The biggest problem was that the infrastructure could not effectively handle processing fluctuations, which had a negative impact on productivity.
• Additionally, the procurement of servers or upgrading the server configuration was an extremely time-consuming endeavor.
• RedBus receives complaints from customers mainly regarding Refunds, Cancellations, Operator queries etc… The process of classifying these emails and redirecting them to the respective customer service agents/queue becomes a humongous task. This also increases during peak days (especially holidays and weekends).

The company has incorporated many of the AWS products into its solution, including Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing, Amazon Relational Database Service (Amazon RDS), Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), and Amazon CloudWatch. Charan Padmaraju, Chief Technology Officer believes that “with features like Elastic Load Balancing and multiple availability zones, AWS provides the required infrastructure to build for redundancy and auto-failover. When you incorporate these in your system/application design, you can achieve high reliability and scale.”

Benefits:

Since migrating to AWS, redBus has seen measurable improvements in the bottom line. Padmaraju says, “By scaling up and down dynamically based on the load, we maintain performance as well as minimize cost. With the time savings that the IT and development staffs obtain from the AWS solution, AWS gives us an overall cost benefit of about 30–40%.” He adds, “By hosting at [the AWS Asia Pacific (Singapore) region], redBus.in gained significantly in terms of website performance by way of reduced latency (about 4x). This is a great advantage when the customers are from India.”

Of the many excellent characteristics of AWS, perhaps the most significant to redBus is the ability to “instantly replicate the whole setup on demand for testing by creating and destroying instances on demand for experimentation, thereby reducing the time to market.” Less time to market translates to increased profitability and success.

Thank You!!