Our Solutions Architect, Kate Orlova and Technical Director, Andrey Stepanov, share their recommendations on which tools to use for continuous integration and delivery for Sitecore projects.
How to Use JMeter for Performance Load Testing
Our Quality Assurance Specialist explains a little more about JMeter before detailing how you can use Apache JMeter for performance load testing and at the same time catch a tricky partial server downtime of a high traffic Sitecore website.
During my three years of work as QA specialist at Delete I have learned that JMeter isn’t just a universal testing tool, it’s much more than that.
So to show you what I mean, in this article I will explain a little more about JMeter before detailing how you can use Apache JMeter for performance load testing and at the same time catch a tricky partial server downtime bug.
What is JMeter?
According to the official JMeter definition ‘Apache JMeter application is open source software’ and is a ‘Java application designed to load test functional behavior and measure performance’. The official definition also explains how ‘it was originally designed for testing Web Applications but has since expanded to other test functions’.
How to Use JMeter
For testing, JMeter runs without a user interface. Test scripts can be created in a special panel where you can essentially do anything you want.
This image shows how to create test script using JMeter panel (you don’t even need to “write” a script).
You need to create a general Test Plan, and add a Thread Group with the necessary test elements into it.
There are additional elements which allow you to set parameters. For example, HTTP Request Defaults allows you to specify the primary server, headers and to enable/disable loading of additional assets (images, styles, fonts and others). You can even run a test from the interface and see the results straight away.
JMeter can record test scenarios. It will create a local proxy-server, if you set up your browser or any other application to use 127.0.0.1 as the proxy. JMeter then records all requests and responses.
You can create a test scenario by selecting from any of the records which will repeat user actions and run tests where and when necessary:
Be aware though that the memory of Java itself is a known issue. It reaches a limit and can crash with 50 threads of concurrent testing - even on machines with a high spec.
However, if your equipment isn’t powerful enough to run a full load test, you can use third-parties who can quickly create a suitable environment with servers running JMeter. Our team have used Octoperf and Blazemeter in the past successfully.
To explain how JMeter can help you find the elusive partial server downtime problem, I can give a real example of an experiment we ran.
Out team has been building web applications on enterprise level platforms for 18 years and we’ve recently developed a single page website (SPW) application for a leading UK football club on Sitecore.
An SPW is a website where all pages open without the reloading of the page itself by a browser. In the backend we built an admin interface for the management of ‘live match day’ functionality.
Live match day is the live text broadcasting of a football game, which sees the main events load automatically from a third-party system called Opta, including photographs, commentary and fan posts from Twitter and Instagram which are moderated and published by content managers at the football club.
The new website was an instant hit and following the launch many online publications picked it up and covered it as digital transformation in the Premier League.
We also ran a full load test prior to launch to make sure that everything worked as it should and could cope with high demand. The server infrastructure was as follows:
- A browser sends a request to a load balancer server.
- A load balancer server checks load on all eight content delivery (CD) servers.
- A load balancer server directs the request to the least loaded CD server.
- The CD server responds to the load balancer, and it responds to the browser.
Dealing with Demand
One year after launch a breaking news story about the club came out and triggered a huge wave of visits to the website, exceeding even peak times on match days. Our client then contacted us saying that some users have reported that the website was down.
We checked the website and it worked fine. The client also checked the website from their phones and desktops and it worked fine too.
Without missing a beat, our QA experts started testing, launched Server Density and saw that servers were fine - but the issue soon reoccurred.
The load across all CD servers, didn’t identify any issues.
A number of users continued reporting that the website was down, however we saw no errors, the servers just simply weren’t responding.
JMeter was the tool that then helped our QA experts to identify this bug.
We decided that we wanted to load all eight servers and closely monitor what happened. It was scheduled to run during the night to minimise the risk of any negative experience for users.
Several computers were used to run JMeter in an interface-less mode which helped running more concurrent threads. The script kept requesting the homepage again and again, and later we found out that this was the main problem.
All homepage assets were already cached and our requests didn’t generate significant load. After a short brainstorm a decision was made to load news pages instead. We randomised URL parameters such as year, month and pagination count and injected them into the URLs. This gave us an opportunity to load a random page every time.
The script started loading new images and other assets which weren’t cached, which generated the significant load that we needed for our experiment.
We discovered that ‘Server Density’ skewed the data for past periods. For example, if the server load reaches 100% for five minutes then drops to 0%, and in 20-30 seconds goes back to 90%, it shows an average value for that period of time - approximately 80-90%. This is why we never saw the true downtime of the servers.
A detailed review of the server logs during the stress tests showed that one of the eight CD servers rebooted every time it reached 100% and didn’t send out notifications.
When the load balancer server checked the CPU usage of all eight servers it saw that the first one was at 20% capacity and all others at 90%, and it directed traffic to the first server. But in reality the first server was rebooting, which is why users saw a white screen.
To make things worse, the load balacer server created a cookie on users’ machines which remembered which server they used, which meant that these users could not be served the website from another operating server and that even after a reload users did not see the website. We solved this problem by changing the logic of load balancer and developed status control procedures on every server.
Without JMeter we simply couldn’t have resolved this issue, which demonstrates why it is a fantastic tool - especially for stress testing and analysing server statuses during testing by third-party tools.
If you enjoyed this article, head back to our blog soon as I will be sharing a new JMeter post explaining how to configure caching for product pages, how to test a mobile application (without using the application itself) and how to create 2,000 users in a system without the access to the database.