One compelling reason to shop online is there are no lines. Another reason to shop online is the ability to search virtual aisles – however one wants and whenever one wants. When you combine those two shopping benefits, be it B2C or B2B, and then force multiply by anticipated sales transactions, merchants can become consumed with thoughts how best to ready themselves.
Magento Load Testing – what does it mean for an eCommerce merchant
An online shopping experience should meet these two minimum standards – a website should be both personalized and fast to provide back results. As a result, a high performing eCommerce website is expected. For merchants the world over, their website’s response to each user and the ability to process tailored experiences translates to purchase (or not). Anything short of those standards should be tested, studied and remedied.
Tools for testing
For eCommerce administrators and developers, there are many ways to test and measure website performance. There is a number of tools to check out and use in the general marketplace. However, a warning to merchants – proceed with caution when selecting tools – know there are many ways to measure performance and as importantly there are many factors any tool must consider to provide back intelligent reporting. At Atwix, we trust and use only a few performance tools. We also built and heavily rely on our own tool that zeroes in on Magento website testing.
The most common performance tool and arguably the most generic is Google Page Speed Insights. For Magento websites, however, this tool has limited value. Google Insights was built for universal platform testing rather than platform-specific testing. It must be understood that any test results using Google’s tool must be evaluated with an understanding that the underlying measurements are platform agnostic. If you are looking for results precalibrated for a Magento store, you can use the Atwix Performance check tool, specifically for Magento be it Open Source, On-Premise, or Commerce Cloud. The Atwix tool takes into account Magento’s various configurations.
Regardless of what tool you select and use, and you should use a tool regularly, you must continually benchmark results taking into consideration snapshot in time conditions. Frequently, these momentary tests do not weigh extreme conditions like a Black Friday sale. To emulate these conditions, we encourage that you run Load Tests in advance of testing to more accurately prepare you for the heavy site traffic.
What is Load Testing
Load testing is, in essence, putting demand on a system and measuring its response. It helps to determine a website’s sustainability, simulating real-life conditions. Load testing factors in not just how many users shop at once but what the various users might request (intentional or not) all at once during API calls, Magento 2 search page, billing lookups, etc.
For example, load tests check how many users can do the following actions at the same time:
- Browse the website
- Add products to cart
- Go through checkout and place order
- Use website search
- Manage account
Load tests are performed to recognize upper limits a website can be expected to support heavy activity at the same time delivering optimal user performance expectations.
As a merchant, you may need to perform load testing if you’re:
- going to release a new website
- moving to a new server
- planning a special sale
- releasing a new product
How to get started?
There are several stages of load testing: preparation, data generation, data gathering, and running the tests (often in several iterations).
Preparation
As a first step, you should prepare a test environment to check the behavior of the system. We suggest you prepare an environment for extreme load testing. You need to make a copy of the website. Make sure to use an exact copy of the production server, both reflecting software and hardware environments. Important note: do not run any performance load tests on the same servers where you host your production website.
Test Scenario Adjustments
For merchants using Magento 2, it has a built-in Performance toolkit. This native functionality helps to create a needed amount of customers, products, etc. For technical details or to just learn more, check out our article on Magento 2 performance toolkit.
While practical for those using Magento out-of-the-box, for those requiring customization, we encourage working with your solution partner or website development agency. (If you require guidance on what tools you need to consider, feel free to contact us.) Effectively, not all tools will work for all testing environments. A customized customer registration flow or other implemented customizations on the store might be required to optimize Magento performance.
The customer registration, for example, will look like this:
During the test scenario adjustments and information about existing data in the store (about products, categories, customers, URLs) are collected. If the store is new or fresh, containing no data, then it should be created and gathered into one array for the main front-end pool test release. This is the adjustment of the load testing scenarios for using the data.
Running scenario iterations
The predefined scenarios are being launched during a few iterations many times. During this period, various metrics should be collected. Now, two possible approaches are available for the run JMeter:
- JMeter GUI (Graphical User Interface)
You can launch the load tests using GUI. This approach allows you to debug and adjust scenarios on the go. The drawback of this approach is it is slow and very resource-consuming.
- JMeter Console Interface
It’s a much more efficient way of running load tests. It’s a recommended way of running tests when all the debugging and adjustments are done.
In our scenario of the new product release, we expected to have the major visitors spike during the first 15-30 minutes. In such cases, the goal should be set per minute. In our latest load testing, it was 50k users for 30 minutes, so we did 850 orders in one minute. We have run 1 scenario on 4 servers and 1 laptop.
Results
In the end, you will receive the following report:
It shows which actions are more server-consuming and which are more lightweight. Also, the highest peaks of the response time can be analyzed. This exact report is performed for the Magento Cloud instance, which showed perfect sustainability after proper settings were applied.
Conclusions
As users go online to shop, merchants ready their eCommerce websites to serve as a complement to retail brick-and-mortar stores or just to become the primary storefront. Ultimately, the website’s performance becomes much more critical. In the end, no matter the look-and-feel, the depth and breadth of products offered, the variation of payment options, etc. — user abandonment will occur if the site does not quickly process and respond.
Merchants must anticipate the same performance their customers expect. Managing backwards from those various user experiences, all at once, takes time and thought centered on load testing. Namely, load testing helps identify performance bottlenecks and it prevents risks related to system downtime. Moreover, the information received back from testing can be used for strategic planning to demonstrate the maximum website operating capacity needed for operational expenditure planning. To perform adequate levels of load testing to best fit your eCommerce platform, you as a merchant are best to do their homework to fully represent their configurations. From there, check out the best tips, tools and solutions available to you by exploring peer vetted resources. Namely, be on the hunt for specialists with deep technical knowledge, separate servers that mirror production environments, and find tools and reports that meet standards that best prepare you for the most intense user experiences and live moments.
Have you ever performed Load Tests on your website? Please, share your experience in the comments and we will be more than happy to discuss this together.