Introduction
Rfswarm (Robot Framework Swarm) is a Robot testing tool – written completely in Python- that allows a Senior PHP Engineer to load already completed Robot Framework test cases and use them in Robot framework load testing or performance.
Rfswarm aims to solve the problem of duplicated development effort for Robot framework performance testing by allowing you to take an existing functional or regression test case written in Robot Framework and run the test case with as many virtual users as you choose in order to generate load for the test.
Rfswarm is made up of 2 components:
- GUI/Server – where you plan, load, and run your test scenario
- Agents – which run the Robot Framework tests.
Rfswarm works quite differently than other performance-testing framework tools. rfswarm starts actual browsers and opens the full pages for running the tests. It’s not just API calls or pure, strict, and limited HTML requests/responses as most of the tools. This makes all of the side requests to all of the other connected sites or vendors for opening the page and measures the overall performance of the site.
Usage
1. Installation
You’ll need to install the Robot Framework Python together, as described in one of our previous posts by our colleague and Robot specialist – Stefan Mandovski, here.
Just note that rfswarm requires Python 3.7 or a newer version in order to run.
If you have already installed everything, you can get rfswarm (which is created by Dave). You can either clone it using GIT, or download the ZIP archive and extract it anywhere you choose.
You’ll need three more Python modules in order to run rfswarm: configparser, requests, and psutil
Get them by typing:
pip install configparser requests psutil
Of course, as with any Robot test, you’ll need the web drivers of the browsers you’ll test on.
And finally, the hardware requirements are located here.
2. Running tests
2.1 Server/GUI
After everything is installed and ready, you can run rfswarm server by typing:
python rfswarm.py
You’ll get a nice GUI:
Here you can create new, open, and save the test, as well as start/run the test.
Here you load a completed Robot script (…) and select the test (or multiple tests) from it, or add a new test (+).
You can set the number of Users (Robots), ramp-up time, and the actual run time of the test. The graph above this section adjusts displaying the timeline as you change these values.
You can run it headless by adding a -n switch when you run it. By adding -r the test will start automatically. -h is for help.
python rfswarm.py -n -r
2.2 Agents
But before you start robot testing you’ll need agents, on which the actual tests would run.
You can start those with:
python rfswarm_agent.py
If you run the agent on the same machine as the Server, you don’t need to change anything, just start both and run the test. But if you need to test with more users – for a realistic test, you’ll need more agents, and that means more machines.
If you start an Agent on a separate machine in the same network, you’ll need to edit RFSwarmAgent.ini file, located in the rfswarm folder. You’ll need to edit the line:
swarmserver = http://localhost:8138/
changing the localhost with the IP address of the machine where the rfswarm server is started. For example, my server is 192.168.3.134, so we change localhost to:
swarmserver = http://192.168.3.134:8138/
As agents start, they will be displayed in the Server/GUI along with the number of robots that run at the moment and their load.
2.3 Reporting
At the end, when the test is done, it produces a nice summary and detailed CSV reports.
Conclusion
rfswarm is a different type of performance/swarm load testing tool that needs quite more resources compared to the other load testing tools, but at the same time, it’s more ‘realistic’ because it mimics actual users that open actual browsers.
We ran the tests on several different machines with different specs and rfsmart distributed the users (robots) equally among the machines at first, but later on, assigning and starting more robots on the high-end machines and less on the lower spec machines. Smart.
We could start about 15 robots on a machine with (6th generation) Intel i5 CPU with 16 GB of RAM before it started peaking at 95-99% CPU usage.
There were no problems running 30 robots on a (10th generation) Intel i7 CPU with 16 GB of RAM.
Running tests with 100 robots would require 8-10 machines or VM’s, but doing so with 500 robots might be a problem.
So, to conclude; If you want to test in a more realistic way, mimicking real users, incorporating ‘think time’ and opening full pages – you should definitely try this tool.
