I'm looking for something more flexible than apache ab, but not as heavy weight as JMeter. Ideally I'd like to be able to specify some random values in urls or post requests. My environment is linux/ubuntu. What are you using these days?
We use wrk since we are often load testing fairly simple services, occasionally with some Lua for cache busting. It's a great tool. Would love if it supported http2 since I work on CDN type stuff where that matters.
I love your 2nd approach (advertising). Any pointers to how to get started?
If you want to purchase advertising for this purpose, I'm happy to help with this. My contact details are linked through my profile.
- Your HTTP endpoints will need to be whitelisted with traffic quality groups and creative scanners (This can take a week or so). I recommend using a separate dns domain name for this since adblockers tend to subscribe to these lists.
- Your JavaScript should not be abusive. There are subjective tests for this. Generally doing an image request or some POST data of a few kb isn't going to be an issue. Some ad servers (like sizmek) download 2mb "test images" to exercise their bandwidth from some locations (like the US).
- Costs vary based on device, region, the web page the ads are on, and silly things like demographic. Some of these factors are interesting to benchmarkers but others are not. Thinking about what you actually need to test can help keep your costs efficient.
JMeter is indeed a tad heavier, but I find it's much more user friendly as in, you spend a couple of hours to build basic tests, or afternoon and have learned most of its functionality. It also has a variety of ways implementing this randomized user behavior [1].
You can definitely do all those things with JMeter, although the user experience is pretty naf, Gatling[3] is definitely much better at that front if you're comfortable writing Scala.
When it comes to Protocol Level Testing, those two tools are still pretty much the best bang for buck, and by far the most popular open source load generators.
If the target you're hitting is a web app and you want to create a more realistic load scenario without crafting individual requests, check out a tool called Flood Chrome, which uses thousands of instances of Google Chrome[1] to generate load. It uses the same scripting language as SeleniumJS, so it's quite easy to learn.
Also on the topic of Flood [2], they're a very good platform for abstracting away the server orchestration and results collection you'll face when you try to scale up from 1 to N servers running either Gatling or JMeter.
Load testing can be hard. Wrk works for me for the most part but it's not ideal for complex flows (say e-commerce trying to simulate a real browse and checkout complete with typeahead search).
However the smarter apps can have problems too. I've pinned all available cores on fairly beefy machines because they are gunzipping bodies (behaving more like a real browser).
If you are at a scale where you need really good testing I recommend that you think of wrk results as comparable to a unit test and something like jmeter as comparable to an integration test. It's not worth the effort for most companies but where I work we serve a decent amount of traffic and downtime costs 6 figures per minute so good performance engineering and good business overlap.
Jmeter is de facto in telecom , it's far from perfect but it let you define complex flows quick , the pain is that you need to ménage the test plans on the server to run all the jmx files
I wish there was full web interface on the server already
* It allows me to focus on latency versus throughput independently
* It has tools for testing performance on pipelined requests (very important in my space)
* It supports scripting complex authentication processes (like magic headers and oauth2)
* It supports scripting fuzzing (random values) for URLs and POST requests
[1]: https://github.com/wg/wrk
For wide-area load testing, I simply buy some advertising since that gives me cheap access to millions of simultaneous users.