I am not a fan of comparison since I believe we cannot just put two things in both pans of a scale and see which one is heavier. Especially when these things got many aspects and features. It almost doesn’t matter we are trying to compare two persons, two technologies, two products or services or even two events. But when it comes to studying things in more details, it would be a good idea to know strength and weakness point of each object from various aspects.
Nowadays in our competitive and rapidly growing IT world, you can find many products and solutions in every small niche market. This diversity got advantages and disadvantages: It’s good because we have many choices and it’s bad exactly for the same reason! Despite shared features and capability each application has its own usage and advantage those make that product unique to users. End-users then can choose among them for the best option based on their needs and their project requirements.
On the other hand, this variety could make users confused and especially when it comes to job market it’s going to be apprehensive. Since you cannot take time and learn all of these tools so you need to choose between these competitors.
In this article, I try to bring two famous and powerful actors in load testing market in a recent decade into boxing Ring: Apache JMeter and HP (Micro Focus) Loadrunner. I will mention their share and different features based on various criteria. At the end, we will have a look at each one’s job market demand based on popular job searching websites in North America: Monster, Indeed, and Workopolis.
JMeter:
JMeter, as I know, is a tool which is used to test and analyze the load on client and server applications, created and developed by Apache Software Foundation. It is an open-source and free software to measure the performance and test the functional behavior. JMeter can run its tests on various platforms both static and dynamic as Java objects, FTP servers, files, servlets, SOAP, databases and queries, Pearl scripts, HTTP, POP3, and much more.
You can use it with many native and 3rd party plugins or even your own developed plugin to gather information and achieve your required functionality. You can integrate it with other tools such as Jenkins, eclipse, and Jira, which allows you to automate the whole of your performance testing.
LoadRunner:
LoadRunner is an automated interactive tool which is used to test the performance of an application. This testing tool is developed by Mercury Interactive to aid in determining the behavior of server and network applications under load normal, stress, and prolonged testing. The LoadRunner performance testing tool was later on taken over by Hewlett-Packard in November 2006. On 7 September 2016, Micro Focus announced its intent to merge with Hewlett Packard Enterprise’s software business segment.
LoadRunner consists of various tools, such as Virtual User Generator or VuGen, Controller, Analysis
Shared points:
Both JMeter and Loadrunner support various protocols and do sampling such as HTTP, FTP, SMTP, JDBC, etc.
Both parties support application environments, databases, and platforms as Web Service, J2EE, .net, ERP/CRM applications from Oracle, SAP, PeopleSoft, and Siebel, streaming and wireless media.
LoadRunner will do most of that and also allow you to GUI test. There are plenty of plugins like SAPGUI, for example, which JMeter does not support. LR is designed to work with/as part of HP ALM/QC, again something JMeter does not offer but many clients use. You can program HPALM to work with JMeter, but it is more work than it is really worth. LR is very expensive and aimed at projects with a larger budget, a complex infrastructure and a longer lifetime.
Their differences:
| JMeter | Loadrunner |
| It is developed in 100% Java | Loadrunner is developed in C and C++ |
| It is open source tool | It is a licensed tool and costly. |
| Cross platform solution | Just works on Microsoft Windows |
| Can generate unlimited virtual users | 50 virtual users in free version |
| It has huge online community support | The online community support is less when compared to JMeter |
| It does not have JMeter team support if you face any issues. | Loadrunner has HP team support when you face issues. |
| The license of virtual users is unlimited | Loadrunner is providing free license, but it is only for 50 users. |
| JMeter is a single component | Loadrunner is a set of components such as Vugen, Controller, and Analysis. |
| JMeter is very good for web applications. | Loadrunner is very good for most of the applications such as the web, java,.net, sap, Siebel etc. |
| Application business transactions are recorded through JMeter Proxy | Application business transactions are recorded through a browser or Loadrunner proxy. |
| It has less number of protocol support | Loadrunner support around 100 protocols including SAP, Siebel etc. |
| As It is purely developed in Java, it only supports to do customization of the script in Java. | The script can be customized in different languages for different protocols such as C, Java etc. |
| Text validation is supported. | In Loadrunner, also text validation supported. |
| Image validation is supported | In Loadrunner, also image validation supported. |
| Think time feature is available between two transactions | Here also think time feature is available. |
| Pacing feature is available between two iterations. | Pacing feature is available between two iterations. |
| Regenerating of script is not possible | It has a feature to regenerate the script by doing changes to the application. |
| It is supported cloud based testing with Amazon Cloud Services. | It is supported cloud based testing by installing the load generators on the cloud. |
| Monitoring of production servers during the performance testing is not available. | You can monitor most of the operating systems, application servers and database servers using Controller |
| JMeter Reports does not look good when compared with the Loadrunner and the features are also very less. | Loadrunner reporting results are very good. |
| JMeter does not have a feature to integrate with the profiler tools. | Loadrunner has the feature to integrate with HP Diagnostics for profiling Java and Siebel applications. |
| Recording of application is complex. | Recording of application is easy when compared with the JMeter. |
| We get a lot of heap memory issues with the JMeter if we run with a single machine and need to use distributed testing. | Loadrunner is stable when compared with the JMeter in terms of running the virtual users in a single machine. |
| JMeter has compatibility to develop the third party tools as JMeter plugins | Loadrunner does not have this feature of SDK. |
| JMeter does not have network virtualization feature. | Loadrunner has the network virtualization feature with the help of third party tool called Shunra network virtualization. |
| JMeter does not give the breakdown of response time. | Loadrunner will give the breakdown of response time. |
| JMeter supports to test the android applications | Loadrunner also supports to test the android application. |
| It supports to run the load generators in Linux machines | Loadrunner also supports to run the load generators in Linux machine. |
| Parallel execution of users on multiple devices is supported | Here also, parallel execution of users is supported. |
| You will find the Apache documentation here http://jmeter.apache.org/usermanual/ | You will find the Loadrunner documentation here http://www.softwarehour.com/loadrunner/ |
Job Market:
To have an overall estimation on job positions on software performance testing, I have searched keywords “JMeter” and “Loadrunner” in 5 famous job search websites in Canada. Please note:
- Some of the results may have indicated both keywords
- There could be some replications between these websites (one job position posted in more than one websites)
- This job search was Canada wide at September 2017
Conclusion:
Performance testing is a process of determining speed or effectiveness of a computer, network, software program or device. Both LoadRunner and JMeter can be used for load and performance testing. As mentioned Load Runner is highly developed which has strong support and many features but it has limited ability to monitor server statistics outside Microsoft windows.
JMeter as a suitable tool for web application Performance testing is a very light-weight tool and can be easily installed. Of course, it is free, so no license costs. This tool mainly used for performance testing (load, stress), User can apply automation frame work (data driven, parameter).
Also, job market stats show that JMeter got a higher popularity among users and companies those searching for a load and performance testing. On the other hand, LoadRunner got many clients in banking and financing industry where customer support plays an important role in selecting a software system solution.
I, as a QA engineer, used to work with both tool but I cannot decline that I have a huge crush! on Apache JMeter but still, try to use LoadRunner as a successful and user-friendly software environment.
Other load testing tools:
There are also other tools available to take into consideration which I just name them here:
IBM Rational Performance Tester
Microfocus Silk Performer
SmartBear LoadUI pro
Radview WebLOAD
Sources:
https://www.trustradius.com/compare-products/hp-loadrunner-vs-jmeter
http://www.jmeter.info/2016/01/jmeter-vs-loadrunner-who-is-the-best-in-the-market/
Great post! Thanks.
I agree with you. I prefer open source software because of community support.