Whenever we received NFR (Non Functional Requirements) of Performance Testing in SRS, generally we fumble around it. . We don’t know where to start and how to end. I have listed down below the most common queries (concerns) that we face while doing Performance Testing and obviously its answers too.
By reading this blog you will learn on
- How to start with the requirement gathering of performance testing?
- How to execute the scripts?
- What are the best practices to use in scripts?
- What are the most common errors and how to resolve them?
- How can we put all these information in final Report which should be easily understood by everyone?
Before starting Performance Testing using JMeter, below are the common questions that one should clarify with the customer.
- How many concurrent users do we expect the application to support on an average at any given point in time?
- How many concurrent users do we expect the application to support on an average during peak duration?
- List of pages/activities that would be used more often to prioritize for Performance testing?
- What is the acceptable response time for the pages for performance testing?
- What will be the duration? How long will the load remain on the system?
- What will be the Infrastructure set up and application architecture?
- How will Server Monitoring be handled?
- Is there any scalability requirement over the period of time?
It would be really easy to prepare a performance Test Plan once we get all answers to above questions. Once Performance Test Plan is prepared, next step is to develop scripts.
Best Practices for JMeter Script Preparation
There are number of tutorials available over the internet which gives us step by step guide for developing scripts in JMeter. Develop the jmx files according to the project requirement. Below are some standards that we should follow while developing scripts:
- Remove unnecessary requests from the plan.
- Organize the samplers and other elements in most easy way.
- Follow Naming convention for all elements.
- Always keep “Retrieve all embedded resources” checked to make it behave more like a browser.
- Add listeners appropriately. Keep Result Tree disable when run for large number of users.
- Graphs use more memory. Keep it disabled, if not required.
- Try to run the script in non-GUI mode for simulating large concurrency.
- Only save the data that you need.
- Make sure how to use cookie manager and cache manager along with your requirement.
- Do not forget to add timers as per your requirement.
- Always use relative path for csv file if used in the script.
- Use multiple instances of JMeter in case the large number of threads/users is required to be simulated.
How to analyze the errors received
Below are the most common errors and their resolutions that we face through executing the scripts:
- Error: jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Java heap space
Solution: Need to increase heap memory. There is a line in jmeter.bat or jmeter.sh script wJVM_ARGS="-Xms512m -Xmx512m". Increase heap memory from here.
- Error: java.net.SocketException: Connection reset
Solution: Change "Implementation" of all your HTTP Request samplers to HttpClient4. The easiest way of doing it is using HTTP Request Defaults configuration element.
Add the following properties in user.properties file which located under /bin folder of your JMeter
httpclient4.retrycount=1
hc.parameters.file=hc.parameters
Add the next line to hc.parameters file (same location, JMeter's /bin folder)
http.connection.stalecheck$Boolean=true
Restart JMeter.
- Error : Connection refused to host : : java.net.ConnectionException : Connection Refused
Solution: Need to check the connection to host. Need to check the port number. Need to check the proxy settings
- Error: Non HTTP response message: The target server failed to respond
Solution: Try Disable KeepAlive.
Check Network Bandwidth (network cards, interfaces and switches), Monitor your environment and Timeout or retries
Also try edit the jmeter.properties file for retries = 3 and timeout = 120.
- Error : java.lang.String">java.net.BindException: Address already in use: connect in jmeter
Solution: Try to change the port. Run your JMeter client on a separate machine than the tomcat.
Also try to increase the ramp up time, if possible.
If you follow the best practices and methods to analyze the errors mentioned above you will breathe easily while handling NFR requests in your testing.
-Mrugali Parikh [CD QA Team]
