Load testing on single server

Application: w3wp.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: System.AccessViolationException

   at System.Threading.Thread.InternalCrossContextCallback(System.Runtime.Remoting.Contexts.Context, IntPtr, Int32, System.Threading.InternalCrossContextDelegate, System.Object[])

   at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)

   at System.Runtime.Remoting.Proxies.RemotingProxy.CallProcessMessage(System.Runtime.Remoting.Messaging.IMessageSink, System.Runtime.Remoting.Messaging.IMessage, System.Runtime.Remoting.Contexts.ArrayWithSize, System.Threading.Thread, System.Runtime.Remoting.Contexts.Context, Boolean)

   at System.Runtime.Remoting.Proxies.RemotingProxy.InternalInvoke(System.Runtime.Remoting.Messaging.IMethodCallMessage, Boolean, Int32)

   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(System.Runtime.Remoting.Proxies.MessageData ByRef, Int32)

   at .....followed by application exception 

When I was running my load tests using JMeter against a single server (usually against a load balancer but for some reason wanted to know the)I happened to get this weird error repeatedly with 100% error rate in Jmeter. The application I was testing was a WCF service hosted on IIS.

My first instinct was to check if the error had occurred because of my recent changes, I checked the event viewer which told me that this error was occurring very rarely.

It took a lot of googling and ended up with nothing, so I decided to check on the IIS 7 configuration.

I found the root cause of the failure, it was due to few configurations that had to be changed in IIS.


Configuration:

IIS Queue Length: The default value is 1000, so after 1000 active connections any new connections will be served a 503 Error.

IIS rapid fail protection: By default the value is set as true. So after 5 application errors, the app pool goes down.

To sum it up, since I ran the load test for 'n' Users. The default IIS Queue size is overloaded with many requests crossing the default 1000 queue size which returns application errors our and the other config “Rapid Fail Protection” pulls down the app pool.

This error has occurred only because I load tested on a single server with a connection limit of 1000 in the IIS.

I increased the queue size to the max value of 9000 and disabled the Rapid Fail Protection and Ran the same LOAD TEST. It FIXED the problem and worked well as I saw a error rate of less than 0.3% for a very large sample and the above error never occurred again. Turns out it only occurred because I ran the load test against a single server. But it did point out to me that all the servers had a default value of 1000 Queue size and in case of an actual load all the servers would have gone crashing down. 

So to conclude the above error was a memory error I believe. If you faced the same error please post below on what you did to fix it.

Here are the configurations for IIS 7.0 as suggested by Microsoft. Be careful in whatever you need for your application, not all is good for you.

https://technet.microsoft.com/en-us/library/cc745955.aspx

SoapUI hosting on Tomcat error

06-Nov-2016 14:18:34.624 SEVERE [http-nio-9092-exec-2] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable
 java.lang.NoClassDefFoundError: Could not initialize class com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport
at com.eviware.soapui.DefaultSoapUICore.initSettings(DefaultSoapUICore.java:361)
at com.eviware.soapui.DefaultSoapUICore.init(DefaultSoapUICore.java:129)
at com.eviware.soapui.DefaultSoapUICore.<init>(DefaultSoapUICore.java:114)
at com.eviware.soapui.mockaswar.MockAsWarServlet$MockServletSoapUICore.<init>(MockAsWarServlet.java:317)
at com.eviware.soapui.mockaswar.MockAsWarServlet.init(MockAsWarServlet.java:71)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1183)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:779)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:133)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:784)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:802)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)

This is the error I happened to get on SOAPUI 5.2.1 on trying to deploy a war file containing mock service on Apache Tomcat 8.5.

This issue is caused because the hosting service is unable to find some jar files required to host the service. This can be FIXed by ensuring soapUI has internet access. Once I ran this on a system having internet access, it worked fine.

If you have a system without internet access, you can use the SOAPUI itself to host the mock service and disable logging, it is good enough to run a Load Test on it. Response time I saw was 0.1ms . 

Managing VirtualBox Better

Getting the better out of the painful VM – Virtual Box

After struggling a long time with VM with slow speed and frequent crashes. I found few tips from various websites which helps the VM run in a better way.

This post contains the following.

• ·         Tweaks for a better VM
• ·         Installing Ubuntu or any other Linux/OS on Oraclebox.
• ·         Misc

Below are a few tweaks that can be done to prevent VM from getting crashed and to make it perform better.

(It would be better if you have the fresh vmdk file before applying the following settings.)

Go to

machine-> settings-System.

Set Base Memory to the max green level.

Switch to Processor Tab

Set the CPU as 2 or the green level.

Set the execution Cap below it to 100%

Switch to Acceleration Tab

Set  both Option in Hardware Virtualization to checked.

Go to Display

set Video memory to 128 mb in the video tab

Other things to do:

• ·         Take a snapshot of the fresh install (initial install) – You can restore to any previous snapshot and prevent the loss of your entire vm
• ·         Run Disk Defragmenter every week on your VM.
• ·         Save the machine state, do not shutdown everytime.
• ·         Always choose fixed disk size to prevent space and performance issues.

In case of network issues:

Go to machine -> settings -> Network

Change Adapter 1 to Attached to : Bridged  (works sometimes- this is due to network issues.)

Installing a linux VM on your own.

Download Ubuntu or any linux distro iso file .

http://linuxlookup.com/linux_iso


· Start VMware.

· From the File menu select "Create a New Virtual Machine"

· Choose to install the operating system later. Click "Next".

· Select Linux as the "Guest Operating System" and then choose Ubuntu as the "Version". Click "next".

· Provide a "Virtual machine name" and "Location" where the machine will be stored on the Windows host. The defaults are fine here. Click "Next".

· For "Maximum disk size (GB)" it is good to start with 40G if possible. This means that it will take up 40G on the Windows host. Make sure that the Windows host has at least this much before proceeding. It is also a good practice to tell VMware to split the virtual disk into 2G files. This will makes the image easier to copy and transport if necessary. Click "Next".

· Click "Finish" to complete the creation of the virtual machine.

• settings -> Storage -> Select the disk Icon
• 
  
• On the right select Attributes browse the ISO you downloaded.Click OK.
• 
  
• Now boot up Ubuntu VM ware - install normally like you do any OS installation.
• 
  
• After installation, you will be prompted for reboot. 
• 
  
• Go to Machine ->Settings -> Storage -> Select the disk Icon
• 
  
• On the right select Attributes Disk image and select remove the iso.
• 
  
• Now boot up your VM again. You can use Ubuntu .

MISC:
Cloning:

When you want to clone the VMDK file. (Time consuming, do not do it unless extremely necessary)

Grouping Boxes:

Group -> group (for better classification)