Home » Archive by category "Continuous Integration"

Selenium Performance Improvements


As many of you know, I’ve been using Selenium to do my website testing.  And, if you’ve done any testing with Selenium yourself, you know that Selenium can be even slower if you are using Selenium Grid.

There are several things you might do today to achieve Selenium Performance improvements in order to increase the speed that your of your test run.

CacheLookup attribute (or Annotation if you are using Java)

This is probably the most obvious place for improvement.  Assuming you are using the Page Model pattern in your testing, your page model should have properties that already look something like:

[FindsBy(How = How.Id, Using = "radioButton")]
private IWebElement RadioButton{ get; set;};

What you may not know is that the way this works is that EVERY time you access the property named RadioButton it will lookup the location of the radio button.  The RadioButton property is actually a proxy for the real IWebElement that it will lookup on the fly.

So, if you are using the element more than once in your test, it will perform the lookup each time you access the RadioButton property.

By adding the CacheLookup attribute to the RadioButton property,

[FindsBy(How = How.Id, Using = "radioButton"), CacheLookup]
private IWebElement RadioButton{ get; set;};

you can force the RadioButton to be resolved once and only once.

Most of my test ask for elements once per test, so I didn’t see any major performance gains by adding this.  Your mileage may vary.

CacheLookup Warning

Be careful when adding CacheLookup to your properties because any time the HTML is recreated, the element will become “stale” and you’ll need to look it up again.

For example, in one of the pages I’m testing, I have two modal windows that get created as part of the test.  Each time they are created, they have to be resolved again because the old ones no longer exist.

It would be like trying to reference a pointer to an object that no longer exist because something else deleted it.  Actually, that’s exactly what is happening.

CacheLookup Only Solves One of Your Problems

But if all you do to your site is add CacheLookup, you are wasting your time.  You have to start thinking like your code.

Did you know that every time you access a property on an IWebElement, Selenium has to make a call to the browser to access the current value?


Actually, if you think about it, this is what you hope it does.  It is what makes code like this work:

Wait.Until(x => Page.RadioButton.GetAttrinbute("disabled"));

But how many times to we write code that goes after a property multiple times when we know the value, or at least we assume the value, hasn’t changed?

Or how many of you are doing the same thing multiple times in the same test?

For example, in the code I’m testing currently, I have a header and a footer that I need to hide so that when an element is scrolled into view by Selenium, it doesn’t scroll under the header or the footer. I’m currently hiding and showing every time I perform some action on an element.  But now that I’m starting to think about performance, I’m asking myself, “What was I thinking?!”

In this case, I would save a lot of time by turning the header off once a the beginning of my test.  I bet my test end up running twice as fast simply by doing that.

So, keep in mind that every Selenium call you make is probably going to have to access the server in order to perform the action or retrieve the value you are looking for and try to write your code so that you are only making that call to the server one time per test.

I bet you see some performance improvements.

Running Selenium In Parallel With Any .NET Unit Testing Tool

Running Selenium in parallel from .NET seems to be a problem because, as of the time of this writing, I’ve yet to find a viable way of running selenium test on multiple browsers using Selenium Grid.  This doesn’t mean that there aren’t a few articles out there that have some kind of solution.  But they’ve never satisfied me as something that I could easily plug into my already created test.

While my preferred testing tools are NUnit and SpecFlow, the method I am about to propose should work with any existing test harness you might want to use.  The only prerequisite is that you are using Page Models to wrap your access to any particular web page.

This article assumes that you already:

  • know how to write Selenium tests
  • know how to use Selenium Grid
  • know how to use the Page Model pattern
  • know how to use your chosen test harness.

OK.  On to the main event.

Here Is The Problem

In order to run multiple browsers at the same time, the easiest way is to provide a wrapper Page Model that calls multiple instances of the page model at the same time.

The hard way of doing this would be to create an Interface that represented the real page model and then create a proxy class that would hold a list of all of the real page model objects we needed to call.  Each time a property or method on the proxy gets called, all it would do would be to pass the call down into the real objects in parallel.

This would work, but the main draw back is that I really don’t want to have to write a method for each method in my real page model.  So the question is, how can we get around this?

DynamicObject To The Rescue

Enter the little known class, DynamicObject.   In .NET 4, Microsoft introduced the dynamic keyword.  One of the main uses is for places where  you need to be able to declare a variable in your code that the compiler won’t know how to resolve the type of until runtime.  I could have used this several years ago when I had two assemblies that needed to reference each other.  In that case, I used reflection.  But dynamic would have worked with a lot less work.

DynamicObject is a specific class that allows us to resolve property and method calls at runtime using our own logic.

We will also use the Task Parallel library to implement our parallel calls.

For completeness, and so that no one is confused when they try to implement this code, you’ll need the following using statements at the top of the CS file.

Using Statements And Constructor

using System;
using System.Collections.Concurrent;
using System.Dynamic;
using System.Reflection;
using System.Threading.Tasks;

So, let’s get started.  The first thing we will need is a class declaration:

class ParallelPageModel<TPage>:  DynamicObject

TPage allows us to specify the Interface the real Page Model implements.  Yes, we still need the interface, but we won’t need to create a new wrapper class for every page model we want to wrap.  The class  inherits from DynamicObject so that all of our on the fly goodness will work.

Next, we’ll need some place to store an array of PageObjects we want to proxy.  So we add a private variable _page for that purpose.

private readonly TPage[] _pages;

By using TPage[], we create a variable that is the same type array as the Page Models we are proxying.

Next we need a constructor.

ParallelPageModel(params TPage[] pages)
    _pages = pages;

By using the params keyword, we can either  pass in page objects as an array or as individual parameters.

The magic happens in three overridden methods that are in DynamicObject:

  • TryInvokeMember – resolves any method calls.
  • TrySetMember – resolves any property setters
  • TryGetMember – resolves any property getters

So let’s add those methods next:

public override bool TryInvokeMember
    (InvokeMemberBinder binder, 
      object[] args, 
      out object result)

public override bool TrySetMember
    (SetMemberBinder binder, object value)

public override bool TryGetMember
    (GetMemberBinder binder, out object result)


Inside of the TryInvokeMember method, the first thing we will want to do is to use reflection to call into the real methods.  Since we could have multiple instances of the same method we need to call we will want to do this in a loop.

When I first worked this out, I started by just implementing a foreach loop but we are going to jump right to using Parallel.ForEach()

Parallel.ForEach() will let us pass in an array and run a lambda expression on each element in the array.  So, our foreach loop will look like this:

var results = new ConcurrentBag<object>();
Parallel.ForEach(_pages, page =>
    var thisResult = typeof (TPage)
        BindingFlags.InvokeMethod | 
        BindingFlags.Public | 
        null, page, args);

Note that our lambda expression is not doing anything more than a simple reflection call.

The result that is returned is added to our ConcurrentBag collection.  ConcurrentBag is a collection that is specifically made for parallel calls.  We could get into trouble if we added something to a List<> collection unless we added some parallelization gatekeeping around it.  I’m for doing as little work as possible.

The second thing we want to do is to process the return results.

For this we need to setup a basic foreach loop.

foreach (var thisResult in results)

Inside the foreach loop we will process the results collection.

If the type that got  returned is the same type as the type that the page is proxying for, we just make our result value, the return value the TryInvokeMember is going to return for us to the code that called the proxy, equal to the proxy object.

if (thisResult is TPage)
    result = this;

If the result is not null, meaning either that a previous result was null or we haven’t processed the loop yet, we want to check to see if the value of the current loop result is the same as the loop results we’ve already processed.  If it isn’t, we throw an exception.

else if (result != null)
    if (!result.Equals(thisResult)) // not the same value
        throw new Exception
           ("Call to method returns different values.");

Finally, we just set the result to whatever we have at this point.

    result = thisResult;

And then the last thing we want to do is to return true to tell the system we were able to process the method.


Since the implementation for TryGetMember looks very similar to TryInvokeMethod we’ll tackle that next.

In fact, the only difference between the two methods is the code inside of the Parallel.ForEach parameter block.

So, here it is:

Parallel.ForEach(_pages, page => 
    var thisResult = typeof(TPage)


TrySetMember is the easiest implementation of all since there are no results to worry about.

     page => typeof (TPage)
        .GetProperty(binder.Name).SetValue(page, value));
return true;


So the code above will work, but you won’t get any intellisense help from Visual Studio if you use this code without tweaking it.

What we need is some way of casting the ParallelPageModel object to the TPage type that we pass in.

For that we are going to use a cool library I found called ImpromptuInterface.

You’ll need to add a using statement.

using ImpromptuInterface;

And then you’ll need to add this method to the ParallelPageModel class.

public TPage Cast()
    return this.ActLike();

You would use this like this:

IMyPageModel p = pageModelProxy.Cast();

Where IMyPageModel is the interface that specifies what your real PageModel class looks like.

Just in case someone is tempted to mention this in the comments, you can’t us operator overloading to achieve the cast because we need it to return TPage, which could be anything and the compiler can’t deal with that.  If you really want to use operator overloading you’ll need to provide your own specific implementation that ends up calling the code above.

Calling The ParallelPageModel

To setup the ParallelPageModel, your code would look something like this, assuming that you have a page model class called MyPageModel with an interface of IMyPageModel.

var pages = new ConcurrentStack<IMyPageModel>();
    () => 
    () => 
var pagesArray = pages.ToArray();
MyTypedPage = 
    new ParallelPageModel<IMyPageModel>(pagesArray).Cast();


I only just started using this.  It works for my current implementation.  But you may need to tweak it so that it works for you.

For example, my assumption here is that you are only dealing with simple types or the page model type you are a proxy for.  There is no code here that will handle a situation where the call to a method would return a entirely new page model. Since the code I am testing is a collection of Single Page Applications and I am not testing navigation at this point, this is not a consideration for me.  But it would be relatively easy code to implement.  If I did that, I would probably handle it but subclassing this main class that does the bulk of the work and override the Try*Member method that needed to deal with that situation.  The other possible way of dealing with the situation is to pass in a list of types that need to be wrapped in their own parallelization object as parameters in the constructor and add some generic code in the ParallelPageModel class.

Finally, I am well aware that this code may have bugs.  If you find one, go ahead and fix it. You can leave a comment so that others will benefit.

Here is the entire class all in one chunk for those of you who just want to copy and paste the solution.


Other Places Talking about Parallel Selenium

And of course a ton of links to people asking how this can be achieved.


Make Your Test Work For You


So far we’ve been talking about creating test as part of the development process.  If all you ever used those test for was to make the design of your systems better, you would already be far ahead of most of your peers.

But now that we have test, we might as well get as much mileage out of them as possible.  To do that, we are going to run our test every time we make a change.

Continue reading “Make Your Test Work For You”