3

I have a very large selenium test framework I use to test a web application. It it built around a page-object pattern.

This week I fixed a bug in a low level piece of the framework, but it broke many other pieces that got built on top of this bug. I didn't have a good way to find these spots without running the full test suite of several thousand UI tests.

Since this isn't ideal, what strategy can I use to unit test new functionality and page-object modifications/additions to the framework?

3
  • Can't think of anything but brute force. Do you have a build system that can distributed the work between several agents and run the tests in parallel? Commented Jan 16, 2016 at 0:30
  • @mpkorstanje The test infrastructure is designed so that all tests are run as parallel as possible across as many browsers as we support. However, I had to run these test suites multiple times as I fixed each issue. Commented Jan 16, 2016 at 0:33
  • It should be noted that I have the concepts of Mock objects, serving my own contrived pages on localhost when running the unit tests, and a couple more basic unit test strategies running around in my head. I am having a hard time connecting them to a coherent test plan. Commented Jan 16, 2016 at 0:35

3 Answers 3

4

When you change the behaviour of a component in your framework, then just unit-testing that component is not sufficient. You must also re-run the integration- and higher-level tests that involve that component to verify what effect your changes have on the rest of the system.

If you find that the effect of your change is very large, the either you should reconsider the need for the change or bite the bullet and iteratively address the adverse effects.

2

Quis custodiet ipsos custodes?

Your test framework had bugs in it, which means that your tests were buggy. This is the problem with complex tests and complex testing systems. One way to deal with this is to create unit tests for the complex parts, and then confirm that they behave correctly.

In other words, it's just software, and if it isn't dead simple, it probably needs testing.

1

You will need to run all tests eventually as you're making a fundamental change. However you don't have to run all tests at once. Rather then fixing the bug, you could deprecate the bug causing functionality, create a new and correct bit of functionality which over time replaces the old functionality.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.