Accurate consistent testing is the key to smooth implementations

Two of the hardest questions that we sometimes face are: ‘How much testing have you done?’ or the worst: ‘How did this get through testing?’

 

After all, comprehensive testing is immensely time-consuming and laborious even if everything goes as you hope it should. And as It’s the last stage before deployment, testing often gets squeezed when the pressures of a successful go live are very high.

In 2014, Microsoft introduced a set of tools that enabled us to create automated tests, although the first version honestly wasn’t that easy to use. Four releases later, they have now reached a point where they work so much better.

 

This means we can now write a scripted test which we can run as often as we like. Not only does this give us a pass or fail for that test in milliseconds, but we can then combine it with the approximately 18,500 tests for standard Dynamics NAV provided by Microsoft. This gives us a massively more comprehensive picture than can be achieved manually, even with our combined efforts.

Here at Technology Management, we are so convinced that automated testing is the future that last month we made a significant investment in getting our team trained. Not only have our developers learnt to write the testing code, our consultants have been coached on how to define and then run the tests. In addition, our customer service and project management teams have also been taught the relevant processes.

There is some up-front time required to initially define and write the tests. To do this, we create a comprehensive test plan and then script that test plan. The aforementioned Microsoft tools support us from here with just a single line creating a customer record, another creating an item and another creating a sales order.

 

The good thing about the scripting is that once we’ve written the first test for an area, executing it again with a single change and then again with an additional change will take mere seconds.

Microsoft’s recommended approach is to write tests so that they can be run against any set of data in any company or database. This is called called unit testing, where the test creates all the data which it is dependent on and then executes the test.

 

We have single line commands that can create G/L accounts, customers, items, orders, journals and pretty much any other type of data you can think of. You can then run the test and discover if it’s done what you defined it should or shouldn’t do.

 

Alternatively, scenario tests are where you test a process end-to-end and can be completed in either a data agnostic (where every piece of data required is created) or data dependent way. Data plays such a critical part of having a smooth Dynamics NAV system that having both types makes sense. For instance, we have a process that can loop or randomise the script so that we can create, ship and invoice a sales order to every single customer for every product within minutes.

The massive advantage with automated tests is that once they've been written they can be used again and again. Every time a new release of customisations or an upgrade goes through, regardless of how small it is, it will only take a few minutes to run the tests, as opposed to hours or sometimes days of manual testing. That’s just the best situation to be in and certainly makes everyone sleep easier the night before a go live.

We are now taking the extra step of adding a new section to our change request and technical design documents that will define what the testing process is going to be (manual, automated, mixed) as well as the test plan.

We know that clients still on Dynamics NAV 2016 or before unfortunately cannot use automated testing right now, although building it into the upgrade process means it will only replace the testing that you’d have had to do as part of the upgrade anyway.

The dream that we’ve all had of a brilliant capable system, that we can get to do what we want with minimum effort, is getting there and automated testing seems to be one of the final pieces.

 

I know from the instances where I still get to write code, that I spend more time testing than I do writing. Getting the data in the situation that I need for the process, running it through, accessing the results, changing or adding more code and then starting all over again. Usually, when I’ve finished that I then have to start end-to-end tests and see what has unexpectedly broken which can prove inevitably frustrating.

 

Often, between everyone involved, testing takes more time than any other single activity. While automated testing should completely remove manual testing, it should also mean that problems identified once an automated test has been written never reoccur. That will be real progress against a real frustration and be great for us both. 

 

If you have any further queries, please do not hesitate to call me, or your account manager today to learn more about how automated testing will support your business during a new implementation.