Nancy and NCrunch – Playing nicely together…at last

If you haven’t tried NCrunch before then stop reading this and go download the trial and try it out, it will forever change your testing habits* – for the better.

NCrunch is the breakfast of champions

-KenR

At DrDoctor we use Nancy (rather than the typically used ASP.Net MVC Framework) the main reason is summed up by the overall goal of the framework:

The goal of the framework is to stay out of the way as much as possible and provide a super-duper-happy-path to all interactions.

One of the greatest things about Nancy vs ASP.Net MVC is that everything can be tested, and we have tests for pretty much all of our Modules (read Controllers) as well as our Views. NCrunch does some serious magic when it comes to running your test intelligently in parallel. Up until a few days ago I couldn’t figure out how to get my tests to work.

Testing Nancy Views

Testing Nancy Views is very easy, for demonstration purposes I created a new Nancy project from the standard ASP.Net and Razor Nancy template.

6-sample-solution

The template creates a module called IndexModule which returns a view called Index.

namespace TestingDemo
{
  using Nancy;

  public class IndexModule : NancyModule
  {
    public IndexModule()
    {
      Get["/"] = parameters =>
      {
        return View["index"];
      };
    }
  }
}

I then created an MS Test project called specs, and added the following test to make sure that when someone browses to “/” they would be served up the Index view

public void Should_Return_IndexView()
{
 //arrange
  var bootstrapper = new ConfigurableBootstrapper(with =>
  {
    with.ViewFactory<TestingViewFactory>();
    with.RootPathProvider<TestRootPathProvider>();
    with.Module<IndexModule>();
  });

  var browser = new Browser(bootstrapper);

  //act
  var response = browser.Get("/");

  //assert
  Assert.AreEqual("index", response.GetViewName());
}

To make this work I needed to implement my own custom IRootPathProvider,

public class TestRootPathProvider : IRootPathProvider
{
  public string GetRootPath()
  {
    string assemblyFilePath = new Uri(typeof(TestingDemo.IndexModule).Assembly.CodeBase).LocalPath;
    var assemblyPath = System.IO.Path.GetDirectoryName(assemblyFilePath);
    var path = PathHelper.GetParent(assemblyPath, 3);
    path = System.IO.Path.Combine(path, "TestingDemo");

    return path;
  }
}

In the Specs project I reference the TestingDemo project and then added the Nancy.Testing and Nancy Razor View Engine (since I’m testing Razor views) NuGet packages.

At this point the test passes in the visual studio test runner, but fails in NCrunch.

Eating Nancy tests with NCrunch

To get the tests to pass in NCrunch you will need to make a few changes to the NCrunch configuration for the test project.

  1. Open the NCrunch configuration window from the NCrunch menu and select the test project (below mine is called Specs)
  2. Under General select “Additional files to include”
    1-ncrunch-config
  3. Click the button with three dots (see highlighted in red above), on the Additional Files to Include dialog click “Add Directory”
  4. Browse to and select the root folder of your Nancy project, then click OK
  5. You will see the folder appear in the Additional Files To Include window, click OK2-additional-files-to-include
  6. Back in the NCrunch configuration window, under Build Settings set “Copy referenced assemblies to workspace” to True
    4-copy-references-assemblies
  7. You can now close the NCrunch configuration window

You should see that the test has now passed in the NCrunch Tests window

5-tests-have-passed

* May not actually change your testing habits.

Advertisements

Building an SSIS Testing Framework

If you compared the Google results for “testing c#” and “testing SSIS” you would quickly realize that testability isn’t one of the strengths of SSIS. I’ve seen a few different frameworks which were either too complicated or didn’t really work, in this post I’m going to give you my take on building an SSIS Testing Framework.

The Goal

The overall goal of what I wanted to achieve from my test framework:

  • Simplicity – writing tests needs to be as simple and easy as possible
  • The ability to load sample data into database tables
  • The ability to run a “setup” sql script
  • The ability to run an assert script to determine if the test has passed/failed
  • Auditing/history of previous test runs

My Solution

I didn’t initially set out to build a testing framework, it just evolved into one over the course of the couple of days I was working on it. I started off by manually testing my SSIS packages and realized I was following the same steps over and over again:

  1. Load sample data
  2. Run SSIS Package
  3. Run some scripts to check if it worked

I could see that this was going to get repetitive so started to think about how to automate it. After a few iterations this is what I’ve ended up with (at time of writing)

1-TestController-package

The TestController package follows the well trodden AAA (assemble, act, assert) approach that is found among the TDD (test driven development) practitioners in the software development industry.

This package does the following:

  1. Gets a list of tests to execute
  2. Then in the foreach loop does the following for each of the test cases:

Assemble:

    1. A script task is used to get a bunch of test parameters from a SQL table, then updates the associated variables in the TestController package, at the moment these are only used to control which tables have data loaded
    2. A pre-data load setup SQL script is then executed
    3. Load Sample Data will then load any sample data for this test run
    4. A post data load setup SQL script is then executed

Act:

  1. Execute the package under test (PUT)

Assert:

  1. Run an assert SQL script

 The Downsides

The solution isn’t perfect, here are a few of the problems that it has at the moment:

  • Unable to pass package parameters to the package under test in the Act phase
  • Makes assumptions about how you want to do your setup routine
  • Is purely focused on packages that touch the database
  • Writing assertions isn’t as easy as I would like, it involves writing lots of T-SQL and copy/pasting code throughout the different assert scripts
  • All your databases need to be on the same server
  • Maintaining the variables and and connection managers is painful

Some of these I’m not terribly bothered about, for example having to have all of your databases on a single server I think is acceptable as this is for testing purposes. Whereas the duplication of assertion code could be easily fixed by creating some reusable stored procedures (e.g. Assert.TableEqual, Assert.TableEmpty etc.) similar to those offered by tSQLt.

One of the more painful areas of the framework is setting up tables to have sample data loaded, the process involves adding variables, setting up connection managers adding components to the data flow task and updating the script component. One way of fixing this issue that I thought of was generating the TestController package with BIML and have all of this automatically generated.

Wrap Up

As a starting point I’m really happy with how the TestController is working at the moment. During the development of it and the use afterwards I picked up a few bugs with the packages that I wanted to test, so it is paying for itself already 🙂

If you’re interested in trying it out for yourself then leave a comment and I’ll put it up on GitHub.