Llewellyn Falco recently posted Using ApprovalTests in .Net 20 Asp.Mvc Views which has been one of his more popular videos in the Using ApprovalTests in .Net series. The level of interest in this video is justified. Unit testing MVC Views is not easy at first. The learning curve is steep, but not long. Once you get started and know the basics, its pretty easy to repeat the process. I was lucky enough to get some pointers from Llewellyn on testing a project of mine, and I’d like to share what I’ve learned about testing MVC. My plan is to take you from the default tests provided by File->New Project all the way up to a project that tests consistently in Visual Studio, under NCrunch and on a build server. In this process, I will try to make every mistake I can possibly think of and show the resolution to each problem. That will make this article significantly longer, but I hope it will also make reading this more useful.
Many bloggers, faced with a topic that can’t be summed up in a few hundred words, will start a blog series. More accurately, many bloggers will declare that they’re starting a blog series, write one post, and then never mention the subject again. Not me. I already wrote all the parts. But I am a merciful blogger, and interest of avoiding being slapped with a TL;DR, I will post each part separately over the next couple of days.
I’ve created an example project at which includes spoilers, but if you want to dive right in you can. By the time I post all three parts of this article, everything in there should be fully explained.
Getting Started
We’ll start with a virgin MVC 3 intranet site. The Razor ViewEngine is all the rage, and if this were anything other than demo code, I’d recommend choosing it. But just to prove that the view engine doesn’t matter to this testing process, I created the project using the old ASPX ViewEngine. Also, using this ViewEngine makes the demo code more closely resemble the project that I learned these techniques on, since that project is a site that’s been slowly evolving since MVC 1, back when a razor was something you shaved with.
I checked the check box requesting “HTML 5 Semantic Markup”. I don’t even know what that means, but clicking checkboxes is fun. Of course, I also requested a unit test project. I selected MSTest as my framework, because its easy and does what I need. After a few moments my new MVC site appeared in Visual Studio, which I have named “MVCTestSite” in a stroke of creative genius.
A note on web servers
I feel the need to digress from the strictly linear telling of the story of the MVCTestSite at this point. If all had gone according to plan then about now I would be explaining how I set up the project to use IIS Express. Indeed, I did set it up to use IIS Express and followed the instructions in the readme to configure my site to use Windows Authentication. Namely, I disabled Anonymous Authentication and enabled Windows Authentication. This worked fine for awhile, right up until I tried to use ApprovalTests with this configuration. With IIS Express configured in this manner, ApprovalTests (as of v1.9) wont be able to retrieve pages for verification, IIS Express will reject the requests with a 401.2 error.
I spent a long time trying to figure out what I had done wrong, since I had used IIS Express and approval tests in projects at work. But like I said, that project has been evolving since MVC 1, and no doubt its web.config is full of all kinds of crazy kruft. Its probably not a good base case for comparison. Anyway, the issue comes down to this, ApprovalTests uses WebClient to make its requests, and by default WebClient does not send the user name to IIS Express. No username means Anonymous Authentication, which we previously disabled when we followed the instructions in the readme file.
So if you want to use IIS Express, you can, but you have to relax the restriction on Anonymous Authentication. This is really just an IIS Express setting, changing those properties does nothing to your project, so if you need to deploy with Anonymous Authentication disabled, that’s separate from what you do to IIS Express. Your other option is to use the Visual Studio Development Server (aka Cassini).
For the rest of the article I will just assume Cassini.
The Default Tests
As we can see in the Solution Navigator, the MVC project template gives us two tests by default, both are controller tests, one for the Index method and one for the About method. Since I chose the Intranet template, MVC did not provide a default AccountController or any tests to go with it.
Lets take a closer look at these two tests:
[TestMethod] public void Index() { // Arrange HomeController controller = new HomeController(); // Act ViewResult result = controller.Index() as ViewResult; // Assert Assert.AreEqual("Welcome to ASP.NET MVC!", result.ViewBag.Message); } [TestMethod] public void About() { // Arrange HomeController controller = new HomeController(); // Act ViewResult result = controller.About() as ViewResult; // Assert Assert.IsNotNull(result); }
These two tests are Controller tests, they test the behavior of the Controllers. That’s nice, but testing Controllers isn’t really that much harder than testing ordinary C# classes. That’s one of the great things about MVC. Even without ApprovalTests to help us, Models have also been easy to test using traditional asserts. It’s the View that’s been difficult. There is no built in method to test the Views, and the closest we usually get to them is the ViewResult. That’s what the default tests use, but the ViewResult is not the View, its just another model that MVC uses to create the View. There is nothing wrong with testing the ViewResult, and when you do you are ensuring that your Controller is providing the correct building blocks for the given input, but you should also verify that the View conforms to your expectations. We can trust MVC to find the View, that’s framework code that we shouldn’t need to worry about. But once MVC does find the View, it’s executing code we wrote.
Some argue that the code in Views should be so trivial as to obviate testing. Certainly that is a good strategy for minimizing bugs, but without automated tests you have no safety net against regressions, and no executable specification. Your Views matter… a lot. To your user, the View is the application. I get more positive feedback from changing the background color on a site that I ever get from improving the Model. ApprovalTests can make testing Views easy enough to be practical. Like I said, the learning curve is steep, but it plateaus early. Scramble up the hill with me, Views are worth testing.
NCrunch
I’d like to digress once more to talk a little bit about.
I like NCrunch. The first time I tried it, it didn’t really work with my project. Something to do with assembly signing. When v1.38b arrived on March 7th 2012, I tried it again, and the problem with my assemblies was gone. Its worked more or less flawlessly ever since. I strongly encourage you to try it. I’m going to assume that its active in this project as I build it, although if you download the source and play with the site, everything should work fine without NCrunch.
HERDING CODE 135: REMCO MULDER AND JEFF SCHUMACHER ON CONTINUOUS TESTING
For me, there are two big advantages to running NCrunch:
- First, I get a tighter feedback loop. Often tests run when I stop to think for a moment. I can see right away if any tests are failing, even if I wasn’t planning on running the tests just then.
- Second, I get a second build environment. NCrunch makes its own workspace to build your project in. Compiling in this workspace reveals hidden dependencies and assumptions in your project and tests. In other words, things break in NCrunch that don’t break in Visual Studio, and correcting these breakages can make our code more robust.
So, if you want to give NCrunch a try, its free while in beta. Download it from and follow the easy wizard to enable it in our project.
I enabled it right before pausing to write down some notes about NCrunch, and it executed the two default tests by the time I was done jotting things down. These two tests passed in NCrunch, and then I ran them in Visual Studio and they passed too. So for the time being, we have tests that run consistently in two environments. Later we will setup the project on a build server, and that will provide yet another environment.
I Thought This Post Was About ApprovalTests?
Yeah, yeah, I’m getting there. All right, lets implement a View test using MvcApprovals. First we need to add ApprovalTests to our test project using NuGet. As of this writing, the most recent version is v1.19:
In my mind, a View test is distinct from a Controller test, so I’m going to create a folder in my test project called “Views” and put a Basic Unit Test in the folder called “HomeViewsTest”. Maybe you are not a fan of putting files into arbitrary buckets, that’s cool. You don’t have to have the folder if you don’t want to, just put the test anywhere that makes you happy.
Import these namespaces into your HomeViewsTest:
using ApprovalTests.Asp; using ApprovalTests.Asp.Mvc; using ApprovalTests.Reporters;
Add a UseReporterAttribute (pro tip: you can also create a default reporter by adding an assembly level attribute to AssemblyInfo.cs).
[TestClass] [UseReporter(typeof(FileLauncherReporter), typeof(ClipboardReporter))] public class HomeViewsTest { [TestMethod] public void TestIndexView() { } }
When working with Views, it’s convenient to start with the file launcher reporter so that you can see the rendered results. The clipboard reporter puts a move command into your clipboard that will approve your received file if you paste it into a command window. Later on you can switch to a diff reporter to look for regressions. You’ll also notice that I’ve added the first test and called it TestIndexView.
Now add a line to the test:
MvcApprovals.VerifyMvcPage(new HomeController().Index);
At first, its easy to get confused by an MvcApproval. I freely admit that when I read a line of code like this in ApprovalTests own tests I made some bad assumptions about what was going on with that HomeController instance. So let me decompose this line into something a little less elegant. But first, lets take a look at the method signature for VerifyMvcPage:
public static void VerifyMvcPage(Func<ActionResult> func)
We can see that the argument if a Func<ActionResult>. Well that’s fine, it just so happens that the Index action method is a function without parameters that returns an action result. Lets make that a little clearer:
[TestMethod] public void TestIndexView() { Func<ActionResult> func = new HomeController().Index; MvcApprovals.VerifyMvcPage(func); }
Now its pretty clear that we are passing a delegate to VerifyMvcPage. But if its still unclear, lets break it down again:
[TestMethod] public void TestIndexView() { HomeController controllerInstance = new HomeController(); Func<ActionResult> func = controllerInstance.Index; MvcApprovals.VerifyMvcPage(func); }
So now its clear that we create a controller instance, then we create a delegate pointing at that instance’s Index method, then we pass that delegate as an argument to VerifyMvcPage. But what’s not so clear is that “controllerInstance” has no impact on the test result. It’s only purpose in life to generate the func delegate. Furthermore, the func delegate is never invoked. It’s only purpose in life is to deliver metadata about the method (such as the method name and the name of the type it is declared on) to MvcApprovals. The same metadata could be delivered using a MethodInfo instance from the System.Reflection namespace, but there would actually be more ceremony involved in getting a MethodInfo instance than there is in creating a Func<T>. If you have ever used Moq, they use a very similar technique.
Under the hood, the MvcApprovals uses the metadata to create a URL. The controller name is inferred from the type that func points at, in this case “HomeController” becomes “Home”. The action is inferred from the method name, in this case “Index”. Once MvcApprovals creates a URL, it passes the string to another method called VerifyUrl. You can call VerifyUrl yourself, but then you would need to maintain the URL strings as well. When you use VerifyMvcPage, ApprovalTests maintains the URLs for you.
The important thing to remember is that the controller instance has no impact on your tests. If you fill up that instance with mocks, set properties, or make any other calls that manipulate the Controller state, it wont change anything in the test results. This is the first “gotcha” when testing Views with ApprovalTests. The reason that none of this has any effect should become clear as we continue this example.
While I’ve refactored this test from one line to three, NCrunch has been quietly running the tests in the background and glaring at me with it’s red “N”, which means I have a failing test. So, lets run the tests in Visual Studio and see what the failure looks like:
Test method MVCTestSite.Tests.Views.HomeViewsTest.TestIndexView threw exception:
System.MissingFieldException: ApprovalTests.Asp.PortFactory.MvcPort is uninitialized.
You are using a method that is using ApprovalTests.Asp.PortFactory.MvcPort,
but you have not set a value for this port first
ApprovalTests knows that the typical setup for ASP.NET development is to run on a port other than 80, but it has no way of knowing which port the development server is listening on. So, lets give that information to MvcApprovals:
[TestMethod] public void TestIndexView() { // Let ApprovalTests know which port to use. PortFactory.MvcPort = 61586; // Only used to generate delegate. HomeController controllerInstance = new HomeController();
// Never invoked. Func<ActionResult> func = controllerInstance.Index;
// Test view. MvcApprovals.VerifyMvcPage(func); }
Now we should be golden, but NCrunch is still notifying me that I have broken tests. Of course, I’m working with ApprovalTests, so I should expect the test to keep failing until I approve some output for the test to Verify. To do that I need to run the test again, but it still errors out before giving me any output to approve. Here is the new error message:
Test method MVCTestSite.Tests.Views.HomeViewsTest.TestIndex
Ok, now that we’ve given MvcApprovals enough metadata, it was able to generate the correct URL and now we can see that. But we can also see that our local machine refused to serve the request. The reason is simple… we haven’t started the web server yet! This is the second gotcha, you have to start the server before you try to run your test. There is no possible way for the test to pass if the server is not running.
You may be about to object that a unit test shouldn’t rely on the web server. Ok, your objection is noted and overruled. Step back and look at what you do when you test a business object in .NET. You never actually test your code. That’s not even possible. What you actually happens when you hit “go” in the test runner is this:
- Your project builds.
- Your test project builds
- Your tests run.
Look at steps 1 & 2. Your code is consumed by the compiler, and the compiler emits MSIL code. In step three, the MSIL code is tested, not your C#/VB.Net code. Yet no one points at the compiler and calls this process an integration test. For MVC, there’s more than one compilation step. Some of the code is pre-compiled, and some of the code is compiled on demand by the web server. For our purposes, the web server is just another compiler host. For the time being it’s the only one that emits the output we want to test.
Maybe you can live with that line of reasoning, maybe not. If you do accept the idea, then you might wonder what kind of wrinkles are introduced by taking a dependency on a webserver. Let’s keep working through our example and find out. Fire up your second compiler by hitting “Start Without Debugging” on your toolbar (or under the Debug menu in Visual Studio). Your configured development server (IIS Express or Cassini) will start in the system tray, and the browser should launch and display the site’s home page.
Run the tests again. They fail because nothing is approved, but this time MvcApprovals was able to connect to the web server and read the page! The file launcher reporter takes the output and shows it to us in the browser.
I’ve circled part of the URL to emphasize that we’re looking at a local file.
I haven’t made any changes to the home page, so this looks like what I expected. I can use the data provided by the clipboard reporter to approve this file using a command window. After re-running the test in the Visual Studio test runner, it passes. NCrunch still shows a failure though because it hasn’t noticed that I approved the file. There are different ways to wake up NCrunch. We can make trivial edits to the file, or we can request a test run by right clicking on a red coverage marker, or by left clicking on the NCrunch notifier (which brings up the NCrunch Test window). All of these work ok, but we can help NCrunch notice the approved file by adding the file to the project. Once the the file is part of the project, NCrunch will notice any time we update the approval and run again. Adding a file to the project also causes NCrunch to wake up, so after adding the approved file, NCrunch turns green.
Happy, happy green.
N + 1
All right, we have one MvcApproval, and we have it working in two build environments, NCrunch and Visual Studio. Our goal is to get it running under CC.NET for a total of three build environments. But before we move on to that challenge, lets create a second MvcApproval and see if we have any opportunities for de-duplication once we have two tests.
We’ll create a test for the About view, since its the only other view our project has at the moment. First, let’s squash our call to VerifyMvcPage back down to one line in the Index test:
[TestMethod] public void TestIndexView() { PortFactory.MvcPort = 61586; MvcApprovals.VerifyMvcPage(new HomeController().Index); }
The About test should look the same:
[TestMethod] public void TestAboutView() { PortFactory.MvcPort = 61586; MvcApprovals.VerifyMvcPage(new HomeController().About); }
We can see immediately that the MVC port assignment is duplicated. Before we try to remove that duplication, lets run the tests. The Index test still passes, but the About test fails because its not yet approved. It looks fine so I approve it and after that both tests pass. (Incidentally, I left the two tests that came with the MVC template in place. And they are still working too.)
Lets try moving the port assignment to the constructor.
public HomeViewsTest() { PortFactory.MvcPort = 61586; }
After this change, the tests still pass. If this site will only ever have one controller, then moving this assignment to the constructor might be enough. I’m a fan of YAGNI but for the purposes of this example lets pretend that we eventually plan on adding more controllers, which would mean more View test classes, and this code would then be duplicated across the separate constructors. So lets move this assignment one more time, this time we’ll put it in a base class that HomeViewsTest can extend. (Since I can see the future, I also know that we will have some additional setup code in the base class before we are done, so bear with me.)
Here’s our new base class:
public class MvcTest { public MvcTest() { PortFactory.MvcPort = 61586; } }
And the current state of the test class:
[TestClass] [UseReporter(typeof(FileLauncherReporter), typeof(ClipboardReporter))] public class HomeViewsTest : MvcTest { [TestMethod] public void TestIndexView() { MvcApprovals.VerifyMvcPage(new HomeController().Index); } [TestMethod] public void TestAboutView() { MvcApprovals.VerifyMvcPage(new HomeController().About); } }
After this refactoring, our tests still pass. So that didn’t seem too hard right? There were a couple gotchas but we overcame them. But so far we’ve just been doing beginner stuff because our two pages only display static content (other than the “Welcome user!” message inherited from the master page). In a real website we’re probably going to need to access some data. So to take things to the intermediate level, we’ll need to introduce some models.
We’ll look at that in the next part of this series, ApprovalTests and MVC Views: Working With Data