I am starting on a new project at a new job. This is my first time working heavily in Python. Mocking is a whole new beast compared to the hoops I had to jump through in a statically typed language. I took it upon myself to look into the team's unit tests and hopefully upgrade some of them from using Dingus to Mock.
Earlier today, I came across some tests that were checking a conversion class. Specifically, it converted strings of hexadecimal numbers into Mongo ObjectIds (unique identifiers). What I expected to see was a test that verified given a valid hex number, an ObjectId with same hex number would be returned -or- given a bad hex number an error would occur. Instead, all that the tests verified were that an ObjectId was created and returned. In fact, ObjectId was mocked out entirely and so was the hex number!
Now, creating an ObjectId from a string doesn't require going out to a server or anything. Everything is run locally.
I asked about this particular test suite with my new coworkers. Their thoughts were that the actual conversion should be verified using an integration test and being a unit test, all the unit test should do is make sure the code flows from top to bottom as expected and the ObjectId is created and returned. So, basically, the tests only verify that this class interacts with the environment in the expected way.
I have been writing unit tests for a long time. In my experience, I wouldn't be using mocks at all and I would just verify the conversions occurred as expected. This means interacting with the ObjectId class from another module. Perhaps my idea of a unit tests is too encompassing. I have always reserved integration tests for connecting to remote servers, files and whatnot.
The way I look at it, working with ObjectId in this example is no different than working with str or list. Sure, I can mock out str and list, but since they are essential to what my code is doing, mocking them out doesn't make much sense in my mind. The only time I should care about interacting with a dependency is when it can change the outcome of the test.
Is there any value in writing unit tests that simply check the flow of code? Shouldn't unit tests be the result of verifying the behavior/correctness of the code in mind?
So, it's tough to see exactly what's going on without seeing the code, but based solely on your explanation...
I would agree with you. The behavior is what is important, not the flow of the code.
What if someone later on needs to change the flow of the code to support a different case (say, using a function with a different argument that accomplishes the same result); they can do so without breaking the existing tests.
What if you upgrade the library that is being used, and now calling the function actually has a different result than what you want? Your test still works (the function is being called), but what the unit test is actually trying to test does not.
Really, how mocks and tests are used is still a pretty young discipline. The jury is still out over whether unit testing (and the various strategies that are used in unit testing, such as mocking) are even considered "good thing". No doubt, however, I have found myself creating tests not to actually test behavior, but just so that I can say I have the test, and improper use of mocks is a great way to pretend you've created a test when really you've just created a false feeling of accomplishment that your code has now been more rigorously tested.
Python Unit Testing and when to Mock