I see this type of mock-heavy testing in lots of places. I used to do the same thing too, but I found that it basically just tightly couples your test to your implementation details. The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass. If you so closely tie your test to your implementation you lose that.
In the example, the tests now really care about how data is being retrieved from the database. Instead, I'd use an in-memory database (or even a Dockerized one). That would then also test that the right queries are being done, but you could still refactor the internals of UserServiceImpl (terrible name btw) and your tests wouldn't fail.
I tend to do mock-heavy testing in Java projects, and it has burned me several times. You write mock tests along your architectural boundaries, but if those need to change due to a deeper refactoring, all your tests need to get refactored as well.
Mock testing essentially tests expected side effects. A more powerful concept is the use of pure functions wherever possible, so that your tests compare input/output pairs, instead of long, difficult to maintain and sometimes non-exhaustive lists of expected side effects.
Does anybody know how one can replace Services, Controllers, Presenters and other such imperative mediator objects with a more functional approach? I'm just speculating, but that should make test maintenance easier.
You might be interested in the "free monad" approach used occasionally in scala and haskell. It separates the effectful operations from the effects by first building a data structure that describes what to do and then interpreting that data structure. The key is that you can interpret the data structure in different ways. That is, one interpreter for production and another for testing. The advantage this has over mocking is that the business logic becomes a collection of pure functions.
Take a look at Gary Bernhardt's talk "Boundaries"[1]. It touches on this very topic and was quite the eye-opener for me (having experienced all the same issues you've described).
Monads. Possibly the Free Monad. Make your business logic pure logic that computes commands, and separate the computation of the command from its execution, then you can use a different execution to test your command chains. http://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Mon... has a basic example, but it works well for replacing your services/controllers/what-have-you too.
I find that you can keep with a class-based approach if you utilise your IOC container in your tests, and only swap out side-effectful classes (ORM, remote service clients, etc) with mocks. That way you don't need to concern yourself with the dependencies of the classes you're testing so you can refactor the underlying relationships and objects with impunity as long as you don't change the interfaces of the classes you're testing.
I don't know if I asked my question as clearly as possible. Mocking your DB, the file system, the remote API, etc are all a given when doing tests. My question was aimed at trying to find out if it's feasible to replace the imperative Presenter/Service/etc patterns with a more functional approach.
What I'd like to see is testing only the returned value of a function, as opposed to calling a void function and checking whether the injected mocks got the proper method invocations, i.e. whether the desired side effects occurred.
if you're calling a void function then by definition you're calling a function that does nothing but side effects or calling off to other classes. In the latter case, they can be good candidates for refactoring in my experience. A function that does something but doesn't either produce mockable side-effects or return information may be a code smell.
A function can return void and still throw an exception. (Like one I wrote today, "insert row into table", which throws on failure and return nothing on success.)
A bunch of functions that all return void all being called in a row, though? That's a much smellier smell. What happens if their order gets mixed up, for example.
Testing that function would be a matter of mocking out the database with a class that can emulate the type of failure that would trigger your exception.
it's almost more worthwhile than testing your happy path - when things go wrong you want to have guarantees that they'll be handled correctly and not kill your server.
"tests now really care about how data is being retrieved from the database".
I'm not sure this correct this case. He mocks the repo. The interface for the repo cares about the intention, the implementation cares about the "how". If your creating a user, you always going to want a "add" to persistent store intention. You don't really care how the add to persistent store is done though, or what persistent store. You just care the service intended to do so.
I have an idea that you if your performing a command, you should mock. The whole points of a command is to coordinate subsystems to perform an action. So you need to check if the right methods were called.
If your performing a query you should stub or in memory db.
> I found that it basically just tightly couples your test to your implementation details
Compilation in strictest form of TDD is the first failing test that you run. Catching this has advantages, such as knowledge and contemplation of altered ABI.
Your interface refactoring could also originate from your tests. Alter the code in your test (even the mock interface) until it is clean at that usage site, and the compile/test run until everything passes.
This is undesirable in my book.
Mere refactoring should just check all boxes in test suite. If I changed tests as well, how would I be sure no functionality changed? Tests did not serve their's purpouse in the end
This whole discussion has been about an "interface refactoring". I.e. a change to internal interfaces so obviously anything that sees that interface is subject to change.
Now it is true that some interfaces are more stable than
others, and you can make a good argument that testing
shouldn't be done at boundaries that are likely to be
unstable.
A test should care about: input, "do something", output.
Then you can refactor the heck out of "do something" and your test (which is ideally documenting the use cases you need to support) doesn't have to change at all.
For me, preferring stubs (dummy implementations of your external dependencies), when possible/appropriate, is a good way to avoid "your test is just a repetition of which method calls your implementation makes":
> The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass.
(assuming the second part of that statement should have been "that your program will still be correct")
I'm back and forth on my agreement/disagreement on this one. One of the major purposes of tests is to have confidence in refactoring, but it isn't the _only_ purpose. Beyond that though, I'd say that unit tests probably aren't the right tool for the job when we're talking about refactoring. The purpose of unit tests is to verify that the unit of code is working in isolation. Integration tests are much better suited for verifying correctness when refactoring. Any good test suite should have a healthy balance of unit tests and integration tests. And, when I say integration tests, I mean whatever you want to call them -- tests that check the system as a whole.
I don't necessarily disagree with you though. Mocks aren't perfect, and they have their own host of problems. I've run into a number of passing tests that should have never been that way because of mocks. It happens. The best way to combat this is to use mocks more sparingly, and have feature tests for many of these cases.
I completely agree with the idea of an "in memory database". I use Entity Framework. I mock the EF context to return regular in memory Lists of objects in unit tests the linq -> sql translation is replace with linq to objects.
I think I would reverse the question. Why does there have to be a UserService interface? If it only exists in order to be mocked in yet another place of the code you've added even more unnecessary layering in your codebase.
I must be one of the few who think this sort of unit-testing is necessary to a certain extent, but in practice seems rather to lend itself to causing terrible code smells all over the place: things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind (rather than whether or not they make the code maintainable or correct), factoring code specifically to increase "code coverage" of the tests, etc.
I think integration and other coarse-grained tests are far more useful more often, and using asserts (which will cause these coarse-grained tests to fail, too) is a better pattern than "unit test all the things".
>but in practice seems rather to lend itself to causing terrible code smells all over the place
I'm working on a code base of about half a million lines at work. We have lots of functional/integration tests, but no unit tests, and the code (C++) is not designed for it.
To add unit tests, we would have to rearchitect a lot of the code.
Last year, as a side project, I took some of our code, rearchitected it, and added unit tests as a proof of concept.
Since I knew I would have to convince management (essentially justifying a lot of churn to add these tests), I had a simple rule:
Every change I make to add unit tests should have some value even if we never add unit tests.
This helped me focus on code changes that I could objectively say improved the code quality, and helped me reject changes that were "just for unit testing".
> things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind
I've done this before. It's very easy to want to make all your function/method calls work the same.
To stop, I applied the following rule whenever I had to choose between making a function static or mockable: does this function's name completely describe its implementation? Small methods like "StringUtils::isEmptyOrWhitespace" pass this test: it's just going to check if the string is empty or only contains whitespace, and that doesn't need to have its implementation swapped out. On the other side, "ImageUtils::detectFacesInImage" goes into no detail about how it does the face detection, and definitely needs an interface to test against.
That said, it all depends. You might allow your users to pick which characters count as whitespace, I don't know.
> I think integration and other coarse-grained tests are far more useful more often!
Yes! Integration tests are the "main" tests. Unit tests are secondary. Use them when you need to debug why your integration tests fail.
You are not alone -- to me, unit tests only make sense when there already exists a well designed set of higher level tests. Otherwise, unit tests produce a large quantity of outputs that does not really help understand the core of the problem.
DHH wrote an article with the title "test induced design damage", not sure if he coined the term.
I think there's a grain of truth there; it is possible, but not necessary, to mangle your architecture to enable testability.
The case that DHH was complaining about is quite interesting in itself; it was an architecture that Weirich suggested (https://www.youtube.com/watch?v=tg5RFeSfBM4) where the framework DB models (in his case Rails) are separated from the Domain Models, to allow you to 1) run your tests of the business logic faster (without spinning up a DB) and 2) decouple your business logic from your choice of framework.
I've felt the need for this factoring in my own code, but actually making it happen is tricky, since you lose a lot of the magic that makes your life easier (but that also couples every level of your architecture to the framework's concepts).
This sort of rigid adherence to a particular paradigm isn't restricted to enterprise developers. I've seen this sort of thing in practice (or attempted) from colleagues in data pipelines for machine learning and other very much not-enterprise settings.
I find such religious devotion rules common among
scientists who have to code even though they are
not programmers by trade. Religious overcommenting
etc. Whatever rule they happened to read in their
FORTRAN-for-beginners book.
ML people are probably more knowledgeable, but their
job is also often more like science than traditional
software engineering.
That doesn't match my experience (as a former academic scientist who moved into software development). The governing rule for us was always "publish or perish." Programming was a means (exploring a problem) to an end (advancing knowledge and publishing the results). I, at least, never came across the jargon for testing, maintenance, etc. until I moved out of academia and into this industry.
Oh I agree, scientists are far less inundated by software "best practices" than professionals. It's just that those which they have heard of, they often stick to religiously.
Personally I prefer the results of these scientists to much of what I see from real software engineers. It might be wreck, but it is only 1000 lines of wreck -- and not a sophisticated wreck created with powerful modern tools.
Trying to cover your state space with coarse-grained tests in database-backed applications can lead to slow tests suites that are laborious to set up/tear down, which is a barrier to fast build times in CI. This harms productivity both while writing tests and in deployment.
I've heard a lot of people say this but I don't find it to be true. Realistic and loosely coupled tests that require minimal maintenance improve my productivity much, much more than a fast build time does.
JD Rainsberger's proscriptions in particular sound like a recipe for creating a massive amount of test code, a huge maintenance headache and tests that are not at all realistic.
I'm not sure what you mean by "Realistic" tests. Perhaps you mean integrated tests that exercise, say, the entire stack from handling the HTTP request down to the database, if we're talking about a REST API.
If you don't care much about covering large portions of the state space and your tolerance for defects is high, then I'm sure your claim about improved productivity is true.
My tolerance for defects is low, which compels me to try and cover the state space more - happy paths, unhappy paths, etc. I've tried doing this with integrated tests, and found it to be very difficult to do in a timely manner. When I switched to using DI, coding to interfaces, and using mocks across architectural boundaries, my ability to cover the state space went up, and my time to deliver code with few defects went down.
I think this is because often the code I care about exercising is limited to a small section of the call stack. In an integrated test, I have to set up too much state and have the machine do too much irrelevant work just to verify this small portion of the code. With DI, interfaces, and mocks, I can isolate just this behavior much more quickly.
That leaves coupling and maintenance. This is a valid concern in the mockist style, but I haven't found it to be a limiting factor in maintaining my code. I think it comes down to testing the right things. If you're using mocks to verify a contract between two objects, and you want to change that contract, naturally your tests for the contract will break. In that case, I think it's fine to just delete them and write new tests that establish your contract. This kind of approach isn't right in all situations, but mockist style testing has helped my write some stable code that would have been a real pain to test due to external dependencies otherwise.
EDIT: That is not to say that your tolerance for defects is high, but rather that productivity is relative to your goals. Sometimes the time spent preventing defects and keeping maintainability high isn't worth it if the repercussions don't translate to dollars and cents.
* Where possible, use the real thing - that is, use an actual database in preference to, e.g. mock objects representing an object that you use to interact with a database.
* Where you need to use a mock (when the real thing is too expensive to use), make it a realistic mock.
>Perhaps you mean integrated tests that exercise, say, the entire stack from handling the HTTP request down to the database, if we're talking about a REST API.
Exactly, but if, say, there was some algorithmic code in there that could be exercised realistically without using the HTTP stack then the HTTP stack is not necessarily necessary.
>If you don't care much about covering large portions of the state space
Code coverage is entirely orthogonal to the type of test you are doing.
>your tolerance for defects is high, then I'm sure your claim about improved productivity is true.
You appear to be fairly confused about the distinction between test coverage and modes of testing.
>My tolerance for defects is low
If you put no emphasis on test realism then I suspect that your tolerance will naturally have to be quite high.
>I've tried doing this with integrated tests, and found it to be very difficult to do in a timely manner.
I found this too sometimes. It's a tooling issue. Integration test tooling is more expensive to build initially but it tends to be much more reusable than unit test tooling. SMTP client stubs may be quicker to build but relatively useless in future projects that use different libraries (sometimes even different versions of the same library) whereas a mock SMTP server is useful forever.
>In an integrated test, I have to set up too much state and have the machine do too much irrelevant work just to verify this small portion of the code.
I have no problem with my tests chewing up 10,000% more CPU than yours and taking an extra 5 minutes if they've got even a 2% higher chance of finding bugs. CPU time is cheap, my time is expensive, bugs are expensive. Easy trade off.
>That leaves coupling and maintenance. This is a valid concern in the mockist style, but I haven't found it to be a limiting factor in maintaining my code.
Still means you're writing and maintaining more code. That means more bugs and more work.
> think it comes down to testing the right things. If you're using mocks to verify a contract between two objects, and you want to change that contract, naturally your tests for the contract will break.
Yup, and since changing the contracts between different subsystems is the most critical and important part of refactoring code, you've just tossed out a large chunk of the mainr benefit of having tests - safe refactoring. This is the worst aspect of unit testing IMO - it cements technical debt because refactoring changes contracts that will turn the tests red.
> Code coverage is entirely orthogonal to the type of test you are doing.
I did not say they were equivalent. Rather, I instead made a connection to the effort required to cover the state space in a given mode of testing. I've found it takes more integration tests to cover the state space than it does with unit tests, given the multiplicative nature of code paths in each layer. J.B. explains this in his talk, and I buy his argument, since I've experienced it myself.
For example, you could collapse several different types of failure at a lower layer into one code path in a higher layer using an exception, and verify that the HTTP layer returns the same thing when that exception happens. In this situation, you avoid having to set up the state necessary to cause the exception that the errors have been collapsed to, which may be 3 or 4 layers deeper. This means less test code, and less effort on my part.
> You appear to be fairly confused about the distinction between test coverage and modes of testing.
Nope, as explained above, the effort required to cover the state space is directly influenced by the mode of testing. For example, if you tested via typing on the keyboard with a pencil, you'd have to expend more effort to cover your state space. Of course, integration isn't as bad as pencil-testing, but the point is that one is easier than the other with respect to state space coverage.
Regarding reusable tooling - I've found this true as well. But tooling doesn't solve the whole problem. My main point is the amount of set up necessary to trigger a certain behavior, which tends to be too time consuming for me to justify the time. You can use tooling here as well, perhaps with data builders and object mothers. But that is also too much setup effort for my tastes.
> I have no problem with my tests chewing up 10,000% more CPU than yours and taking an extra 5 minutes if they've got even a 2% higher chance of finding bugs. CPU time is cheap, my time is expensive, bugs are expensive. Easy trade off.
Here you seem to assert that integration tests are more likely to find bugs. This is highly dependent on your skill in designing testable code that can be exercised without irrelevant portions of the stack. However, you do have a point that integration tests can find classes of bugs that unit tests may not (environmental setup, connection issues, etc.). I don't unit test those things of course.
Regarding CPU and the 5 minute difference - try 25 minutes difference. I have seen this in practice, and at that scale it really slows down the team. You can ease the pain with more machines, but I'd rather have focused, fast tests that can run right on my machine in as short a time as possible. I value the fast feedback loop.
> Still means you're writing and maintaining more code. That means more bugs and more work.
I haven't found this to be a problem. Sometimes I view messages between objects as behavior, so if the tests are inspecting that then I would actually want them to break if I change messages between objects. I've often found that if this is a hindrance to refactoring, there are problems in the code itself, not just the tests.
>Yup, and since changing the contracts between different subsystems is the most critical and important part of refactoring code...
I don't think that's true. If I'm changing contracts so often that it's a burden to maintain, I've likely missed an abstraction opportunity. Instead of blaming unit tests, I just fix the problem. Further, refactoring is not equivalent to changing subsystems. It encompasses many more useful activities that improve your code for the better. And further, if you take the stance that object interactions are behavior (sometimes, when useful), then changing that is not a refactor, but a re_work_. My favorite article on refactor vs. rework: http://www.daedtech.com/rewrite-or-refactor/
>I did not say they were equivalent. Rather, I instead made a connection to the effort required to cover the state space in a given mode of testing. I've found it takes more integration tests to cover the state space than it does with unit tests, given the multiplicative nature of code paths in each layer.
This is more about the level of testing than their nature. You can have high level unit tests (not common, but you can) and you can also have low level integration tests (I have many).
I agree that the "combinatorial explosion" Rainsberger talked about is real, sort of (not to integration test, but to automated end to end tests) but it comes with many caveats:
* If your application is, by and large, gluing together a lot of different libraries that are tested independently of your application then you effectively have a test pyramid even if you only write tests for the top layer because the tests for layers beneath are all written by somebody else. i.e. just because you wrote any kind of app on top of the linux kernel means you somewhat are relying upon the kernel being well tested, but that doesn't mean you're the one doing it directly.
* Those components in the 'application pyramid' you are stringing together require testing independently but IMO it's still better to use integration testing all the way down because all of those components are integrating with something else.
* I've found great success mitigating the effects of the combinatorial explosion using randomized property based testing. Even with slow tests you can get very, very far with this.
* The combinatorial explosion can also be mitigated with stronger typing and more frequent and stringent sanity checking which shuts down invalid code paths.
>For example, you could collapse several different types of failure at a lower layer into one code path in a higher layer using an exception, and verify that the HTTP layer returns the same thing when that exception happens. In this situation, you avoid having to set up the state necessary to cause the exception that the errors have been collapsed to, which may be 3 or 4 layers deeper. This means less test code
I agree with this but I still see it as being orthogonal to the kind of test you use to test each layer.
>Nope, as explained above, the effort required to cover the state space is directly influenced by the mode of testing. For example, if you tested via typing on the keyboard with a pencil, you'd have to expend more effort to cover your state space.
>Of course, integration isn't as bad as pencil-testing, but the point is that one is easier than the other with respect to state space coverage.
I personally found that the up front investment for integration tests is usually higher to put the initial infrastructure all in place - but, once you have gotten past an inflexion point, integration tests are typically easier to write.
For unit tests the investment is roughly constant.
>Here you seem to assert that integration tests are more likely to find bugs. This is highly dependent on your skill in designing testable code that can be exercised without irrelevant portions of the stack
Not really. They find more bugs because they test more realistically. When you are testing against a fake model of a database with fake data that you built for a unit test you are going to inevitably miss stuff that you will pick up on if you use a real database with real data. That's just life.
I do think that the less your code is about linking together different pieces of code and systems and the more it is about 'pure algorithmic calculation', the less you have to face this problem. However, with most code in real life the hard part is typically about integrating things together - most business problems aren't facebook's spam filter.
>However, you do have a point that integration tests can find classes of bugs that unit tests may not (environmental setup, connection issues, etc.). I don't unit test those things of course.
Right. I actually track and classify bugs that I see day to day and these are typically way more common than logical bugs. Moreover, it's not like integration tests don't catch logical bugs as well. They do.
>Regarding CPU and the 5 minute difference - try 25 minutes difference. I have seen this in practice, and at that scale it really slows down the team. You can ease the pain with more machines, but I'd rather have focused, fast tests that can run right on my machine in as short a time as possible. I value the fast feedback loop.
I do as well, but I find that until individual tests take > 30 seconds, or regression test suite breakages become very common, the cumulative effect on my productivity is very minor.
If an entire regression test suite takes an hour it doesn't bother me in the slightest. Hell, I used to have a 32 hour regression test suite once (long story) and even that didn't bother me that much - a release schedule of once a day with a small (~10%) chance of skipping a day when there were failures was more than acceptable for the business.
If that project had unit tests they would have caked themselves around a bunch of really horrible API contracts between module layers and they would have required a rewrite pretty much every time we rewrote anything. Faster to run, yes, but vastly more work to maintain.
>I don't think that's true. If I'm changing contracts so often that it's a burden to maintain, I've likely missed an abstraction opportunity. Instead of blaming unit tests, I just fix the problem.
IME it likely means somebody who came before you missed an opportunity. Since I largely work on projects that existed years before I joined them did I don't see the solution of "just fix the API" as viable. One fix leads to another which leads to another and before you know it it's been weeks or months and you haven't delivered anything. Refactoring needs to be staggered.
It's better to start out with the presumption that you will benefit more by not locking down your API contracts more than you need to conserve CPU horsepower.
I've seen this in practice too. Lots of unit tests for small things that actually make the overall codebase brittle.
I think it comes from a pretty decent practice of coding from the tests outward. However I'd argue the long term value of many of those tests is minimal.
Yes. I write lots of unit tests, but they are
mostly part of the coding process: I find they
are superior to the manual testing I would have
to do otherwise every time I add new code.
But as long term protections against regressions, such
tests are less valuable than more coarse-grained
integration tests.
I wish people would stop using that title. They seem to think it is a clever click-grabber, which is probably true for most people, but for those even remotely familiar with the book, Zen and the Art of Motorcycle Maintenance: An Inquiry into Values, opening up the article and finding no application of Pirsig's Metaphysics of Quality is disappointing.
Appropriation of eastern asian (particularly Japanese) cultural terms is an odd trend in the American tech community. I cringe a bit every time I see a listing for a Java/Javascript/.NET/$LANGUAGE "ninja", "samurai", or "sensei".
I know it's not done to be offensive, and maybe defended by saying "we're just honoring their culture" or something like that. It's just kind of awkward, can't we agree on that at least?
I've been writing tests for a long time, and I went through a phase where I depended heavily on DI. While it's still useful in some conditions, I rarely make use of it.
First, DI is simply the least subtle for of IoC available. When all else fails, you can always rely on DI. But, this is something languages should be looking to help developers with. Elixir, for example, supports IoC as part of the language, and is much more elegant than DI [1]. When monkey patching is available, this is also a possible solution (easy to abuse, yes, but suitable in simple cases). Functions as first class values can also help (instead of passing a clock around, you can have a SystemClock.Time func that you can override in your tests, which is similar to what Date.now() is in javascript).
But, perhaps more importantly, and as much as I tried to deny it, integration tests are absolutely and totally worth the trouble. If you unit test two parts independently, you run the very real risk of having them pass under test, but fail once deployed. I've seen more production downtime caused by incorrect assumptions between services than anything else.
Also, lately, I've been writing more and more fuzz tests. I'm probably not very good at it yet, but I think for the couple of projects we did it, it's been a worthwhile effort (moreso when we started...they barely catch anything now since we're all coding much more defensively).
If you make your so-called-'unit' tests too closely coupled to you don't actually work faster though, because you constantly have to refactor them to match the internal implementation.
Your testing feedback might be quicker, but quickly getting useless information (ie. your implementation changed) is just as bad as slower tests, because you need to fix the test, recompile and again before you get useful information.
Also not all 'integration tests' are slow. If you can test using SQlite in memory, instead of pointing at a PG instance that needs to be provisioned, for instance, you can get a lot of your end-to-end testing running really quickly.
I agree. Unit tests can definitely slow you down. Testing against high level interfaces or having full integration tests is better. Then the failure is much more likely to be a genuine failure rather than a legitimate change in implementation.
If you have good logging and debugging tools the failure can be quite quick to pin down. James Coplien suggests liberal use of assertions in code which is a really good idea and something C/C++ programmers used to do a lot of. Assertions can then be as effective as unit tests in terms of localising failure.
Try CREATE TABLE UNLOGGED with postgres. The diversity of RDBMS and SQL dialects defeats the point of integration testing the persistence layer with SQLite.
even this is less clear - for some dbs it's normal to use a sequence for rowid's (vs some sort of autoincrement) and i don't think sqlite offers all the same behavior there.
Yes, integration tests are worth it, but to take your two part example, testing that only those parts talk correctly together should be enough. You shouldn't need an end-to-end test just for integration.
I found the art of testing is more tied to the reality on the ground as opposed to the testing methodology.
1. Do you have a a QA department doing manual or even automated testing?
2. Do you have performance testing?
3. Is the code base large or small?
4. Transactions versus analytics?
5. Social media? Some code doesn't need testing. Facebook doesn't test their code.
Each new project requires a flexibility to understand the reality of time and resources available for testing, the type of application and performance considerations. If one works at a bank or on transactional software then testing needs to verify exactness. If you are working on analytical software then what's another anomaly between friends, errors in your errors, who cares. If you are working on social media then just pump and dump like NodeJS.
That is the art of testing to me. A mind set of one size fits all with respect to the of value unit testing, integration testing, regression testing or manual testing is the opposite of art: it's ideology.
Ouch. It's 2017. I know it's a blog about unit testing, but using SHA-1, let alone unsalted SHA-1, for password hashing even if for illustration purposes is dangerous.
At the risk of spitting into the wind, so is using String in Java for passwords; the actual password-centric interfaces in the JVM use char[] because one can zero-out those, but cannot zero-out immutable Strings
It's a shame that the functional requirements, which were neatly stated in the spec, weren't preserved in the test code.
Rather than "arrange", "act", "assert", I've taken to writing the Gherkin spec inline with the test code, e.g.
@Test
public void onlyExistingEmailsCanResetTheirPasswords() {
//Given an email address which is not in the database
String email = "unknown@example.com";
when(userRepository.findByEmailAddress("unknown@example.com"))
.thenReturn(null);
//When a password reset is requested for that email
UserServiceImpl userService = ...
userService.startResetPasswordProcess("unknown@example.com");
//Then no password reset notification should be sent
verify(notificationService, never()).sendResetPasswordNotification(any());
//And the user should be given a success message so they can't determine if the email address is already present
verify(userInterface, once()).sendMessage("You have been sent Please check your email.");
}
This way you preserve the intent for the programmers that will come after you, but you don't need a complicated Gherkin runner
I find it very unusual to talk with developers who regularly modify their testing practice based on time and money.
The most expensive is to do it all: unit testing, integration testing, component testing, system testing, end-to-end testing, stochastic testing, performance testing.
The art of testing in no small part comes down to the art of money. How much bang-for-the buck does any testing provide given limited resources? I answer that question.
Take the workload difference between in house versus public usage.
If one releases software into the public domain then one has no reasonable expectation as to the kinds of data driving your code. For this kind of general purpose, heavily reused software then code coverage with, unit testing and mocking makes the most sense is my experience.
However, one can also write code for a very specific, limited usage. In those cases I find regression testing with regression data works best. They are not paying me to find bugs to satisfy code coverage.
I find understanding the workload is a significant factor in determining the kinds of testing I use.
It's sad that this is the state of the art when unit testing (but I have not found any viable alternative when the premise is that I have to use JUnit).
In the example, the tests now really care about how data is being retrieved from the database. Instead, I'd use an in-memory database (or even a Dockerized one). That would then also test that the right queries are being done, but you could still refactor the internals of UserServiceImpl (terrible name btw) and your tests wouldn't fail.