Timeline for Should we design our code from the beginning to enable unit testing?
Current License: CC BY-SA 4.0
25 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 4, 2019 at 5:58 | comment | added | rasmus91 | Also: no matter how final you believe your design is, in my experience there's always bound to be a change, and it is almost always so far into the future one barely recalls why the old code looks the easy it does. At that point unit testing really starts to matter, as well as the initial approach. Always use interfaces as to code by addition and NOT by modification. | |
| Feb 1, 2019 at 10:17 | comment | added | Flater | @kukis: There's a difference between a tradeoff and an abuse warning. In the end, all that counts is total hours spent on the project. Unit testing will on average lower that total; and the benefits will be more pronounced with larger projects. Unit tests do not prevent all types of bugs. Unit tests simply protect you from breaking changes to the intended behavior (i.e. the business logic). It does not protect against runtime failures, technical limitations, developer misunderstandings, or badly designed business decisions (i.e. not bad code, but a bad analysis) | |
| Feb 1, 2019 at 7:47 | comment | added | Christian Hackl | @MrSpudtastic: Well, then that shows once again that anecdotes and personal experience are a poor substitute for scientific research and broad studies, for how can it be that your experience is so vastly different to mine? I can just repeat what I said above: "As always in software engineering, every good practice and every good rule has its share of liabilities." Part of the reason why I'm so harsh on unit tests are the cargo cult and absolutism surrounding them, which makes it very important to remind people of their disadvantages, much more so than with other types of testing. | |
| Jan 31, 2019 at 18:22 | comment | added | MrSpudtastic | @ChristianHackl I have experienced exactly the situation that Hughes describes, once spending a full two months just trying to sort out which project(s) caused the integration tests to fail, and which team(s) should address it. If there'd been effective unit tests in place, I could have just run those tests and said "Oh, that's where our code is failing" or "Well, our code isn't failing, so it must be at another layer" | |
| Jan 31, 2019 at 5:28 | comment | added | Christian Hackl | @NathanHughes: I have never experienced anything of the things you mention due to relying on integration tests. Do you have studies, citations or any references to back your claim? I also don't see how integration tests are less interface-friendly than unit tests. If anything, I believe unit tests are more dangerous in this regard, simply because they test smaller things by considering all of a module's inner "units" and not just its outermost API level. | |
| Jan 30, 2019 at 21:20 | comment | added | Nathan Hughes | It's very common for people to think integration tests can serve in place of unit tests. Very common to distrust interfaces and write specifically to an implementation thinking you're simplifying your life. And very common for these projects to get mired in long-running, fragile, high-maintenance integration tests that fail to attain decent coverage or detect bugs. Every step seems perfectly reasonable up to the day you find yourself up to your neck in the tarpit saying "how did i get here?" | |
| Jan 30, 2019 at 15:20 | comment | added | kukis | Why so rarely anyone suggests that writing unit tests is a trade-off? The industry nowadays religiously imposes writing unit tests on every developer. I've been to projects and teams where we spent more time writing unit tests than actual code. Was it worth it? It is hard to tell. We still had some bugs in production, like we would have without unit tests. It was still hard to introduce new features (one could argue it was even harder because you had to readjust unit tests). | |
| Jan 30, 2019 at 14:35 | comment | added | Flater | @ChristianHackl: The investigation for the precise problem will generally be aided by unit tests, which are able to localize issues in particular components. That's the point of having unit tests to begin with. The point of automated testing is to prevent having to sniff around until you stumble on the source of the problem. I'm not saying unit tests catch everything (otherwise we'd have no need for other types of tests), I'm saying that integration tests shouldn't be used to also catch things that unit tests are designed to catch. | |
| Jan 30, 2019 at 14:16 | comment | added | Christian Hackl | @Flater: When an integration test fails, it's typically easy to start a more thorough investigation of the problem in order find out the precise source of the problem, for example by using a debugger or analysing log output. Good integration tests are determinstic and deliver reproducible results (just like unit tests, by the way). | |
| Jan 30, 2019 at 14:07 | comment | added | Christian Hackl | @Flater: You say unit tests cannot break encapsulation because when you don't access anything internal, then you don't break anything. Well, by that logic, no code at all can break encapsulation, because the compiler prevents you from doing so anyway. The point is, of course, that something has been made public that should have been private in the first place (hence the broken encapsulation). This can also happen with the tiniest of classes, not just what you call "god classes". | |
| Jan 30, 2019 at 13:06 | comment | added | Flater |
@KonradRudolph it’s extremely common to find code that breaks encapsulation to let unit testing be done on its entrails This is code which was not developed with testing in mind, and when testing was introduced, was not actually refactored. The blame here lies with the developer (or manager) who refuses to refactor the code. The blame does not lie with the concept of unit testing.
|
|
| Jan 30, 2019 at 13:03 | comment | added | Flater | @KonradRudolph: When integration testing instead of unit testing, it only makes sense to do so if the object that needs to be tested is uniquely used by the consumer with which it is tested in the integration test. If it is used by multiple consumers, then the failure of the integration test of consumer A (which uses library Z) does not prove library Z has a fault - it could just as well be that consumer A is doing something wrong (and consumer B's integration tests do actually pass). In essence, you can no longer be sure which part of the integration failed, only that something failed. | |
| Jan 30, 2019 at 12:56 | comment | added | Flater | @ChristianHackl: Unit tests cannot break encapsulation. Unit testing only cares about the public input and output of the object that's being tested. It does not care about the internal workings. If you're in a situation where you want to test smaller chunks than the current object allows (because it hides everything privately), then you've not sufficiently abstracted your code and are dealing with a god class. The desire to want to test smaller chunks proves that these chunks are separate parts of the logic with their own value/responsibility. | |
| Jan 30, 2019 at 12:54 | comment | added | Flater | @MatthieuM.: You're correct that we should never create "only for test" code in the business logic. However, that's not what OP's question is about. OP's question is about refactoring the code so that it can be tested (which happens in the test project). It makes no sense to create a hook that is not used in production, because that inherently means that this hook is not actually part of the business logic and thus irrelevant as to the question of whether the application does what it's supposed to do (which is the question testing aims to answer). | |
| Jan 30, 2019 at 12:34 | comment | added | Konrad Rudolph | @opa “should” vs “often do in practice”. I completely agree with you that a unit test that breaks encapsulation shouldn’t be written. In practice, it’s extremely common to find code that breaks encapsulation to let unit testing be done on its entrails. | |
| Jan 29, 2019 at 18:08 | comment | added | gidds | Agreed on how easy it is to underestimate the importance of unit tests. (One anecdote: I rewrote a 10K-line system in a different language, and it was later put live without my knowledge or any user testing. I put its almost complete success (only one minor issue, which took a week to notice) down to several hundred unit tests.) But as @Malcolm says, it's a trade-off; some code tweaks may well be worthwhile, but not doing major surgery to squeeze the last few % of coverage. | |
| Jan 29, 2019 at 17:18 | comment | added | Krupip | @ChristianHackl Unit tests should never break encapsulation, if you are trying to access private, protected or local variables from a unit test, you are doing it wrong. If you are testing functionality foo, you are only testing if it actually worked, not if local variable x is the square root of the input y in the third iteration of the second loop. If some functionality is private then so be it, you're going to be testing it transitively anyway. if it's really big and private? That's a design flaw, but probably not even possible outside of C and C++ with header implementation seperation. | |
| Jan 29, 2019 at 16:29 | comment | added | Baldrickk | @ChristianHackl why would a unit test break encapsulation? I've found that for the code I've worked on, if there is a perceived need to add extra functionality to enable testing, the actual problem is that the function you are to test needs refactoring, so all the functionality is at the same level of abstraction (it's differences in abstraction level that usually create this "need" for extra code), with lower level code moved to their own (testable) functions. | |
| Jan 29, 2019 at 15:56 | comment | added | Malcolm | @KonradRudolph I fully second that. In general, the answer says that changing the code for the sake of tests is OK because tests are important. They are, but to what extent should we modify the tested code? We aren't necessarily improving the code, we are making it easier to write tests at the expense of readability, maintainability, simplicity, and so on. In some cases it is a justified trade-off, but not always. So I think the answer doesn't fully address the question and maybe even gives some wrong ideas. | |
| Jan 29, 2019 at 15:17 | comment | added | Konrad Rudolph | @Lee Sure but you need to consider whether having a specific type of tests warrants the increase in code complexity. My personal experience is that unit tests are a great tool up until the point where they require fundamental design changes to accommodate mocking. That’s where I switch to a different type of tests. Writing unit tests at the expense of making the architecture substantially more complex, for the sole purpose of having unit tests, is navel-gazing. | |
| Jan 29, 2019 at 13:34 | comment | added | Lee | Integration tests and especially user acceptance tests are very important, but they're much slower. I don't think you can do TDD with just integration tests. Like with most thinks the middle road of a mix of all testing techniques is probably best. | |
| Jan 29, 2019 at 13:24 | vote | accept | Lee | ||
| Jan 29, 2019 at 13:17 | comment | added | Christian Hackl | Unit tests often break encapsulation and make the code under test more complex than would otherwise be required (e.g. by introducing additional interface types or adding flags). As always in software engineering, every good practice and every good rule has its share of liabilities. Blindly producing a lot of unit tests can have a detrimental effect on business value, not to mention that writing and maintaining the tests already costs time and effort. In my experience, integration tests have a much greater ROI and tend to improve software architectures with fewer compromises. | |
| Jan 29, 2019 at 13:16 | comment | added | Matthieu M. | I would argue it depends of the kind of change. There's a difference between making code easier to test, and introducing test-specific hooks that should NEVER be used in production. I am personally wary of the latter, because Murphy... | |
| Jan 29, 2019 at 11:31 | history | answered | Kilian Foth | CC BY-SA 4.0 |