When should I nest one module inside another?

In practice, I think the first option makes it much easier to tightly-couple A to B, so I prefer the second option to encourage modularity; however, this taken to an extreme means you end up with a project with a single level of modules (i.e. a single folder that contains all of your modules), and that sounds kind of crazy.

Coming from a C# perspective, there are two impacts of the first option:

1. Assuming B is a private nested class, it can only spoken by A (I use "spoken" rather than accessed because it is the name that is being hidden; references to instances of A can be passed around).
2. B can access private members of A.

As you state, these factors do make for a very tightly coupled pair of classes. Coupled classes tend to be less maintainable, since changes in one class tend to require changes in another class. However, with a nested class, this concern is mitigated by the fact that the nested class's surface is encapsulated inside of the parent class. Thus, changes to the nested class do not impact other classes.

So, I would say a nested class is appropriate when dealing with intentionally coupled classes. E.g., https://stackoverflow.com/a/48879/18192 offers the example of a SortedMap class containing a private TreeNode class. By nesting TreeNode, the class can design a highly specialized TreeNode class for use only by SortedMap, without concern that other users will be using it.

The question is a bit too generic to give a definite answer either way. If you're trying to make this judgement call for yourself, then it mostly depends how much time you have to get something working out the door.

So with a large pinch of salt, and based on a hypothetical A/B situation without any further context:

Advantages of coupling B inside A:

• In the real world, deadlines and business needs trump technical ideology.
• The code may be easier to write.
• The code may be faster to write.
• The overall solution may have slightly less complexity/indirection.
• It will not hinder your ability to create a working, correct solution.
• In some (rare) cases B may need to contain functionality which must never be reused outside of A.
• A pragmatic choice when you just need to get something out the door ASAP.

Disadvantages of coupling B inside A:

• You're effectively treating A and B as the same class.
• You cannot isolate either of them for reuse on their own.
• You cannot unit test them in isolation.
• You risk losing separation and end up with some overlapping responsibilities between A and B.
• Your unit tests may take longer to write.
• You may need more unit tests to get complete coverage.
• Your unit tests may be more complex and need more Set-Up.
• You will not be able to change B without implicitly changing A.
• Whenever you change B you need to think about testing for A.
• If you need to reuse A or B in isolation you will need to refactor.
• Your future risk of incurring "technical debt" is higher in the future when the requirements change.

Unless yoi're talking about two classes with hundreds of lines of code and a lot of complex logic (in which case both classes should be broken down further), then none of this is really a big deal.

If 'polluting namespaces' is a concern, then tuck B into its own subnamespace. I don't know which language you're using, but you can usually have multiple namespaces in the same module/package/dll/etc. Namespaces usually shouldn't be a factor in your decision.

Just remember, for any technical or design decision:

• When time is critical, then the answer to how you choose to structure A and B is completely academic; your main priority in that scenario is always to get something which just works within the time allowed.
• Good design is irrelevant when you hit a deadline with incomplete or broken software. Working (correct) software is always preferable, even when its design and structure stinks. You can always refactor working software, but you can't refactor software which is incomplete or broken.