Start with logging. This will have the greatest impact. Implement a logging framework into the code base, like Log4Net or similar. Start logging what the code does.
Debugging should be possible locally. If not, work on getting the symbol files (PDBs) so you can debug into 3rd party dlls to gain a complete picture of the issues that are occurring. Tools like WINDBG can point which DLLs are problematic if the system is crashing. One can configure any server to take a memory dump when there is a crash. It's basically a snapshot of what was going on when the problem occurred. One can examine the dumps locally to find clues to what was happening. If debugging is not possible, work on making it possible. Document the steps necessary to debug. Sometime on complex systems, there is quite a lot of setup necessary to fully debug.
Bug tracking...If you not using one, start using one. This goes hand and hand with a proper version control system. Basically, start tracking defects and revisions of your code. Start to build a history of the system.
Perform Static Code Analysis. Invest in a tool like ReSharper. It will quickly point out all possible null reference exceptions and other bad coding practices. It can help get the code in better shape with just a few clicks and automate tedious items like code formatting, variable naming, etc. Measure your code, find out where the hot spots for refactoring via code metrics.
Refactor and Unit tests. I'm going to assume that probably most of the code written is not very testable, so I wouldn't bother trying to add tests for it. Any new code, create a tests project and start writing tests, both unit and integration. If the unit tests fail, fail the build. So, as you refactor, there should be tests. One thing with tests is that one can write a test to call any method and debug into that method without loading up the entire application or code base. This is useful to help troubleshoot issues.
Document any tribal knowledge as needed. The code should be self documenting so comments so be sparse, but many system have "unusual" ways of doing things, point those out in a coding WIKI or other type of informal repository. Also, consider coming up with coding standards and practices. Enforce those standards via a toolset like Resharper. Since most of the code is probably not following any standard and guidelines, implement the standards on new code that is written.
Since your new, I would treat this like a tour of duty. 6 months to 2 years, and then make the choice to stay or move on. Take satisfaction from making things slightly better than the day before.