There is nothing wrong in using open-source software in your company, it's free, you can adapt it to your needs and sometimes it has more support or documentation than a paid one. The only thing you should have in mind is if the project is relatively new and/or the support it has, as there can be flaws or lack of help if something don't work.
In the case of LibeOfficeLibreOffice vs. Microsoft Office it is a matter of how many people usesuse each one and the incompatibilities that appear when opening one format in another.
Eclipse IDE, Intellij IDEA (Community Edition), mySQL, Inkscape and Notepad++ are some widely known open-source projects. Check more open-source projects here.
And, at least for starting a business, well chosen open-source software is more than a good choice to alleviate your initiall budget.