The optimal approach is to embed the characters in the string for each locale, as that typically ensures that the context is correct, assuming you have done your research as to what your target audience expects.
Better still is to put the symbols in a separate internationalised string, as different languages use different symbols for the same grammatical scenario. For example, Armenian uses the colon as its full stop, so one would have a 'sentence terminator' string, or a 'word separator' string, which would be blank for many languages.
Each country typically provides a style guide, which dictates all the 'correct' places for punctuation. So, when I started to work out how I would handle quotes for different languages for some web design tools, I first looked at such style guides.
However, while written publications tended to follow style guides, the online world is quite different! Typically, a large number of non-English European and South American sites use US style quotes, as opposed to the guillemets (« ») of their style guides. Just shows how much the US domination of the early web permeates online language usage around the world.
The MIT Foreign Language News and Newspapers: Home has links to hundreds of online sites. Looking at these helped me find the best approach for my dilemma, which was to provide the facility for the site owner to select one of the 19 most popular sets of quotes-embedded quotes combinations appropriate for their target audience.
Chrome tries to automatically use a country's style guide, but fortunately it can be overridden by specifying a locale in the lang attribute of the q tag. This highlights the problem with automatic approaches that don't take into account the real world, but rely upon theory for their implementations.
To the OP, research those online newspaper sites to see what are actually used in various countries, so that you can see which approach will give the more consistent results.
While some languages have traditionally used a different character for the English colon, online usage may target audiences used to that colon. Also, different locales may have different usage, requiring specifying language strings by each full locale, rather than just by language.