You can encrypt just the portions of the .config file which contain the sensitive material, in this case connection strings.
The answer is aspnet_regiis.exe
The location may vary depending on your system and version of .NET, but a file search should narrow it down.
Let's say you have a program called myapp located in c:\dotnetaps\myapp.
I use the following command to encrypt the "connectionStrings" portion of web.config:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe -pef connectionStrings C:\dotnetapps\myapp
Wait a second: web.config is only for web apps, not local ones, right?
aspnet_regiis.exe only works for web.config, but the format is exactly the same as app.config. Therefore, all you need to do is rename app.config to web.config, run the tool, and rename it back. A little awkward, but nothing that can't be automated by a batch file.
@ Encrypts connectionStrings in app.config
ren C:\dotnetapps\myapp\app.config web.config
path\to\exe\aspnet_regiis.exe -pef connectionStrings C:\dotnetapps\myapp
ren web.config app.config
Important note: Using literally these commands will encrypt the file against the particular system it is running on. You'll want to do a little reading to get it to encrypt against a key that allows you to distribute your code, but this is the general idea.