Edit: As others pointed already out: String.Split has a good overload for your usecase. The answer below is still correct (as in working), but - not the way to go.
That's because this string.Split overload takes an array of separator chars. Each of them splits the string.
You want: Regex.Split
Regex regex = new Regex("<^>"@"<\^>");
string[] substrings = regex.Split("TestString 1 <^> TestString 2 <^> Test String3");
And - a sidenote:
"<^>".ToCharArray()
is really just a fancy way to say
new[]{'<', '^', '>'}