How to Split a String by Spaces Not Surrounded by Quotes Using Regex

How can I split a string by spaces that are not surrounded by single or double quotes?

Input: "This is a string that \"will be\" highlighted when your 'regular expression' matches something."

To split a string by spaces while ignoring spaces in quoted sections, we can utilize regular expressions with lookaheads and lookbehinds. By defining a regex pattern that selectively identifies valid split points—spaces that are not enclosed by quotes—we can achieve the desired results effectively.

import re

input_string = 'This is a string that "will be" highlighted when your 'regular expression' matches something.'
pattern = r'(?<=^|\s)(?:(?:(?:(?!").)*?)"([^"
]*?)"|(?:'([^']*?)')|(\S+))(?:\s|$)'
result = re.findall(pattern, input_string)
# Flattens the tuples returned by findall
final_output = [item for sublist in result for item in sublist if item]

for word in final_output:
    print(word) # Outputs each word or phrase on a new line.

Causes

• Using simple space-splitting methods returns unwanted results when quotes are present in the string.
• Regular expressions need to account for different scenarios, such as starting and ending quotes and consecutive quoted texts.

Solutions

• Use a regex pattern that incorporates negative lookarounds to ignore spaces surrounded by quotes.
• The following regex pattern can be used: `(?<=^|\s)(?:(?:(?:(?!").)*?)"([^"]*?)"|(?:'([^"]*?)')|(\S+))(?:\s|$)` which allows capturing specified patterns.

Mistake: Not escaping quotes properly in the regex pattern.

Solution: Ensure that quotes are correctly escaped with a backslash when included in regex.

Mistake: Overlooking the handling of empty strings or spaces that might occur at the start or end of input.

Solution: Include anchors (^ and $) in the regex to properly handle edge cases.