Splitting string on white space but preserving quoted substring

You're writing a thing called lexer or tokenizer. It is one of the core components of the programming language implementations like compilers and interpreters. The shell script itself is a programming language.

And why you can't use regex here? Chomsky hierarchy strikes again. The lexer should behave differently only within the string literal. So the lexer syntax here is not a context-free grammar, which is a superset of the regular grammar which the regular expression(regex) can parse. I'm not sure the PCRE can technically parse it as it can do more than regular expression, but please avoid to write dozen lines of regex code nobody can read.

In general lexer impls takes a sequence of bytes or characters and produces a sequence of tokens. You can refer to xshell's implementation which supports bash-like syntax.