I'm taking some text and want to split it into an array. My goal is to be able to split it into phrases delimited by stopwords (words ignored by search engines, like 'a' 'the' etc), so that I can then search each individual phrase in my API. So for example: 'The cow's hat was really funny' would result in arr[0] = cow's hat and arr[1] = funny. I have an array of stopwords already but I can't really think of how to actually split by each/any of the words in it, without writing a very slow function to loop through each one.
Add a comment
|
3 Answers
Use split(). It takes a regular expression. The following is a simple example:
search_string.split(/\b(?:a|the|was|\s)+\b/i);
If you already have the array of stop words, you could use join() to build the regular expression. Try the following:
regex = new RegExp("\\b(?:" + stop_words.join('|') + "|\\s)+\\b", "i");
A working example http://jsfiddle.net/NEnR8/. NOTE: it may be best to replace these values than to split on them as there are empty array elements from this result.
4 Comments
Jason McCreary
Be leary of single characters. I have updated my answer to include word boundaries.
Jason McCreary
@patrick, in retrospect this is probably not the best way to do this, but it does work. However, it does provide a couple empty elements. I updated my answer to include a sample link.
user113716
At the time when I commented it still didn't work, even after adding the
\b. Now that you've changed it to add case insensitivity, it is closer, but the () is still capturing in your first example (although you've changed it in your test page). Also, since you added \s to the regex, you're now also removing spaces, which doesn't give OP the result requested.Jason McCreary
@patrick, I did make updates after the fact. But the OP has already marked it answered, so I left my inconsistencies. The working example is there for all to fork.
This does a case insensitive .split() on your keywords, surrounded by word boundries.
var str = "The cow's hat was really funny";
var arr = str.split(/\ba\b|\bthe\b|\bwas\b/i);
You may end up with some empty items in the Array. To compact it, you could do this:
var len = arr.length;
while( len-- ) {
if( !arr[len] )
arr.splice( len, 1);
}
1 Comment
Jason McCreary
+1 for compacting the array, this is a problem with using the
split() solution.Quick and dirty way would be to replace the "stop word" strings with some unique characters (e.g. &&&), and then split based on that unique character.
For example.
var the_text = "..............",
stop_words = ['foo', 'bar', 'etc'],
unique_str = '&&&';
for (var i = 0; i < stop_words.length; i += 1) {
the_text.replace(stop_words[i], unique_str);
}
the_text.split(unique_str);
1 Comment
pettazz
That's exactly what I was thinking, but I wanted to see if I could find a quicker method.