I've many files in a directories structure. I want to extract some string (namely urls) by using a regexp, from those files.
I've tried this :
find . -path "*alder/ * / * .html" -print | xargs sed -n "/http:\/\/[^'\"]*/p" > urls.txt
... but it does not work as expected. The find part works ok, the xargs one, ok but the sed one, no. All I get in urls.txt is the concatenation of all files.
Using your same find command, this will return the URLs that match the regex:
find . -path "*alder/ * / * .html" -exec grep -oh "http://[^'\"]*" {} +
Unlike find...-print | xargs command..., this approach will work on files whose names contain whitespace or other difficult characters
The -o option to grep tells it to return only the matching part, not the line that the match is on. -h tells it to omit printing of the file names from which the matches were found.
The find command in the OP only matches files whose names have spaces in the path. Since I suspect that this is not what you want, here is an alternative form of the find command that finds all .html files at any depth under the subdirectories of the current directory whose names end in alder:
find *alder/ -name '*.html' -exec grep -oh "http://[^'\"]*" {} +
To guard against other kinds of bad html files, cas suggests letting whitespace or > also signal the end of a URL and also accepting https as well as http:
find . -path "*alder/ * / * .html" -exec grep -oEh "https?://[^'\"[:space:]>]*" {} +
[:blank:] and a > to the negated character-class in grep, to cope with a few more varieties of (potentially broken) html.
[:blank:] to [:space:] so that \r, \n, \v, and \f would also signal the end of a URL.)
[:space:] but newlines etc won't match anyway because grep only sees one line at a time....so changed it to [:blank:]. BTW, you forgot the > in the char-class...to catch, e.g. <a href=http://unquoted.url/>. probably should use https?: instead of just http: too. None of this will catch relative URLs, though. Which is, of coursse, one of the many reasons why a HTML parser should be used instead of grep.
space seems worth keeping even if only because of the \r characters that DOS adds.)
Thanks for all the quick answers. Sorry for the extra spaces in the path, but removing them turns the path into something odd when trying to post here.
I wanted to use sed for grep is really, really slow on more than 1.2 gB, 25,000 files.
I've found an answer. A first sed command to chop the files with "'= chars, and then a second sed command to print the lines. Here is it :
find . -path "*alder.com/ * / * .html" -print| xargs sed -r "s/[\"'=]/\n/g"| sed -n "/^http\s?:/p" > urls.txt
http://on it, the entire line, is that what you actually want? If not, isseda requirement, or would solutions that do not involvesedalso be acceptable?-path "*alder/ * / * .html"looks suspicious. What files are you really trying to find?