0

I am trying to extract some URLs from a web page using cURL command. Initially, I use the cURL command as below. 

curl www.website.com/

Now, the website contains links to some other websites which am interested in extracting. So, I do a grep on the cURL command as below. 

curl www.website.com/ | grep "<a href=" > new1.txt

It is extracting all lines which have <a href= in them. But am particularly interested only in lines which start with <a href= and end with title=

How can I modify the grep command?

Improve this question
1
  • 2
    If that's all you need, you can do grep "<a href=.*title=" but this can get complicated when parsing HTML. Commented Feb 10, 2014 at 19:31

2 Answers 2

Reset to default
2

This should work:

curl www.website.com/ | grep '^<a href=.*title=$' > new1.txt

This will select all lines that begin with <a href= and end in title=

Just saw Terdon's comment. You can use -P option with grep and make a non-greedy aka lazy alternative as follows:

curl www.website.com/ | grep -P '^<a href=.*?title=$' > new1.txt
Improve this answer
0
2

Keeping in mind that HTML is not a regular language, and parsing it with regular expressions is nigh-impossible, you could try:

... | grep '^<a href=.*title=.*' > ...

Edit: Saw that you specified lines that start with <a href; the caret takes care of that condition.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.