Return to Answer

websites can and will put multiple title tags - grab first title tag in head only

Source Link

edit approved Nov 2, 2020 at 9:38

With curl and pup:

while IFS= read -r url
do
   printf "%s " "$url"
   curl -sL "$url" | # fetch the page
       pup 'title'head title:first-of-type text{}' # get the text of  the first title tag in head
done < input

With curl and pup:

while IFS= read -r url
do
   printf "%s " "$url"
   curl -sL "$url" | # fetch the page
       pup 'title text{}' # get the text of  the title tag
done < input

With curl and pup:

while IFS= read -r url
do
   printf "%s " "$url"
   curl -sL "$url" | # fetch the page
       pup 'head title:first-of-type text{}' # get the text of the first title tag in head
done < input

Source Link

answered Jun 12, 2019 at 17:34

muru

77.9k
16
212
318

With curl and pup:

while IFS= read -r url
do
   printf "%s " "$url"
   curl -sL "$url" | # fetch the page
       pup 'title text{}' # get the text of  the title tag
done < input