Scraping only text using selenium

Question

I'm trying to scrape the text (dynamic) using selenium which is present as mentioned below in website. I searched all over the internet but I got how to scrape a text when it is present in separate div/p/span or any other tag only. Please help if you have any idea.

<div class='css-87uc0g e1tk4kwz1'>
    Text
    <span>
        4.5 
        <span> ::before </span>
    </span>
<div>

I want to extract only text here but that span tag is not allowing it.

Note: The value of text is dynamic.

KunduK · Accepted Answer · 2021-04-15 14:11:12Z

2

Find the div element using following xpath and use javascript executor and get the first child value.

element=driver.find_element_by_xpath("//div[.//span[contains(.,'Rating')]]")
print(driver.execute_script('return arguments[0].firstChild.textContent;', element).strip())

Or you can simply replace the value.

element=driver.find_element_by_xpath("//div[.//span[contains(.,'Rating')]]")
print(element.text.replace(driver.find_element_by_xpath("//span[contains(.,'Rating')]").text,'').strip())

edited Apr 15, 2021 at 14:11

answered Apr 15, 2021 at 9:29

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Prophet Over a year ago

Thank you for an idea, but is there more generic way to get the element's text itself while it can be different amount of elements inside that element?

KunduK Over a year ago

@Eliyahu : It depends on how the html structured. Java scripts executor found very useful. If the html structure has new line </br> we can use python splitline() to get the value.

KunduK Over a year ago

@Bidhya Pokharel : Please check the updated answer and let me know how this goes?

Bidhya Pokharel Over a year ago

I'm sorry, I updated the question. There is no static text as ' Rating', I had just wrote it down to show there is some span tag. But I did it using the css class and then replaced the rating (4.5) as you mentioned. Thank you for sharing!!

Collectives™ on Stack Overflow

Scraping only text using selenium

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related