Selenium script is returning null to the element

Question

I am running this script that fetches some content after finding an id, which will be populated by AJAX call, it should go to the second URL. After going to the second URL it doesn't find any content with the same id.

Code snippet is here:

from selenium import webdriver
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
import json
from selenium.webdriver.support import expected_conditions as EC

path_to_chromedriver = 'D:\Mangilal\Downloads\chromedriver_win32\chromedriver.exe'  #Change path 
as needed.
browser = webdriver.Chrome(executable_path=path_to_chromedriver)
#Sample list of URLs.

lists_of_url = ['http://facebook.com', 'http://twitter.com', 
                'http://google.com', 'http://youtube.com',
                'http://linkedin.com', 'http://wordpress.org', 
                'http://instagram.com', 'http://pinterest.com',
                'http://wikipedia.org', 'http://wordpress.com',
                'http://slideshare.net', 'http://e-recht24.de', 
                'http://washingtonpost.com', 'http://etsy.com',
                'http://eventbrite.com', 'http://archive.org', 
                'http://cpanel.net', 'http://miibeian.gov.cn',
                'http://sourceforge.net', 'http://telegraph.co.uk', 
                'http://ameblo.jp', 'http://amazon.co.uk',
                'http://ebay.com', 'http://fc2.com',
                'http://free.fr', 'http://bing.com']


for i in range(10):
    url = 'https://www.shareaholic.com/sharecounter?url=' + lists_of_url[i]
    browser.get(url)
    element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')))
    str = ''
    #Finding element here.
    count = browser.find_element_by_xpath('//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')
    str = count.get_attribute('innerHTML')
    print(str)

Output of this is shown in the image, which fetches content for the first URL after that null value:

Does the line 'http://wikipedia.org', 'http://wordpress.com', ', occur the same way in your code? There is a quoting error there — boethius
– boethius, Commented Dec 14, 2017 at 21:37
@boethius there are the links like 100 so in hurry it made something, I am changing it — mlbishnoi
– mlbishnoi, Commented Dec 14, 2017 at 21:41
Is it guaranteed that the xpath you have there will exist on every page? Is it just the logos? — boethius
– boethius, Commented Dec 14, 2017 at 21:47

Buaban · Accepted Answer · 2017-12-15 03:44:35Z

First, str is a built-in function of Python. Do not use it as a variable.
Second, the querystring should be encoded.
Third, you should wait until the element is visible. See my code below.

from urllib.parse import urlencode
...
for i in range(10):
    url = 'https://www.shareaholic.com/sharecounter?' + urlencode({"url":lists_of_url[i]})
    driver.get(url)
    element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')))
    txt = ''
    #Finding element here.
    count = driver.find_element_by_xpath('//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')
    txt = count.get_attribute('innerHTML')
    print(txt)

Collectives™ on Stack Overflow

Selenium script is returning null to the element

Code snippet is here:

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

Code snippet is here:

1 Answer 1

1 Comment

Related