2

EDIT I have made huge mess! I'm sincerely sorry! Issue is not about last 2 print statement. Those were added when I run out of idea. Whole issue is about passing title and final_price form check_price to send_email and use them in body and subject.

I've just hit brick wall and don't know here to find answer. I have tried to create web scraper according to some guide on YT. However due to my lack in knowledge and experience im stuck on issue how to pass 2 variable title and final_price from function check_price() to function send_mail()

Everything will work if I'll try to send email without those variables using just plain text.

import os
import smtplib
import requests
from email.message import EmailMessage
from bs4 import BeautifulSoup

EMAIL_ADDRESS = os.environ.get('GMAIL_USER')
EMAIL_PASSWORD = os.environ.get('GMAIL_PASSWORD')

URL_AMZ = 'https://www.amazon.de/Logitech-kabelgebundene-fortschrittlicher-Muskelbelastung-fortschrittliche/dp/B07FNHV4MW/ref=sr_1_3?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=26227QRRLWQHF&keywords=logitech+mx+vertical&qid=1571861245&sprefix=logitech+mx+%2Caps%2C342&sr=8-3'
# URL_MBNK = 'https://www.mbank.pl/serwis-ekonomiczny/kursy-walut/'
URL_GGL ='https://www.google.com/search?rlz=1C1GCEU_plPL839PL839&sxsrf=ACYBGNSHqUQOq6lZRXHyeKLPAd0peUegqg%3A1571862894642&ei=brmwXevuJuaFk74P1Mi12A8&q=euro&oq=euro&gs_l=psy-ab.3..0i71l8.0.0..1236884...0.2..0.0.0.......0......gws-wiz.D2K7kmd_GB8&ved=0ahUKEwjr3eHLnbPlAhXmwsQBHVRkDfsQ4dUDCAs&uact=5'


headers = {"User-Agent":'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'}

def check_price():
    # global title
    # global final_price
    page_amaz = requests.get(URL_AMZ, headers=headers)
    # page_mbnk = requests.get(URL_MBNK, headers=headers)
    page_ggl = requests.get(URL_GGL, headers=headers)

    soup_amaz = BeautifulSoup(page_amaz.content, 'html.parser')
    # soup_mbnk = BeautifulSoup(page_mbnk.content, 'html.parser')
    soup_ggl = BeautifulSoup(page_ggl.content, 'html.parser')


    title = soup_amaz.find(id='productTitle').get_text()
    price = soup_amaz.find(id='priceblock_ourprice').get_text()
    converted_price = float(price[0:-2].replace(',','.'))

    # convert_ratio = soup_mbnk.find(id="currencies").get_text()
    convert_ratio_ggl = soup_ggl.find('div','dDoNo vk_bk').get_text()
    clean_convert_ratio = float(convert_ratio_ggl[0:4].replace(',','.'))

    final_price = converted_price * clean_convert_ratio 

    if(final_price > 200):
        send_email()

    return title, final_price

def send_email():
    title, final_price = check_price()
    msg = EmailMessage()
    msg['Subject'] = f'Zmiana ceny produktu {title}' #% (title)
    msg['From'] = EMAIL_ADDRESS
    msg['To'] = 'I know it's bit to late... but here was my email'
    msg.set_content(f'Cena {final_price} -- Link:{URL_AMZ}') # % (final_price  ,URL_AMZ))

    with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp:
        smtp.login(EMAIL_ADDRESS , EMAIL_PASSWORD)
        smtp.send_message(msg)
        print("Mail Wysłany")
#print(title)
#print(final_price)

EDIT Issue was solved. I,m r thankful for all assistance.

Below fixed and cleaned code.

import requests
import smtplib
import os
from bs4 import BeautifulSoup
from email.message import EmailMessage

EMAIL_ADDRESS = os.environ.get('GMAIL_USER')
EMAIL_PASSWORD = os.environ.get('GMAIL_PASSWORD')

URL_AMZ = 'https://www.amazon.de/Logitech-kabelgebundene-fortschrittlicher-Muskelbelastung-fortschrittliche/dp/B07FNHV4MW/ref=sr_1_3?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=26227QRRLWQHF&keywords=logitech+mx+vertical&qid=1571861245&sprefix=logitech+mx+%2Caps%2C342&sr=8-3'
URL_GGL ='https://www.google.com/search?rlz=1C1GCEU_plPL839PL839&sxsrf=ACYBGNSHqUQOq6lZRXHyeKLPAd0peUegqg%3A1571862894642&ei=brmwXevuJuaFk74P1Mi12A8&q=euro&oq=euro&gs_l=psy-ab.3..0i71l8.0.0..1236884...0.2..0.0.0.......0......gws-wiz.D2K7kmd_GB8&ved=0ahUKEwjr3eHLnbPlAhXmwsQBHVRkDfsQ4dUDCAs&uact=5'


headers = {"User-Agent":'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'}

def check_price():

    page_amaz = requests.get(URL_AMZ, headers=headers)
    page_ggl = requests.get(URL_GGL, headers=headers)

    soup_amaz = BeautifulSoup(page_amaz.content, 'html.parser')
    soup_ggl = BeautifulSoup(page_ggl.content, 'html.parser')

    title = soup_amaz.find(id='productTitle').get_text()
    final_title = title.strip() # variable title contain monstrocity that contained 5 8x \r\n above and belowe title.
    price = soup_amaz.find(id='priceblock_ourprice').get_text()
    converted_price = float(price[0:-2].replace(',', '.'))

    convert_ratio_ggl = soup_ggl.find('div','dDoNo vk_bk').get_text()
    clean_convert_ratio = float(convert_ratio_ggl[0:4].replace(',','.'))

    final_price = converted_price * clean_convert_ratio

    if final_price > 200:
        send_email(final_title, final_price)
   

def send_email(final_title, final_price):
    msg = EmailMessage()
    msg['Subject'] = f'Zmiana ceny produktu {final_title}'
    msg['From'] = EMAIL_ADDRESS
    msg['To'] = 'EMAIL_ADDRESS'

    msg.set_content(f'Cena {final_price} -- Link:{URL_AMZ}') 

    with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp:
        smtp.login(EMAIL_ADDRESS , EMAIL_PASSWORD)
        smtp.send_message(msg)
        print("Mail Wysłany")
check_price()

Later i've hit wall where Send_email() generated error that ValueError: Header values may not contain linefeed or carriage return characters Which was caused by variable title (it was not striped from all empty lines above and below.

One more time big thank's you to all.

Improve this question
5
  • 1
    Can you try a minimal example? With just a function inner, an outer, with the prints? By the way - the prints are out in main, they don't seem to be inside send_email. You could perhaps print them before doing the email message logic, to avoid problems there changing the result. Commented Oct 24, 2019 at 12:25
  • 2
    One further note, when hitting a problem with code on stack overflow, it's probably a good idea to post the exception/error message you are seeing with the code. Commented Oct 24, 2019 at 12:27
  • Sorry, Traceback (most recent call last): File "/mnt/c/_KOD_/github/Web-Scrapper/Scrapper.py", line 58, in <module> print(title) NameError: name 'title' is not defined Commented Oct 24, 2019 at 12:28
  • That is almost certainly because the print is outside the send_email function. Try putting the prints after the call to check_price in send_email. Commented Oct 24, 2019 at 12:28
  • Those two line are not the issue. When i comment them I still got title not defined Commented Oct 24, 2019 at 12:32

5 Answers 5

Reset to default
5

I believe your way of getting the variables title and final_price is just fine, it should work.

However, the last two prints shouldn't work as they are outside of the send_email() function and therefore title and final_price are unknown. Have you tried putting the print inside the function and calling it?

Update:

I see the error now, you are calling send_email() inside check_price(), and check_price() inside send_email(), which can lead to a neverending loop.

To solve this, if the function you wish to call is check_price(), you could make send_email() accept two parameters and pass them when you call it. So, in check_price() you would do:

...
if(final_price > 200):
   send_email(title,final_price)
...

and then send_email would be as follows:

def send_email(title,final_price):
   msg = EmailMessage()
   ...

Update 2:

You are also using an apostrophe inside a single quote string in:

'I know it's bit to late... but here was my email'

you should do:

"I know it's bit to late... but here was my email"
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

I'm sorry for misunderstanding. Those 2 print statement should be commented.
I see @florian-bernard and I answered almost at the same time :) I would add to his answer that you should call send_email() somewhere in order to see the prints, otherwise you are just defining your function, not executing it.
Oh! @Niviral could you explain the error then? I think you assigned the values of the variables just fine.
I have modified code to: if(final_price > 200): send_email(title, final_price) def send_email(title, final_price): msg = EmailMessage() However even if price is above 200 (currently around 320) it wont triger send_mail()
First of all, are you calling check_price somewhere? can you say exactly what are you doing with this file? Also, I saw another error, you are using an apostrophe inside a single quoted string: 'I know it's bit to late... but here was my email', you should use "I know it's bit to late... but here was my email"
|
0

To pass variable(s) into a function, you need to define them inside the brackets when you define the function.

def functionName(variable1, variable2)

And then to use them, pass the values into function when you use it.

send_email(title, price)

After you get those values from the previous function by using:

title, price = check_price()
Improve this answer

Comments

0

What you want to do is make send_email() so that it accepts the two variables you want to pass. So send_email(title, final_price)

And then at the end of your check_price() function, instead of returning the values, you can simply do send_email(title, final_price)

Based on @raquel.horta's reply, you are calling both the functions, inside both the functions. I think you don't mean to call check_price() again in send_email(), you are using it to get the values. But the truth is, it will re-run that entire function to get those values. So remove title, final_price = check_price() from send_email() and simply call send_email from after your if statement.

if (final_price > 200):
    send_email(title, final_price)

OR

If you want to keep your code as is, you need to understand when you do return title, final_price this will create a list with the two values.

You could do:

title = check_price()[0]
final_price = check_price()[1]

I have made a simple example for you to understand this:

def mee():
    a = "This"
    b = "That"
    return a,b

def printt(a, b):
    print("Printing from here")
    print(a)
    print(b)

printt(mee()[0], mee()[1])

And the output:

daudnadeem: daudn$ python3.7 test.py
Printing from here
This
That

I hope that was helpful!

Improve this answer

Comments

0

The problem with you print errors is due to those variables not existing in the global namespace, only the local namespaces of the functions in which you define them.

The below resolves your issue (+ tidied):

import os
import smtplib
import requests
from email.message import EmailMessage
from bs4 import BeautifulSoup

EMAIL_ADDRESS = os.environ.get('GMAIL_USER')
EMAIL_PASSWORD = os.environ.get('GMAIL_PASSWORD')

URL_AMZ = 'long-url-string'
URL_GGL = 'long-url-string'

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'}


def check_price():

    page_amaz = requests.get(URL_AMZ, headers=headers)
    page_ggl = requests.get(URL_GGL, headers=headers)

    soup_amaz = BeautifulSoup(page_amaz.content, 'html.parser')
    soup_ggl = BeautifulSoup(page_ggl.content, 'html.parser')

    title = soup_amaz.find(id='productTitle').get_text()
    price = soup_amaz.find(id='priceblock_ourprice').get_text()
    converted_price = float(price[0:-2].replace(',', '.'))

    convert_ratio_ggl = soup_ggl.find('div','dDoNo vk_bk').get_text()
    clean_convert_ratio = float(convert_ratio_ggl[0:4].replace(',','.'))

    final_price = converted_price * clean_convert_ratio

    if final_price > 200:
        send_email(title, final_price)

    print(title)
    print(price)


def send_email(title, final_price):

    msg = EmailMessage()
    msg['Subject'] = f'Zmiana ceny produktu {title}'
    msg['From'] = EMAIL_ADDRESS
    msg['To'] = 'EMAIL_ADDRESS'
    msg.set_content(f'Cena {final_price} -- Link:{URL_AMZ}') # % (final_price  ,URL_AMZ))

    with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp:
        smtp.login(EMAIL_ADDRESS , EMAIL_PASSWORD)
        smtp.send_message(msg)
        print("Mail Wysłany")
Improve this answer

3 Comments

This resulted in : File "/mnt/c/_KOD_/github/Web-Scrapper/Scrapper.py", line 41, in send_email msg['Subject'] = f'Zmiana ceny produktu {title}' #% (title) File "/usr/lib/python3.6/email/message.py", line 409, in __setitem__ self._headers.append(self.policy.header_store_parse(name, val)) File "/usr/lib/python3.6/email/policy.py", line 145, in header_store_parse raise ValueError("Header values may not contain linefeed " ValueError: Header values may not contain linefeed or carriage return characters
ok - I've edited the above. This error is just because I split your user agent string on to two lines to make it easier to read.
I think you are missing check_price() at the end.
0

Your error arises from the fact that your print statements are outside your functions. Two solutions.

Option 1

title, final_price = check_price() 
print(title) 
print(final_price)

Option 2

Or directly inside your mailing function.

def send_email(): 
    title, final_price = check_price() 
    msg = EmailMessage() 
    msg['Subject'] = f'Zmiana ceny produktu {title}' #% (title) 
    msg['From'] = EMAIL_ADDRESS 
    msg['To'] = EMAIL_ADDRESS  
    msg.set_content(f'Cena {final_price} -- Link:{URL_AMZ}') # % (final_price ,URL_AMZ)) 
    with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp: 
         smtp.login(EMAIL_ADDRESS , EMAIL_PASSWORD) 
         smtp.send_message(msg) print("Mail Wysłany")
    print(title) 
    print(final_price)

Error

Traceback (most recent call last):
  File "/mnt/c/_KOD_/github/Web-Scraper/Scraper.py", line 58, in <module>
    print(title)
NameError: name 'title' is not defined
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.