Spark Sql query works with hardcoded value but not with variable

Question

I have a PySpark program in which there is a SQL query like this:

id = 'abc1'

info = sqlc.sql("SELECT * from df WHERE isn = 'id'")

This query does not work. I am returned with a blank set of values. However, if I hardcode the value, it works.

info = sqlc.sql("SELECT * from df WHERE isn = 'abc1'")

I have tried str(id) just to be sure. How can I resolve this?

Alper t. Turker · Accepted Answer · 2017-12-06 13:50:06Z

2

Use format in Python 3.5 or before:

info = sqlc.sql("SELECT * from df WHERE isn = '{id}'".format(id=id))

or

info = sqlc.sql("SELECT * from df WHERE isn = '{}'".format(id))

Literal string formatting in Python 3.6 and later:

info = sqlc.sql(f"SELECT * from df WHERE isn = '{id}'")

answered Dec 6, 2017 at 13:32

Alper t. Turker

35.3k9 gold badges89 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1