Flask+cURL concatenates multi-line CSV input into a single string [duplicate]

Question

I have a route in my flask app that starts off like this:

@app.route('/invocations', methods=['POST'])
def predict():
    """
    Do an inference on a single batch of data.
    """
    if flask.request.content_type == 'text/csv':
        X_train = flask.request.data.decode('utf-8')
        X_train = pd.read_csv(StringIO(X_train), header=None).values

To test this path I am sending a POST request to the server from a csv formatted file with multiple lines in it: curl -X "POST" -H "Content-Type: text/csv" -d @health-check-data.csv http://localhost:5000/invocations.

However, to my surprise, when I execute X_train = flask.request.data.decode('utf-8'), I get the contents of the csv concatenated into a single string with newlines removed.

Why is flask (or curl?) doing this, and how do I get around this behavior?

Elliot B. · Accepted Answer · 2018-12-18 22:01:49Z

2

That's just the expected behavior for flask.request.data.decode('utf-8') -- it returns a string.

You can convert that string to a buffer that read_csv can read:

upload = flask.request.data.decode('utf-8')
buffer = StringIO()
buffer.write(upload)
buffer.seek(0)
X_train = pd.read_csv(buffer), header=None).values

However, you can bypass that process by reading the csv file via Flask's request.files.get which returns a buffer:

X_train = pd.read_csv(request.files.get('name'),  header=None).values

edited Dec 18, 2018 at 22:01

answered Dec 18, 2018 at 20:36

Elliot B.

17.7k11 gold badges85 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Aleksey Bilogur Over a year ago

As I understand this answer is actually incorrect. For the second part: I am running POST from the command line with the -d argument, and that data is not being passed through a form, so request.files is empty. For the first part: cURL will strip newlines when passed data with the -d flag (ref). You can avoid this behavior by using the --data-binary flag, which is what I did, and that worked. So the data is already mangled by the time it hits flask.

Aleksey Bilogur Over a year ago

It's unfortunate that the man entry for the curl -d doesn't specify how the encoding is handled. I wonder how else the -d flag mutates input...

Elliot B. Over a year ago

@AlekseyBilogur Ahh, I see that you edited your question after I posted my answer to now say string with newlines removed instead of the original string with no newlines removed -- that is a very different meaning for this question :)

Aleksey Bilogur Over a year ago

Yes, that was a terrible, no good, very bad typo, my apologies!

Collectives™ on Stack Overflow

Flask+cURL concatenates multi-line CSV input into a single string [duplicate]

1 Answer 1

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Linked

Related