16

EDIT: I put it in the title, but just realized I didn't mention it in the body. This seems to be specific to Windows.

I'm having a hard time writing output using the csv Python module in a script that works with both Python 2.7 and 3.3.

First try which works as expected in Python 2.7:

with open('test.csv', 'wb') as csv_file:
    writer = csv.DictWriter(csv_file, ['header1', 'header2'])
    writer.writeheader()
    for item in items:
        writer.writerow(item)

However, when that same thing is run in Python 3.3 you wind up with:

TypeError: 'str' does not support the buffer interface

So I change 'wb' to 'wt' and it runs, but now I have an extra blank row every other line in the file.

To fix that, I change:

with open('test.csv', 'wt') as csv_file:

to:

with open('test.csv', 'wt', newline='') as csv_file:

But now, it breaks Python 2.7:

TypeError: 'newline' is an invalid keyword argument for this function

I know I could just do something like:

try:
    with open('test.csv', 'wt', newline='') as csv_file:
        writer = csv.DictWriter(csv_file, ['header1', 'header2'])
        writer.writeheader()
        for item in items:
            writer.writerow(item)
except TypeError:
    with open('test.csv', 'wb') as csv_file:
        writer = csv.DictWriter(csv_file, ['header1', 'header2'])
        writer.writeheader()
        for item in items:
            writer.writerow(item)

However, that has some seriously bad duplication.

Does anyone have a cleaner way of doing this?

EDIT: The test data is simple and has no newlines or anything:

items = [{'header1': 'value', 'header2': 'value2'},
         {'header1': 'blah1', 'header2': 'blah2'}]
Improve this question
3
  • Can't you just use 'w' instead of 'wb' or 'wt'? Commented Apr 24, 2015 at 7:29
  • Are the strings in your items list unicode strings when you're running the script in Python 2? Are the values always ASCII, or could they include extra characters that need to be encoded? Even if you're able to run the same code under both versions of Python, you might not get the same results! Commented Apr 24, 2015 at 8:21
  • @Blckknght - I added the test data to the bottom of the question. It is just ASCII text. Commented Apr 24, 2015 at 8:32

2 Answers 2

Reset to default
8

I've tried a few ways. As far as I can see, simple using 'w' could be a solution:

with open('test.csv', 'w') as csv_file:
    writer = csv.DictWriter(csv_file, fieldnames=['header1', 'header2'], lineterminator='\n')
    # write something
Improve this answer
Sign up to request clarification or add additional context in comments.

11 Comments

If I do that, I still get blank lines every other row. Did you try this in Windows or on something else?
@Tamerz You get extra new lines because you've got extra new lines in your data... .strip() could be what you need.
@Tamerz I tried with some fake data and it turned out good. So I'm thinking there is something wrong with your data, too.
w is the same as wt
@skyline75489 - Added the test data to original question. You can see it is nothing but a couple strings in a dict.
|
8

Here's a simpler generic way:

import sys

if sys.version_info[0] == 2:  # Not named on 2.6
    access = 'wb'
    kwargs = {}
else:
    access = 'wt'
    kwargs = {'newline':''}

with open('test.csv', access, **kwargs) as csv_file:
    writer = csv.DictWriter(csv_file, ['header1', 'header2'])
    writer.writeheader()
    for item in items:
        writer.writerow(item)

The principle here is not to try to fight the differences between Python 2 and 3 but to have conditional code. You can only go so far in writing code without this kind of test, sooner or later you will have to test the Python version.

Improve this answer

3 Comments

I thought maybe getting **kwargs involved may be a good solution. It still is not pretty but is significantly better than all the duplication I had. This will absolutely work in my scenario. Thank you.
I went with the answer that @skyline75489 gave but I still like this for future use. There are times I've needed to do exactly this but didn't know the best way.
@Tamerz: Check out my answer to a similar question. It works in both versions of Python and handles opening files for both reading and writing (plus. like open(), defaults to read mode if one wasn't explicitly specified). It also doesn't require the use of global variables.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.