0

I am trying to fetch data from the internet to save it to a csv file.
I am facing a problem with writing to the csv file, the library leaves an empty row in the file

Example

The data is random.org integers in text/plain format.


I'm using urllib.request to fetch the data and I am using this code to get the data and decode it

req = urllib.request.Request(url, data=None, headers={
    'User-Agent': '(some text)'})

with urllib.request.urlopen(req) as response:
    html = response.read()
    encoding = response.headers.get_content_charset('utf-8')
    data = html.decode(encoding)

I am using this line of code to open the csv file :csvfile = open('data.csv', "a")
Writing to the file:
writer = csv.writer(csvfile, lineterminator = '\n') writer.writerows(data)
and of course I close the file at the end


Things I tried and didn't help :

  • Using (lineterminator = '\n') when writing
  • Using (newline = "") when opening the file
  • Defining a "delimiter" and a "quotechar" and "quoting"
9
  • Possible duplicate of CSV file written with Python has blank lines between each row Commented May 25, 2017 at 19:34
  • @HFBrowning Mentioned that newline="'' didn't help Commented May 25, 2017 at 19:35
  • Okay, are you using Python 2.x? You may need to provide more code to explain if your problem is not covered by the post I linked to Commented May 25, 2017 at 19:38
  • I am using Python 3.6.1. On the code side I don't have much to share, but I'll update my urllib request section Commented May 25, 2017 at 19:41
  • Could you please add the relevant part of your script/view to your question? Commented May 25, 2017 at 19:42

1 Answer 1

1

Updated Answer

If when you create the list that is being written to the data list, if you add it as a list so that data becomes a list of lists, then set your line delimiter to be '\n', it should work. Below is the working code I used to test.

import csv
import random

csvfile = open('csvTest.csv', 'a')
data = []
for x in range(5):
    data.append([str(random.randint(0, 100))])

writer = csv.writer(csvfile, lineterminator = '\n')
writer.writerows(data)
csvfile.close()

and it outputs

enter image description here

Sign up to request clarification or add additional context in comments.

5 Comments

as an additional note, this was done with the default line terminator.
I was using binary mode, then I got into problems of converting str to byte(either an error or writing wrong data like 48 and 49), any help ?
you can set the dialect to excel-tab which will cause the numbers > 9 to be separated by a tab but they will appear correctly in Excel. writer = csv.writer(csvfile, dialect = 'excel-tab')
Also, using binary mode didn't help unfortunately.
This works but it keeps using the same value that is getting fetched the first time like : (0,0,0,0,0), I want to figure it out by myself. Thank you <3

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.