How to add header row to a pandas DataFrame

Question

I am reading a csv file into pandas. This csv file consists of four columns and some rows, but does not have a header row, which I want to add. I have been trying the following:

Cov = pd.read_csv("path/to/file.txt", sep='\t')
Frame = pd.DataFrame([Cov], columns = ["Sequence", "Start", "End", "Coverage"])
Frame.to_csv("path/to/file.txt", sep='\t')

But when I apply the code, I get the following Error:

ValueError: Shape of passed values is (1, 1), indices imply (4, 1)

What exactly does the error mean? And what would be a clean way in python to add a header row to my csv file/pandas df?

Answer 1

You can use names directly in the read_csv

names : array-like, default None List of column names to use. If file contains no header row, then you should explicitly pass header=None

Cov = pd.read_csv("path/to/file.txt", 
                  sep='\t', 
                  names=["Sequence", "Start", "End", "Coverage"])

Answer 2

Alternatively you could read you csv with header=None and then add it with df.columns:

Cov = pd.read_csv("path/to/file.txt", sep='\t', header=None)
Cov.columns = ["Sequence", "Start", "End", "Coverage"]

Answer 3

col_Names=["Sequence", "Start", "End", "Coverage"]
my_CSV_File= pd.read_csv("yourCSVFile.csv",names=col_Names)

having done this, just check it with:

my_CSV_File.head()

Answer 4

Simple And Easy Solution:

import pandas as pd

df = pd.read_csv("path/to/file.txt", sep='\t')
headers =  ["Sequence", "Start", "End", "Coverage"]
df.columns = headers

NOTE: Make sure your header length and CSV file header length should not mismatch.

Answer 5

To fix your code you can simply change [Cov] to Cov.values, the first parameter of pd.DataFrame will become a multi-dimensional numpy array:

Cov = pd.read_csv("path/to/file.txt", sep='\t')
Frame=pd.DataFrame(Cov.values, columns = ["Sequence", "Start", "End", "Coverage"])
Frame.to_csv("path/to/file.txt", sep='\t')

But the smartest solution still is use pd.read_excel with header=None and names=columns_list.

Answer 6

When reading a file without headers, existing answers correctly say that header= parameter should be set to None, but none explain why. It's because by default, header=0, which means the first row of the file is inferred as the header. For example, the following code overwrites the first row with col_names because the first row was read as the header and it was replaced by col_names.

Note that it's assumed that the columns are separated by a space ' ' here.

col_names = ["Sequence", "Start", "End", "Coverage"]
df = pd.read_csv("path/to/file.txt", sep=' ')                   # <--- wrong
df.columns = col_names

To get the correct output, you can do one of the following two things:

• set header=None:

  df = pd.read_csv("path/to/file.txt", sep=' ', header=None)      # <--- OK
  df.columns = col_names
  

• or use names= parameter to assign column names in one function call:

  df = pd.read_csv("path/to/file.txt", sep=' ', names=col_names)  # <--- OK
  

header=None way is often preferred if the number of columns is not known (because it is important that len(col_names) is equal to the number of columns inferred from the file, otherwise only the last column will be read as a column and all preceding rows will be read as index levels) or if the specific column names are not important. For example, calling add_prefix() after read_csv can add prefix to the default column names:

df = pd.read_csv("path/to/file.txt", sep=' ', header=None).add_prefix('col')

---

On the other hand, if the file has a header, i.e. first row in the file is meant to be read as column labels, then passing names= will push the first row as the first row in the dataframe. In that case, if you want to set the column labels during the pd.read_csv call, pass header=0.

import io
data = """
ab,bc
10,2.
"""

df = pd.read_csv(io.StringIO(data), names=['a', 'b'])           # <--- wrong
df = pd.read_csv(io.StringIO(data), names=['a', 'b'], header=0) # <--- OK

Answer 7

Since this is mentioned that we are reading from a csv, so the delimiter should be ','[as default, not need to mention]' and the given file has no header so header=None`

Sample Code :

import pandas as pd
data = pd.read_csv('path/to/file.txt',header=None)
data.columns = ["Sequence", "Start", "End", "Coverage"]
print(data.head()) #Print the first rows