Import CSV Utf-8

Question

I'm using this python script for import all the data to my application form a CSV file.

# -*- encoding: utf-8 -*-
#CSV ubication
csv_filename="route"
#django
ruta_django="route"
ruta_project = "route"

import sys,os
sys.path.append(ruta_django)
sys.path.append(ruta_project)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

from gestion.models import Cliente, Oficina, Departamento

import csv
lectorCSV = csv.reader(open(csv_filename), delimiter=';', quotechar='"')

for fila in lectorCSV:
        if fila[0] != 'cliente':
                departamento = Departamento()
                c = Cliente.objects.filter(nombre=fila[0])
                o = Oficina.objects.filter(nombre=fila[1],cliente_id=c[0].pk)
                departamento.cliente_id = c[0].pk
                departamento.oficina_id = o[0].pk
                departamento.nombre_departamento = fila[2]
                try:
                        departamento.save()
                except:
                        pass

In the CSV file there are words like "administración" and when it saves in the database it appears like: "Administraci?n"

i tried with # -- encoding: utf-8 -- but the problem persist.

Answer 1

The csv module outputs str objects, whild Django uses and expects unicode strings. You need to convert the strings from the CSV document to unicode strings.

If your CSV contains UTF-8 encoded strings, use .decode('utf-8'):

c = Cliente.objects.filter(nombre=fila[0].decode('utf-8'))
o = Oficina.objects.filter(nombre=fila[1].decode('utf-8'), ...

You can replace 'utf-8' with another encoding, of course. More details on unicode and str can be found here.

The # -*- encoding: utf-8 -*- header is used to determine the encoding of strings in the source code of the current file, if they contain special characters. Since all strings in the code are simple ASCII, it is not required here.