pickling error in python?

Question

I am getting this error, and I dont know what it means. How can I fix this problem?

my code looks like this, I've used it before and it has worked:

parentdir = os.getcwd()
dirlist = os.listdir(parentdir)

for dir in dirlist:
    if not dir == "pubs_edits": continue
    if os.path.isdir(os.path.join(parentdir, dir)):
                        os.chdir(os.path.join(parentdir, dir))
                        file_list = os.listdir(os.path.join(parentdir, dir))
                        for f in file_list:
                            in1 = open(f, 'r')
                            dict2 = pickle.load(in1)

This is the error message:

    File "/home/md202/pmid_editor.py", line 18, in <module>
        dict2 = pickle.load(in1)
    File "/usr/lib/python2.5/pickle.py", line 1370, in load
        return Unpickler(file).load()
    File "/usr/lib/python2.5/pickle.py", line 858, in load
        dispatch[key](self)
KeyError: '\x00'

Full tacebacks please....is this your own code? How to reproduce? — user2665694
– user2665694, Commented Jun 13, 2011 at 14:40
What are you unpickling? If it's an instance of a class you defined, have you imported the class? — Shawn Chin
– Shawn Chin, Commented Jun 13, 2011 at 14:44
what is the value of in1? Print it out to take a look. It looks like there may be some unicode characters in there that Pickle doesn't like — inspectorG4dget
– inspectorG4dget, Commented Jun 13, 2011 at 14:51
It looks like you are trying to load all the contents of a directory, that might include a file which holds no pickled data. — a1an
– a1an, Commented Jun 16, 2015 at 8:36

nobody · Accepted Answer · 2014-10-14 14:25:45Z

13

This exact error occurred for me when I tried to unpickle (using pickle.loads) a string representation that I had stored in a database via django. Django changed the charactee representation of my string so that pickle.loads(mystring) threw me that error. When I added an explicit string conversion in, it was fine: pickle.loads( str(mystring) )

EDIT: looking at the comments on the original post, I think this is related to the unicode string issue mentioned. I put a normal string into the database, and django gives me back a unicode string that produces this error.

edited Oct 14, 2014 at 14:25

nobody

20.3k17 gold badges59 silver badges80 bronze badges

answered Mar 9, 2012 at 2:24

Matt

1391 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Chris Dutrow Over a year ago

Your answer just helped me with my problem. I was using app engine and had stored the pickled string as a db.Text() property when I should have used db.Blob()

Anto Over a year ago

Thanks Matt, I have the exact error what you specified i tried to save it in db using Django and get this error while unpickling the object.

Demis Over a year ago

presumably this error would not occur on Python 3+?

drevicko · Accepted Answer · 2015-06-16 22:11:04Z

4

I had a similar problem, resulting in KeyError: '\x1f'.

In my case, if was pickled to a gzip file (ie: gzip.open(fileName,'wb')), and I was trying to read it with a normal file object (ie: open(fileName,'rb')).

edited Jun 16, 2015 at 22:11

answered Oct 14, 2014 at 0:38

drevicko

15.3k15 gold badges80 silver badges104 bronze badges

3 Comments

a1an Over a year ago

It seems pickled files play tricky with gzip. My pickled file looked like a gzip (according to 'file') but when I tryed to gunzip it, it returned a tiny (from 200M to 10K) crippled file with no useful data in it (pickle.load gives then the same KeyError: '\x00' issue with such an unzipped file.

drevicko Over a year ago

looks like the file is incomplete or corrupted. If it was pickled in a text format, you could have a look at zcat myfile - does it look reasonble?

a1an Over a year ago

It was probably corrupted, I tried regenerating it and I got no more errors when loading

JayLav · Accepted Answer · 2017-04-26 13:17:15Z

2

Pickle is binary so you have to read it as such. Instead of ('r') try using ('rb') read binary. Also, if your writing the file ensure you are wrting the pickle file as binary as well ('wb'). That should work, hope it helps.

answered Apr 26, 2017 at 13:17

JayLav

1011 silver badge8 bronze badges

1 Comment

user2677280 Over a year ago

dunno why you are downvoted, this is the answer I was looking for. an easy mistake to make!

nZeeR · Accepted Answer · 2011-06-13 22:09:28Z

1

try

pickle.loads()

answered Jun 13, 2011 at 22:09

nZeeR

312 bronze badges

1 Comment

EliadL Over a year ago

For the life of me I can't understand why this fixed it for me.

Trufa · Accepted Answer · 2011-06-13 15:46:11Z

-2

maybe you should try another protocol try pickle.load(in1, 2) !

edited Jun 13, 2011 at 15:46

Trufa

40.9k44 gold badges133 silver badges194 bronze badges

answered Jun 13, 2011 at 14:47

blob42

5492 silver badges12 bronze badges

2 Comments

blob42 Over a year ago

Take a look at the module documentation link, the load function takes an optional parameter which defines how data is pickled .

a1an Over a year ago

According to the doc, load() takes no optional parameter, only dump does.

Collectives™ on Stack Overflow

pickling error in python?

5 Answers 5

3 Comments

3 Comments

1 Comment

1 Comment

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

3 Comments

1 Comment

1 Comment

2 Comments

Linked

Related