如何使用pickle保存dict(或任何其他Python对象)?

我已经看了Python文档给的信息，但我还是有点困惑。有人可以发布一个示例代码，编写一个新文件，然后使用pickle将字典转储到其中吗?

835538 次浏览
小开
最佳答案

试试这个:

import pickle


a = {'hello': 'world'}


with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle, protocol=pickle.HIGHEST_PROTOCOL)


with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)


print(a == b)

上面的解决方案没有任何特定于dict对象的内容。同样的方法也适用于许多Python对象，包括任意类的实例和任意复杂的数据结构嵌套。例如，将第二行替换为以下几行:

import datetime
today = datetime.datetime.now()
a = [{'hello': 'world'}, 1, 2.3333, 4, True, "x",
("y", [[["z"], "y"], "x"]), {'today', today}]

也会生成True的结果。

由于某些对象本身的性质，它们不能被pickle。例如，pickle包含打开文件句柄的结构是没有意义的。

小开 
>>> import pickle
>>> with open("/tmp/picklefile", "wb") as f:
...     pickle.dump({}, f)
...

通常情况下，最好使用cPickle实现

>>> import cPickle as pickle
>>> help(pickle.dump)
Help on built-in function dump in module cPickle:


dump(...)
dump(obj, file, protocol=0) -- Write an object in pickle format to the given file.


See the Pickler docstring for the meaning of optional argument proto.
小开 
import pickle


dictobj = {'Jack' : 123, 'John' : 456}


filename = "/foldername/filestore"


fileobj = open(filename, 'wb')


pickle.dump(dictobj, fileobj)


fileobj.close()
小开 
# Save a dictionary into a pickle file.
import pickle


favorite_color = {"lion": "yellow", "kitty": "red"}  # create a dictionary
pickle.dump(favorite_color, open("save.p", "wb"))  # save it into a file named save.p


# -------------------------------------------------------------
# Load the dictionary back from the pickle file.
import pickle


favorite_color = pickle.load(open("save.p", "rb"))
# favorite_color is now {"lion": "yellow", "kitty": "red"}
小开

一般来说，pickle dict将失败，除非其中只有简单的对象，如字符串和整数。

Python 2.7.9 (default, Dec 11 2014, 01:21:43)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import *
>>> type(globals())
<type 'dict'>
>>> import pickle
>>> pik = pickle.dumps(globals())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle module objects
>>>

即使是真的简单的dict也经常会失败。这取决于内容。

>>> d = {'x': lambda x:x}
>>> pik = pickle.dumps(d)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <function <lambda> at 0x102178668>: it's not found as __main__.<lambda>

然而，如果你使用更好的序列化器，如dill或cloudpickle，那么大多数字典都可以被pickle:

>>> import dill
>>> pik = dill.dumps(d)

或者如果你想保存你的dict到一个文件…

>>> with open('save.pik', 'w') as f:
...   dill.dump(globals(), f)
...

后一个例子与这里发布的任何其他好的答案是相同的(除了忽略了dict内容的可选择性之外，其他答案都很好)。

小开 
import pickle


your_data = {'foo': 'bar'}


# Store data (serialize)
with open('filename.pickle', 'wb') as handle:
pickle.dump(your_data, handle, protocol=pickle.HIGHEST_PROTOCOL)


# Load data (deserialize)
with open('filename.pickle', 'rb') as handle:
unserialized_data = pickle.load(handle)


print(your_data == unserialized_data)

HIGHEST_PROTOCOL的优点是文件变得更小。这使得解腌有时要快得多。

重要的通知: pickle的最大文件大小大约是2GB。

替代方法

import mpu
your_data = {'foo': 'bar'}
mpu.io.write('filename.pickle', data)
unserialized_data = mpu.io.read('filename.pickle')

选择格式

对于您的应用程序，以下内容可能很重要:

  • 其他编程语言的支持
  • 读写能力
  • 紧凑性(文件大小)

参见:数据序列化格式的比较

如果您正在寻找一种制作配置文件的方法，您可能需要阅读我的文章Python的配置文件

小开

如果您只想将字典存储在单个文件中，可以像这样使用pickle

import pickle


a = {'hello': 'world'}


with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle)


with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)

如果你想在多个文件中保存和恢复多个字典 缓存和存储更复杂的数据， 使用# EYZ0。 它完成了您在pickle

周围所需的所有其他内容 
from anycache import anycache


@anycache(cachedir='path/to/files')
def myfunc(hello):
return {'hello', hello}
Anycache存储不同的myfunc结果，具体取决于参数

. cachedir中不同的文件，并重新加载它们

更多细节请参见文档。

小开

将Python数据(例如字典)转储到pickle文件的简单方法。

import pickle


your_dictionary = {}


pickle.dump(your_dictionary, open('pickle_file_name.p', 'wb'))
小开

供你参考，熊猫现在有办法拯救泡菜了。

我发现它更容易。

pd.to_pickle(object_to_save,'/temp/saved_pkl.pickle' )
小开

如果你想在不打开文件的情况下在一行中处理写入或读取:

  import joblib


my_dict = {'hello': 'world'}


joblib.dump(my_dict, "my_dict.pickle") # write pickle file
my_dict_loaded = joblib.load("my_dict.pickle") # read pickle file

提交答案