Elegant Way To Store Dictionary Permanently With Python?

Currently expensively parsing a file, which generates a dictionary of ~400 key, value pairs, which is seldomly updated. Previously had a function which parsed the file, wrote it to

Solution 1:

Why not dump it to a JSON file, and then load it from there where you need it?

import json

withopen('my_dict.json', 'w') as f:
    json.dump(my_dict, f)

# elsewhere...withopen('my_dict.json') as f:
    my_dict = json.load(f)

Loading from JSON is fairly efficient.

Another option would be to use pickle, but unlike JSON, the files it generates aren't human-readable so you lose out on the visual verification you liked from your old method.

Solution 2:

Why mess with all these serialization methods? It's already written to a file as a Python dict (although with the unfortunate name 'dict'). Change your program to write out the data with a better variable name - maybe 'data', or 'catalog', and save the file as a Python file, say data.py. Then you can just import the data directly at runtime without any clumsy copy/pasting or JSON/shelve/etc. parsing:

from data import catalog

Solution 3:

JSON is probably the right way to go in many cases; but there might be an alternative. It looks like your keys and your values are always strings, is that right? You might consider using dbm/anydbm. These are "databases" but they act almost exactly like dictionaries. They're great for cheap data persistence.

>>>import anydbm>>>dict_of_strings = anydbm.open('data', 'c')>>>dict_of_strings['foo'] = 'bar'>>>dict_of_strings.close()>>>dict_of_strings = anydbm.open('data')>>>dict_of_strings['foo']
'bar'

Solution 4:

If the keys are all strings, you can use the shelve module

A shelf is a persistent, dictionary-like object. The difference with “dbm” databases is that the values (not the keys!) in a shelf can be essentially arbitrary Python objects — anything that the pickle module can handle. This includes most class instances, recursive data types, and objects containing lots of shared sub-objects. The keys are ordinary strings.

json would be a good choice if you need to use the data from other languages

Solution 5:

If storage efficiency matters, use Pickle or CPickle(for execution performance gain). As Amber pointed out, you can also dump/load via Json. It will be human-readable, but takes more disk.