Lesson 26. Saving Objects With Pickle

Pickle is a Python module that helps you save and load python objects to permanent storage like hard disks. When needed you can restore these files back to objects. You can also share these files with other people who can convert them back to python objects using Pickle's load() function. When we are working with data on Jupyter notebooks or any other IDE, all information is stored in RAM which is volatile. All your data is lost when you close the notebook's kernel or shut your computer down. So saving processed information to disk is indispensable.

Pickle converts objects into a byte stream, a process called serialization, which is sequential binary form of objects. When you pickle an object into a binary file, it could be later converted back to a python object (deserialization). This process comes in handy when you are performing complex processing on python objects like dictionaries, Numpy or Pandas arrays. Pickle module is commonly used for saving trained machine learning models' weights which could be restored for making predictions without resorting to training the model again and again.

Pickling is not limited to single python objects. You can store multiple or nested python objects into a single output file. However, when you are dumping complex information, you should know the order in which you have stored it.

Basic Pickle Operations

import pickle

#Store objects
object_name = "some_object"
filename = open('some_file_name.txt','wb')
pickle.dump(object_name,filename)
filename.close()

#laod objects
infile = open('some_file_name.txt','rb')
inputFile = pickle.load(infile)
infile.close()
inputFile

Examples

Example #1: Saving A Dict

Let's save an employee's information into file using pickle. Run the code below then look in the example folder for the output.

import pandas as pd
import shutil
import glob
import os
import pickle

if not 'script_dir' in globals():
    script_dir = os.getcwd()
    
data_directory = 'data\\'
example_directory = 'PickleExample\\'
target_file_name = 'employee.txt'

target_path = os.path.join(script_dir, data_directory, example_directory, target_file_name)

employee = {"Name": "Sam", "Age":25, "Height": 177, "Country" : "Brazil"}

filename = open(target_path,'wb')
pickle.dump(employee, filename)
filename.close()

#Here is how you will unpickle a saved file.
inputFile = open(target_path,'rb')
loaded_object = pickle.load(inputFile)
inputFile.close()

print(loaded_object)

Example #2: Saving A List

Let's save a list into file using pickle.

grades = [34,53,23,56,67,74,3,33,2,6,7,8,83,34,2,34,64,65]

target_file_name = 'grades.txt'
target_path = os.path.join(script_dir, data_directory, example_directory, target_file_name)

filename = open(target_path,'wb')
pickle.dump(grades,fiename)
filename.close()

#Here is how you will unpickle a saved file.
inputFile = open(target_path,'rb')
loaded_object = pickle.load(inputFile)
inputFile.close()

print(loaded_object)

Now you try it!

Don't copy and past. Type the code yourself!