Schema evolution Python

(11 ratings)

By: Patrick O'Brien

Added: 02 June 2008 Views: 120

Path : Computers

Programming

Python

Keywords: computers python programming code coder language coding schema

Do you like this tutorial? Now you can support our team to add more :

Schema evolution

Over time you'll find yourself having to make changes to your class definitions. If you've already pickled instances of a class that needs changing, you'll likely want to retrieve and update those instances so that they continue to function properly with the new class definition. We already saw some of the errors that can occur when changes are made to classes or modules. Fortunately, the pickling and unpickling processes provide hooks that we can use to support this need for schema evolution.

In this section, we'll look at ways to anticipate common problems and work around them. Because a class instance's code is not pickled, you can add, change, and remove methods without impacting existing pickled instances. For the same reason, you don't have to worry about class attributes. You do have to ensure that the code module containing the class definition is available in the unpickling environment. And you must plan for the changes that can cause unpickling problems: changing the name of a class, adding or removing instance attributes, and changing the name or location of the class definition module.

Class name change
To change the name of a class without breaking previously pickled instances, follow these steps. First, leave the original class definition intact so that it can be found when existing instances are unpickled. Instead of changing the original name, create a copy of the class definition, in the same module as the original class definition, giving it the new class name. Then add the following method to the original class definition, using the actual new class name in place of NewClassName:

Listing 16. Changing a class name: Method to add to the original class definition

def __setstate__(self, state):

    self.__dict__.update(state)

    self.__class__ = NewClassName

When existing instances are unpickled, Python will locate the original class definition, the instance's _setstate_() method will be called, and the instance's _class_ attribute will be reassigned to the new class definition. Once you are sure that all the existing instances have been unpickled, updated, and re-pickled, you can remove the old class definition from the source code module.

Attribute addition and subtraction
Once again, the special state methods, _getstate_() and _setstate_(), give us control over each instance's state and the opportunity to handle changes in an instance's attributes. Let's take a look at a simple class definition to which we will add and remove attributes. Here is the initial definition:

Listing 17. Original class definition

class Person(object):



    def __init__(self, firstname, lastname):

        self.firstname = firstname

        self.lastname = lastname

Let's assume we've created and pickled instances of Person, and now we've decided that we really just want to store one name attribute, rather than separate first and last names. Here is one way to change the class definition that will migrate previously pickled instances to the new definition:

Listing 18. New class definition

class Person(object):



    def __init__(self, fullname):

        self.fullname = fullname



    def __setstate__(self, state):

        if 'fullname' not in state:

            first = ''

            last = ''

            if 'firstname' in state:

                first = state['firstname']

                del state['firstname']

            if 'lastname' in state:

                last = state['lastname']

                del state['lastname']

            self.fullname = " ".join([first, last]).strip()

        self.__dict__.update(state)

In this example we added a new attribute, fullname, and removed two existing attributes, firstname and lastname. When a previously pickled instance is unpickled, its previously pickled state will be passed to _setstate_() as a dictionary, which will include values for the firstname and lastname attributes. We then combine those two values and assign them to the new fullname attribute. Along the way, we eliminate the old attributes from the state dictionary. After all the previously pickled instances have been updated and re-pickled, we can remove the _setstate_() method from the class definition.

Module modifications
A module name or location change is conceptually similar to a class name change but must be handled quite differently. That's because the module information is stored in the pickle but is not an attribute that can be modified through the standard pickle interface. In fact, the only way to change the module information is to perform a search and replace operation on the actual pickle file itself. Exactly how you would do this depends on your operating system and the tools you have at your disposal. And obviously this is a situation where you will want to back up your files in case you make a mistake. But the change should be fairly straightforward and will work equally well with the binary pickle format as with the text pickle format.

About the Author :

Patrick O\'Brien is a Python programmer, consultant, and trainer. He is the author of PyCrust and a developer on the PythonCard project. He most recently lead the PyPerSyst team that ported Prevayler to Python, and continues to lead that project into interesting new territory.

Schema evolution Python
Articles and Tutorials Directory by www.learnfobia.com

Rate this tutorial :