PyrofORM - Python Object Relational Mapping (ORM)

Over the past week I've been working on an ORM and have been talking about it in its various stages of development on this blog. In this post I will detail its API and follow that with some sexy code examples of the ORM in action.

If you're wondering about the name—PyrofORM—I was trying to think of how to use "Py" and "ORM" together. My thought process followed this rationale: styrofoam, pyrofoam, pyroform, PyrofORM.

Definitions

A definition is everything. It describes how specific data is stored, it deals with the basic read/write/delete/etc functions, it has subclasses to deal with individual rows of data (records), sets of data (record sets), and fields, and it deals with loading and storing the pickled data. The following is a description of the API to the ORM.

  • Definition
      Hooks
    • install() - If the cache file is empty, this installs default data.
    • setUp() - This tells the definition which fields to support, their types and their relations.
    • tearDown() - This is a hook that is called in Definition.__del__()
    • Methods
    • read(row_id) - Reads a single row into a Record object, which is similar to a dictionary.
    • write(data={}) - Inserts/updates a row in the cache.
    • delete(row_ids=[]) - Deletes one or more rows from the cache.
    • filter(**filter_fns) - Filters data out of the data set given either values or lambda functions. Returns a new Definition.RecordSet object.
    • __iter__() - Lets the programmer put the definition instance directly into a loop. __iter__ calls Definition.filter() with no filtering functiosn—thus filtering all rows.
    • store() - Stores the data in the cache file.
    • create() - Returns a new empty Definition.Record object.
    • Classes
    • Field - Represents a single allowed field in the data (like a SQL table column)
        Methods
      • default(val) - Allows the programmer to change the default value of a field. Otherwise, the default value of a field is None
      • hasOne(cls, alias, **filter_fns)
      • ownsOne(cls, alias=None, **filter_fns) - Allows one to specify a one-to-one relationship with the definition whose class is cls. If the relation is not aliases then it will be called by cls.__name__. Relations are accessible through Definition.Record.__getattr__ (see below). Ownership means that if a row in the owner definition is deleted, then all related rows in owned definition will also be deleted (cascade delete). The filters are optional to automatically apply filtering functions in the same way that Definition.filter() works.
      • hasMany(cls, alias, **filter_fns)
      • ownsMany(cls, alias, **filter_fns) - Allows one to specify a one-to-many relationship with another definition. See Field.ownsOne for more details.
    • Record - Represents a single row from the data set (like a dictionary / associative array)
        Methods
      • __getitem__(item) - Allows one to access the fields as record["field"] or also access them in numeric order, eg: record[0].
      • __getattr__(attr) - Allows one to access any table relations as record.attr(). See Definition.Field.ownsOne.
      • save() - Saves any data in this record to the definition's data set.
      • delete() - Deletes any data related to this record from the definition's data set.
      • id() - Returns the value of the record's implied primary key if it is saved. Otherwise it raises an IndexError.
    • RecordSet - Represents one or more rows from the data set.
        Methods
      • __len__() - Returns the number of records in the record set, for example: len(recordset)
      • __getslice__(start, end) - Allows the programmer to slice a record set as if it were a list. This can be used to limit the number of results, for example: record_set[:limit].
      • __getitem__(item) - Allows the programmer to access an individual row by its numeric position in the record set. Position 0 will be the first row, etc.
      • next() - Returns the current record in the record set and moves the internal pointer to the next one.
      • delete() - Deletes all records in the record set from the definition's data set.
      • sort(**filter_fns) - Sort the rows in the record set by specific filters. To sort by a single field in ascending or descending order, do the following: recordset.sort(field=-1) (descending order).

Example: Basic Usage

Extending the Definition class is easy. In general all one would overwrite are the hook functions; however, anything can be overwritten. For example, I wrote a nested sets (modified pre-order tree traversal algorithm) implementation using this ORM that required overwriting several of the non-hook methods such as write, save, and delete.

Here is a simple demonstration of how to extend the definition class. A People class will be defined. When that class is instantiated People.setUp() will be called. It describes how data is stored in the rows of the data set of that definition. It also happens to be the place where relations to other definitions are defined. After that, if a cache file doesn't exist for it then the People.install() method will be called to create the initial data set.

#!/usr/local/bin/python

import PyrofORM
import types

class People(PyrofORM.Definition):
    def setUp(self):
        """Set up the fields that this definition recognizes
        and the types that it expects their data to be in."""

        self["name"] = types.StringType
        self["age"] = types.IntType

    def install(self):
        """Install the default data if the cache is empty."""

        peter = self.create()
        peter["name"] = "Peter Goodman"
        peter["age"] = 19
        peter.save()

# now lets instantiate the people class and filter out the people!
for person in People():
	print person["name"], "is", person["age"], "years old"

# output:
# Peter Goodman is 19 years old

As you can see it's very straightforward. One thing that might throw you off is how fields work. Above, we set fields by doing self["field"] = and then we tell it the type it expects by telling it what it equals. Behind the scenes this actually sets self["field"] to be a new Definition.Field() object. To use the Definition.Field.default() function one would do: self["field"].default(val).

Example - Relations

There are several ways that definition relations be accomplished. The first was is by using the built in functions that were described under Definition.Field in the above API (hasOne, ownsOne, hasMany, ownsMany). Another way to describe definition relations would be to make custom filtering functions in the Definiton.Record class (or if in the above case, define a People.Record class which extends Definition.Record) and use those.

It is important to get one point across before describing the relations system: all rows in all data sets have implied primary keys. This means that all rows will automatically have a primary key set to them. There are two ways to access the value of the primary key: record.id() and record["pk"]. By design, the relations system will match a field in the current definition to the primary key field in another definition. However, this behavior can be easily changed.

#!/usr/local/bin/python

import PyrofORM
import types

class People(PyrofORM.Definition):

    def setUp(self):
        """Set up the fields that this definition recognizes
        and the types that it expects their data to be in."""

        self["name"] = types.StringType
        self["age"] = types.IntType

        self["job_id"] = types.IntType

        self["job_id"].hasOne(Jobs, alias="job")

    def install(self):
        """Install the default data if the cache is empty."""

        jobs = Jobs()
        job = jobs.create()
        job["name"] = "Programmer"
        job.save()
        jobs.store() # __del__ is finicky so we will make sure the job is stored

        peter = self.create()
        peter["name"] = "Peter Goodman"
        peter["age"] = 19
        peter["job_id"] = job.id()
        peter.save()

class Jobs(PyrofORM.Definition):
    def setUp(self):
        self["name"] = types.StringType


# now lets instantiate the people class and filter out the people!
for person in People():
    print person["name"], "is a", person.job()["name"]

# output:
# Peter Goodman is a Programmer

In the above example I have added a job_id field onto the People definition and also gave it a relation. The relation maps People.job_id to Jobs.pk (primary key). However, assuming we wanted to change the default behavior of mapping one field to the primary key of another definition then we could use one of the following:

# related self.key to ForeignDefinition.pk
self["key"].hasOne(ForeignDefinition, alias="foreign")

# implied other primary key, equivalent to the above
self["key":].hasOne(ForeignDefinition, alias="foreign")

# map self.key to ForeignDefinition.foreign_key
self["key":"foreign_key"].hasOne(ForeignDefinition, alias="foreign")

# implied self primary key, this table. Maps self.pk to ForeignDefinition.foreign_key
self[:"foreign_key"].hasOne(ForeignDefinition, alias="foreign")

Another thing to keep in mind is that named filters can be passed as a third parameter to the has/ownsOne/Many functions.

Conclusions

With nifty slicing functions, lambdas, and method chaining, Python has allowed me to make a pretty spiffy system with some neat syntax. In my next post I will show the code implementation of nested sets for a category system and release the code. Until then, tell me what you think of the API to the ORM.


Comments

There are no comments, but you look like you have something on your mind.


Comment