MongoUserDict

Basic Usage

By subclassing MongoUserDict you can manage MongoDB documents with your own custom classes. MongoUserDict is itself a subclass of UserDict and supports all dictionary access methods.

To define your class, provide the pymongo database object and the name of the collection:

import mongo_objects

class Event( mongo_objects.MongoUserDict ):
    db = ...          # provide your pymongo database object here
    collection_name = 'events'

CRUD Operations

mongo_objects supports all CRUD (create, read, update, delete) operations.

Create an object by creating an instance of your MongoUserDict subclass:

event = Event( {
    ... document data here ...
} )

Saving a new object will write it to the database and assign it a MongoDB ObjectId. _created and _updated timestamps will also be added to the document:

event.save()

To load multiple documents, use the familiar find() method. Filter, template and other arguments are passed to the underlying pymongo.find(). The resulting documents are returned as instances of your MongoUserDict document subclass:

for event in Event.find():
    ...

Single documents may be loaded with find_one():

event = Event.find_one()

load_by_id() is a convenience method to load a document by its MongoDB ObjectId. The method accepts either a string or BSON ObjectId:

event = Event.load_by_id( '662b0d705a4c657820625300' )

To update an document, simply update the instance of your document class with the usual methods for modifying dictionaries and then save() it. For existing documents (i.e. documents that already have a MongoDB ObjectId), save() automatically uses pymongo.find_one_and_replace() to update the existing document in place.

To prevent overwriting modified data, save() will only replace objects that haven’t already been modified in the database. See the save() method reference for more details on this behavior.

save() updates the _updated timestamp to the current UTC time for all objects successfully saved.

event['new-key'] = 'add some content to the object'
event.save()

Deleting an object is accomplished with the delete() method. The document is deleted from the database and the ObjectId is removed from the in-memory object. If the object is saved again, it will be treated as a new document.

event.delete()

Read-Only Documents

It is possible to use projections that return incomplete documents that can’t be safely be saved by mongo_objects without data loss. mongo_objects doesn’t attempt to determine whether a projection is safe or not.

The find() and find_one() methods accept a readonly keyword argument with three potential values:

  • None (default): All documents created with a projection are marked readonly. All other documents are considered read-write.

  • True: The documents will be marked as readonly.

  • False: The documents will be considered read-write. This is a potentially dangerous choice. With great power comes great responsibility.

save() refuses to save a readonly object and raises a MongoObjectsReadOnly exception instead.

Document IDs

Once a document has been saved and an ObjectId assigned by MongoDB, the id() method returns a string representation of the ObjectId.

load_by_id() may be used to load a specific document by its ObjectId or by the string returned by id():

# load a random document
event = Event.find_one()

# save the ObjectId for later
eventId = event.id()

... time passes ...

# reload the document from its ObjectId
event_again = Event.load_by_id( eventId )

ObjectIds are represented as lowercase hex digits, so the result of id() is safe to use in URLs.

Authorization

mongo_objects does not implement any authorization itself, but does provide the following hooks that the user may override to implement access control over each CRUD action.

Create

Creating new objects is authorized by the authorize_init() method. The method may inspect the contents of the document to see if creating it is allowed. Since reading documents from the database also involves creating a new object, this method will also be called for each find() and find_one() document as well. If the method does not return True, a MongoObjectsAuthFailed exception is raised.

Read

There are two read hooks:

authorize_pre_read() is a classmethod that is called once per find() or find_one() call before any data is read. If the method does not return True, a MongoObjectsAuthFailed exception is raised.

authorize_read() is called after a document is read but before the data is returned to the user. The method may inspect to contents of the document to see if the user is permitted to access this particular document. If authorize_read() does not return True, the document will be suppressed. For find_one(), if the first document found is suppressed, None will be returned. No additional (potentially authorized) documents will be evaluated.

Update

authorize_save() is called by save() before new or updated documents are saved. If the method does not return True, a MongoObjectsAuthFailed exception is raised.

Delete

authorize_delete() is called by delete() before a document is deleted. If the method does not return True, a MongoObjectsAuthFailed exception is raised.

Object Versions

mongo_objects supports an optional document schema versioning system. To enable this functionality, provide an object_version value when defining the class:

class Event( mongo_objects.MongoUserDict ):
    db = ...          # provide your pymongo database object here
    collection_name = 'events'
    object_version = 3

The current object_version will automatically be added by save() to each document as _objver.

By default find() and find_one() will then automatically adjust each query to restrict the results to the current object_version. This guarantees that only objects at the current object version will be returned. This is equivalent to the following:

Event.find( {
    ... other filters ...,
    '_objver' : Event.object_version
    } )

To manage this functionality, find() and find_one() accept an optional object_version parameter with the following meaning:

  • None (default): documents will automatically be filtered to the current object_version

  • False: no filtering for object version will be performed

  • any other value: only documents with this value as the object version will be returned

Object versioning provides a convenient workflow for migrating database schemas and protecting the application from inadvertently reading data in an obsolete format. First increment the object_version of the MongoUserDict document subclass, then loop through all objects at the previous version and perform the migration.

For example, to update the layout of the object defined above:

# object_version is now 4
class Event( mongo_objects.MongoUserDict ):
    db = ...          # provide your pymongo database object here
    collection_name = 'events'
    object_version = 4

# loop through all objects at version 3
for event in Event.find( object_version=3 ):
    ... perform migration steps ...

    # saving the document object automatically updates _objver
    # to the current value
    event.save()

Polymorphism

Subclass the PolymorphicMongoUserDict class to enable mongo_objects support for polymorphic user document classes. Specifically, from the same MongoDB collection find() and find_one() will return instances of different document classes.

Each polymorphic subclass defines a separate key which save() adds to the document. When the document is read back from the database, the key is compared to a list of potential classes and the correct instance type returned.

Note the strong recommendation to define an empty subclass_map so each set of polymorphic classes use their own namespace for subclass keys.

import mongo_objects

# create a base class for the collection
class Event( mongo_objects.PolymorphicMongoUserDict ):
    db = ...          # provide your pymongo database object here
    collection_name = 'events'

    # Recommended: define an empty subclass_map in the base class
    # This creates a separate namespace for the polymorphic
    # subclass keys.
    # Otherwise, subclasses will share the PolymorphicMongoUserDict
    # subclass namespace and risk collisions with other subclasses
    # from other collections.
    subclass_map = {}

    .. your generally useful event methods ...

# now create subclasses for each object variation
# each subclass requires a unique key
class OnSiteEvent( Event ):
    subclass_key = 'onsite'

    .. your onsite-specific event methods ...

class OnlineEvent( Event ):
    subclass_key = 'online'

    .. your online-specific event methods ...

class HybridEvent( Event ):
    subclass_key = 'hybrid'

    .. your hybrid-specific event methods ...

Creating Documents

Create and save the objects using a subclass. save() automatically adds the appropriate subclass key to the document.

hybrid = HybridEvent()
hybrid.save()
# save the document ID for later
hybridId = hybrid.id()

Loading Documents

If you load documents using the base class, all available documents will be returned. The resulting objects will each be an instance of the correct subclass based on the subclass key.

# event is an instance of HybridEvent
event = Event.load_by_id( hybridId )

# retrieve all events as the correct type
Event.find()

If you load documents using a subclass, only those documents of that subclass type will be returned.

# Only hybrid event objects will be returned
HybridEvent.find()

If a document has a missing or invalid subclass key, an instance of the subclass with a None subclass key is returned. If no such subclass is defined, MongoObjectsPolymorphicMismatch is raised.

Proxy Support

MongoUserDict seamlessly supports the subdocument proxy objects also included in this module.

get_unique_integer() provides per-document unique integer values.

get_unique_key() uses these unique integers to create unique string values suitable for subdocument proxy keys.

split_id() separates a full subdocument ID value into its components: a parent document ObjectId plus one or more proxy keys.

The class method load_proxy_by_id() accepts a subdocument ID as generated by id() and a list of one or more proxy classes. The parent document is loaded and the proxy objects are instantiated. See MongoDictProxy and MongoListProxy for more details.

For parent documents already loaded in memory, load_proxy_by_local_id() accepts the proxy ID portion generated by proxy_id() and the related list of proxy classes to instantiate the requested proxy objects.

Class Reference

class mongo_objects.MongoUserDict(doc={}, readonly=False)

Access MongoDB documents through user-defined UserDict subclasses.

User classes must provide collection_name and database or override the collection() method to return the correct pymongo collection object.

collection_name

Required: override with the name of the MongoDB collection where the documents are stored

database

Required: override with the pymongo database connection object

object_version = None

Optional: If object_version is not``None``, find() and find_one() by default restrict queries to documents with the current object_version. This enables a type of schema versioning.

subdoc_key_sep = 'g'

The character sequence used to separate the document ID from proxy subdocument keys. This may be overridden but it is the user’s responsibility to guarantee that this sequence will never appear in any ID or subdoc key. Since the default IDs and subdoc keys are hex, g is a safe, default separator

__init__(doc={}, readonly=False)

Initialize the custom UserDict object flagging readonly objects appropriately.

Raises:

MongoObjectsAuthFailed – if authorize_init() has been overridden and does not return a truthy value

authorize_init()

Called after the document object is initialized but before it is returned to the user.

This hook is called when creating a new object and during calls to find() and find_one().

Override this method and return False to block creating this document.

Returns:

Whether creating the current document is authorized (default True)

Return type:

bool

authorize_delete()

Called before the current document is deleted.

Override this method and return False to block deleting this document.

Returns:

Whether deleting the current document is authorized (default True)

Return type:

bool

classmethod authorize_pre_read()

Called before a read operation is performed. This is a class method as no data has been read and no document object has been created.

Returns:

Whether reading any documents is authorized (default True)

Return type:

bool

authorize_read()

Called after a document has been read but before the data is returned to the user. If the return value is not truthy, the data will not be returned.

Note that find_one() only inspects the first document returned by the underlying pymongo.find_one() call. If the document returned does not pass authorization, no attempt is made to locate another matching document.

Returns:

Whether reading the current document is authorized (default True)

Return type:

bool

authorize_save()

Called before the current document is saved.

Returns:

Whether saving the current document is authorized (default True)

Return type:

bool

classmethod collection()

Return the pymongo.Collection object from the active database for the named collection

For complex situations users may omit the database and connection_name attributes when defining the class and instead override this method.

This method must return a pymongo.Collection object.

classmethod count_documents(filter={}, object_version=None, **kwargs)

Count the documents matching the filter. Arguments are passed through to pymongo.count_documents()

Parameters:
  • filter (dict) – Updated with cls.object_version as appropriate

  • object_version (None, False, any scalar value) –

    Only if cls.object_version is not None, implement object schema versioning.

    1. If None, update the filter to only return documents with the current cls.object_version value

    2. If False, don’t filter objects by cls.object_version

    3. For any other value, update the filter to only return documents with the given object_version

Returns:

the count of matching documents

Raises:

MongoObjectsAuthFailed – if authorize_pre_read() has been overridden and does not return a truthy value

delete()

Delete the current object. Remove the id so save() will know this is a new object if we try to re-save. Other data values are still available in this dictionary even after the data is deleted from the database.

Raises:

MongoObjectsAuthFailed – if authorize_delete() has been overridden and does not return a truthy value

classmethod find(filter={}, projection=None, readonly=None, object_version=None, **kwargs)

Return matching documents as instances of this class

Parameters:
  • filter (dict) – Updated with cls.object_version as appropriate and passed to pymongo.find()

  • projection (dict) – Passed to pymongo.find()

  • readonly (None or bool) –

    1. If None and a projection is provided, mark the objects as readonly.

    2. If None and no projection is given, consider the objects read-write.

    3. If True, mark the objects as readonly regardless of the projection.

    4. If False, consider the objects read-write regardless of the projection.

  • object_version (None, False, any scalar value) –

    Only if cls.object_version is not None, implement object schema versioning.

    1. If None, update the filter to only return documents with the current cls.object_version value

    2. If False, don’t filter objects by cls.object_version

    3. For any other value, update the filter to only return documents with the given object_version

Returns:

a generator for instances of the user-defined MongoUserDict subclass

Raises:

MongoObjectsAuthFailed – if authorize_pre_read() has been overridden and does not return a truthy value

classmethod find_one(filter={}, projection=None, readonly=None, object_version=None, no_match=None, **kwargs)

Return a single matching document as an instance of this class or None

Parameters:
  • filter (dict) – Updated with cls.object_version as appropriate and passed to pymongo.find()

  • projection (dict) – Passed to pymongo.find()

  • readonly (None or bool) – as with MongoUserDict.find()

  • object_version – as with find()

  • no_match (None or any value) – Value to return if no matching document is found

Returns:

no_match value; None by default

Raises:

MongoObjectsAuthFailed – if authorize_pre_read() has been overridden and does not return a truthy value

get_unique_integer(autosave=True)

Provide the next unique integer for this document.

These integers are convenient for use as keys of subdocuments. Starts with 1; 0 is reserved for single proxy documents which don’t have a key.

Parameters:

autosave (bool) – Should the document be saved after the next unique integer is issued

Returns:

The next unique integer for this document

Return type:

int

get_unique_key(autosave=True)

Format the next unique integer as a hexidecimal string

Parameters:

autosave (bool) – passed to get_unique_integer()

Returns:

The lowercase hexidecimal string representing the next unique integer for this document.

Return type:

str

id()

Convert this document’s ObjectId to a string

Returns:

The document ObjectId

Return type:

str

Raises:

KeyError – if the document has not yet been saved and has not been assigned an ObjectId

classmethod load_by_id(doc_id, **kwargs)

Locate a document by its ObjectId

Parameters:
  • doc_id (str or bson.ObjectId) – the ObjectId for the desired document

  • kwargs – passed to find_one()

Returns:

an instance of the current class or None if not found. Invalid ObjectIds return None.

classmethod load_proxy_by_id(id, *args, readonly=False)

Based on a full subdocument ID string and a list of classes, load the Mongo parent document, create any intermediate proxies and return the requested proxy object.

Parameters:
  • id (str) – a full subdocument ID string separated by cls.subdoc_key_sep. It includes the ObjectId of the top-level MongoDB document and one or more subdocument keys as generated by id().

  • args – one or more user-defined proxy classes in descending order, one per subdocument key. The top-level parent MongoUserDict subclass is not included.

  • readonly (bool) – passed to load_by_id()

Returns:

an instance of the final, rightmost proxy subdocument class from args

load_proxy_by_local_id(id, *args, readonly=False)

Based on a local subdocument ID string and a list of classes, create any intermediate proxies within the current document and return the requested proxy object.

Parameters:
  • id (str) – a local subdocument ID string generated by proxy_id() containing one or more subdocument keys separated by cls.subdoc_key_sep.

  • args – one or more user-defined proxy classes in descending order, one per subdocument key.

  • readonly (bool) – passed to find_one()

Returns:

an instance of the final, rightmost proxy subdocument class from args

proxy_id(*args, include_parent_doc_id=False)

Assemble a list of proxy IDs into a single string

Parameters:

include_parent_doc_id (bool) – whether to include the parent document ID in the resulting ID string

Returns:

One or more proxy IDs separated by subdoc_key_sep

Return type:

str

save(force=False)

Intelligent wrapper to insert or replace a document

A current _updated timestamp is added to all documents.

The first time a document is saved, a _created timestamp is added as well.

If the class defines a non-None object_version, this added as _objver to the document as well.

  1. Documents without an _id are inserted into the database; MongoDB will assign the ObjectId

  2. If force is set, document will be saved regardless of update time or even if it exists. This is useful for upserting documents from another database.

  3. Otherwise, only a database document whose _id and _updated timestamp match this object will be replaced. This protects against overwriting documents that have been updated elsewhere.

Parameters:

force (bool, optional) – if True, upsert the new document regardless of its _updated timestamp

Raises:

MongoObjectsAuthFailed – if authorize_save() has been overridden and does not return a truthy value

classmethod split_id(subdoc_id)

Split a subdocument ID into its component document ID and one or more subdocument keys.

Parameters:

subdoc_id (str) – a full subdocument ID starting with a document ObjectId followed by one or more subdocument proxy keys separated by cls.subdoc_key_sep.

Returns:

A list. The first element of the list is the document ObjectId. The remaining elements in the list are subdocument proxy keys.

Return type:

list

static utcnow()

MongoDB stores milliseconds, not microseconds. Drop microseconds from the standard utcnow() so comparisons can be made with database times.

Returns:

The current time with microseconds set to 0.

Return type:

naive datetime.datetime

Polymorphic Class Reference

Polymorphic proxies are supported by PolymorphicMongoUserDict. All the attributes and methods of MongoUserDict are supported with the following overrides.

class mongo_objects.PolymorphicMongoUserDict(doc={}, readonly=False)

Like MongoUserDict but supports polymorphic document objects within the same collection.

Each subclass needs to define a unique subclass_key

subclass_map = {}

Map subclass_keys to subclasses

Strongly recommended: Override this with an empty dictionary in the base class of your subclass tree to create a separate namespace

subclass_key

Must be unique for each subclass.

One class (usually the base class of the subclass tree) may leave this as None. That subclass will be treated as the default subclass for any documents with a missing or unknown subclass key.

subclass_key_name = '_sckey'

Name of internal key added to each document to record the subclass_key

classmethod find(filter={}, projection=None, readonly=None, object_version=None, **kwargs)

Return matching documents as appropriate subclass instances

Parameters:
  • filter (dict) – Updated with cls.object_version as appropriate and passed to pymongo.find()

  • projection (dict) – Passed to pymongo.find()

  • readonly (None or bool) – as with MongoUserDict.find()

  • object_version – as with MongoUserDict.find()

Returns:

a generator for instances of the user-defined PolymorphicMongoUserDict subclasses, each correct for the document being returned

Raises:

MongoObjectsAuthFailed – if authorize_pre_read() has been overridden and does not return a truthy value

classmethod find_one(filter={}, projection=None, readonly=None, object_version=None, no_match=None, **kwargs)

Return a single matching document as the appropriate subclass or None

Parameters:
Returns:

an instance of the user-defined PolymorphicMongoUserDict subclass correct for this document

Raises:

MongoObjectsAuthFailed – if authorize_pre_read() has been overridden and does not return a truthy value

classmethod get_subclass_by_key(subclass_key)

Look up a subclass in the subclass_map by its subclass_key. If the subclass can’t be located, return the class with subclass_key None. If there is no such class, raise an exception.

Parameters:

subclass_key (str) – subclass key

Returns:

polymorphic document subclass

Raises:

MongoObjectsPolymorphicMismatch – if the subclass key isn’t in the subclass map and no subclass with a None key was registered as the default.

classmethod get_subclass_from_doc(doc)

Return the correct subclass to represent this document.

Parameters:

doc (dict) – document dictionary

Returns:

polymorphic document subclass

Raises:

MongoObjectsPolymorphicMismatch – if the document doesn’t have a subclass key, the subclass key isn’t in the subclass map and no subclass with a None key was registered as the default.

save(**kwargs)

Add the subclass_key to the document and call MongoUserDict.save()

Parameters:

kwargs – passed to MongoUserDict.save()