In order to have a declarative way of initializing our database, we need to make all models inherit from a Base class. This class will contain the model’s metadata such as the field mappings, indexes, collection name, and other information we might need. Also since pymongo treats database documents as dictionaries, we can make the Base class to extend dict and this allows all models to also be dict instances.

class Base(dict, metaclass=BaseMetaClass):
    def __init__(self, *args, **kwargs):
        super(Base, self).__init__(*args, **kwargs)

    def __getattr__(self, key):
        try:
            return self[key]
        except KeyError:
            raise AttributeError(
                "{} has no attribute {}".format(
                    self.__class__.__name__,
                    key,
                )
            )

    def __setattr__(self, key, value):
        self[key] = value

    def __str__(self):
        return dict.__str__(self)

    def __repr__(self):
        return dict.__repr__(self)

So far it’s a pretty simple class. It inherits from dict and calls the super’s constructor method to initialize just as if it was a dictionary. The key concept of this class is the metaclass argument, with a BaseMetaClass class.

In order to register mappings and overall database metadata we need to use a metaclass. A metaclass is like a class that creates other classes. So when the Base class is declared (not instanced) it uses a metaclass class. The metaclass gets instanced and runs a __new__ method to create the class using this metaclass.

Let’s take a look on the metaclass code:

class BaseMetaClass(type):
    """Metaclass for Base model class that creates mappings."""

    metadata = Metadata() # this class will contain overall database metadata

    def __new__(cls, name, bases, attrs):
        """Create class with mappings and other metadata."""
        if name == "Base":
            return type.__new__(cls, name, bases, attrs)
        fields = {}
        class_dict = {
            "__fields__": {},
            "__tablename__": name
        }
        for k, v in attrs.items():
            if k == "__tablename__":
                class_dict["__tablename__"] = v
            if isinstance(v, Field):
                class_dict["__fields__"][v.name] = v

        cls.metadata.collections[class_dict["__tablename__"]] = {
            "fields": class_dict["__fields__"],
        }
        return type.__new__(cls, name, bases, class_dict)

The __new__ method returns a new class declaration. We are using type as the parent class and returning its __new__ method but customizing the attributes of the new created class with the class_dict dictionary. We are adding some model metadata to this class_dict, stuff like __tablename__, __fields__, and as we will add later, __indexes__ metadata can be added as well.

Fields mapping is very simple with PyMongo. We do not need to create a mapping between a engine specific syntax and the python types. We will define the Field class later, but so far you only need to understand that the actual mapping is just taking a name defined by the user and using that name as the key or attribute name inside the database document.

The last part of the method adds all the model’s information to a single Metadata class that contains the full database metadata and also methods for initializationd and deletion.

The Metadata class contains metadata of our database, such as database name, the list of collections and their metadata, etc. Database initialization and deletion is also performed through this class if we provide a valid MongoClient.

class Metadata:
    database = "DEFAULT_DATABASE"
    collections = {}

    def create_all(self, client: MongoClient, database: str = None):
        self.database = database or self.database
        db = client[self.database]
        for c in self.collections:
            collection = db[c]

    def drop_all(self, client: MongoClient, database: str = None):
        self.database = database or self.database
        client.drop_database(self.database)

The class’s code is pretty straight forward. It has a dictionary with all collections and their metadata, a create_all and drop_all methods to initialize and delete the database. The thing is that with PyMongo, collections are lazy loaded. This means that a collection object is not really persisted to the database until a document is inserted or an index is created. This means that so far, our code does nothing on the database. But what we want is to create some indexes declared on our models. The following sections cover the models definition and modify the current code to allow proper database initialization.

We are going to create only two simple field classes. In a regular ORM you would have to create fields for different types of database types, such as integer, float, string, spatial coordinates, etc… For this use case we don’t really need field type definition since we can actually store anything in a mongo document. So our field class can only map a custom name for the document:

class Field:
    def __init__(self, name):
        self.name = name

    def __repr__(self):
        return f"Field<name={self.name}>"

    def __str__(self):
        return f"Field<name={self.name}>"


class SpecialField(Field):
    def __init__(self, name, callback):
        super(SpecialField, self).__init__(name)
        self.callback = callback

The SpecialField class can be used to perform operations on the actual model attributes. For example, we can store a field using other field’s value. To do that the user would have to code a callback function that takes a dictionary with the model fields and return the processed value that will be added as a new field.

from pymongo import IndexModel, TEXT
class MyModel(Base):
    def my_function(**kwargs):
        return str(kwargs["my_field"]).upper()

    my_field = Field("my_field")
    my_special_field = SpecialField("special", my_function)

    __table_args__ = [IndexModel([("my_field", TEXT)])]

We previously defined a simple Field class and a SpecialField class that takes a function and receives the model arguments to perform some operation and create a new field based on that. In this case we defined a function which converts the my_field field to upper case and store it in my_special_field.

We are also defining a __table_args__ attribute to define other metadata supported by the BaseMetaClass.

IndexModel is a way of defining indexes provided by PyMongo