Max Rosenbaum
Software developer
Tuesday, 30 November 2021
Plugin architecture
When I started object orientated programming, one of the things that was not explained so clearly is class inheritance ( and classes in general) can be their own type system.
In school, I knew that we have scalar
types like string
, int
, bool
and float
etc... which is the "type" system. I have also been dragged through the inevitable OOP 101 lesson that tells
you that if you have a base class of Animal
then the next class Dog
and
Cat
should inherit from that, they should have 4 legs and everything will be neat and perfect in your code forever
more.
The standard example is something similar to this;
class Animal:
def __init__(self, legs):
self.legs = legs
@property
def legs(self):
return self._legs
@legs.setter
def legs(self, x):
self._legs = x
class Dog(Animal):
pass
class Cat(Animal):
pass
# Dog has 4 legs
dog = Dog(4)
# Cat has 3 legs :(
cat = Cat(3)
Pretty simple right?
Looking at the above code we know that Cat
and Dog
inherit behaviour and properties from Animal
. The hidden
information
(that often isn't taught) is Cat
and Dog
are a type-of Animal
. This might seem trivial at first since, well,
what is the benefit of thinking about the inheritance relationship that way?
It forces us to think about these classes as "complex types", types that have much more functionality and behaviour than
our scalar types. In the same way that our number types are reasoned about in a hierarchy of Byte
-> Short
-> Int
-> Long
-> Float
-> Double
, we can now reason about our Animal
s like Animal
-> Cat
/Dog
-> Bengal
/Malamute
.
Thinking about inheritance as a type hierarchy allows us to start asking questions like;
- What are all the
Animal
s we have? - How many types of
Animal
s do we have?
Which then feeds into more practical usages like;
- This function returns all the
Animal
s we have - This function returns all the
Animal
s with only one leg
This is why we should favour composition over inheritance. If we create too large of a hierarchy, we create an implicit
type system for things that are completely unrelated from one another. Inheriting from a Cache
base class in Dog
and
Cat
is saying Dog
/Cat
is a type-of cache. That doesn't make any sense and is normally the cause of buggy code that
brings in behaviour you would not expect an Animal
to have. This a violation of
the principle of least astonishment.
So now we know that class inheritance creates a type hierarchy, what interesting things can we do with it?
Plugin architecture
A minimal implementation of a plugin architecture (in python) is to call __subclasses__()
or __init_subclass__()
.
This code looks through all the classes defined in scope and returns them. All the magic in this script happens in
the def animals():
function.
Here is a minimal implementation of plugin architecture:
from abc import ABC
class Animal(ABC):
@staticmethod
def id():
pass
class Dog(Animal):
@staticmethod
def id():
return 'woof'
class Malamute(Animal):
@staticmethod
def id():
return 'Bork!'
class Cat(Animal):
@staticmethod
def id():
return 'meow'
class Bengal(Animal):
@staticmethod
def id():
return 'Roar!'
def animals():
return Animal.__subclasses__()
for plugin in animals():
print(plugin.id())
Check this out in action at online python.
To further the point, much of Drupal's module system is a plugin architecture (blocks as an example) . It uses Symphony's annotation functionality, so the required information is encoded into the annotation, and the functionality is then built into the class. The annotation plugin system is much more feature rich than this example I've provided, it's missing a lot of functionality from a fully realised plugin architecture.
Something that could improve this solution would be a way to dynamically declare classes (like the PHP annotations), and a way to dynamically pass constructor arguments in a quasi dependency injection method.
Here is a slightly more complicated example;
from abc import ABC
class Animal(ABC):
@staticmethod
def id():
pass
@staticmethod
def route():
return '/animal/'
class Dog(Animal):
@staticmethod
def id():
return 'woof'
@staticmethod
def route():
return '/dog/fido'
class Malamute(Animal):
@staticmethod
def id():
return 'Bork!'
@staticmethod
def route():
return '/malamute/rex'
class AnimalManager():
@staticmethod
def get_animals():
""" Gathers all defined `Animal`s """
initialised_animals = []
for animal in Animal.__subclasses__():
initialised_animals.append(animal())
return initialised_animals
@staticmethod
def filter_animal_by_route(animal, path):
if animal.route() == path:
return animal
return []
@staticmethod
def get_by_route(path):
""" This class allows us to get a single `Animal` class via its defined url"""
for animal in AnimalManager.get_animals():
if AnimalManager.filter_animal_by_route(animal, path):
return animal
return []
print(AnimalManager.get_by_route('/malamute/rex').id())
Check this out in action at online python.
This pattern is very handy and easy to implement when you're in a pinch working with legacy code that has a lot of repeating, but slightly different sections.
- You can abstract the functional parts into easily unit testable chunks in classes.
- Force lots of code into classes making it unit testable
- The code you are working with is not performance critical
- The project has a reasonably defined scope
Gotchas
The downside to this pattern is performance. There comes a threshold where you need to be careful about how you are instantiating and gathering classes. In the past, I've unwittingly had a database query run with the class instantiation. The meant that every time a single graph was loaded, it would load fire off about 150 database queries. 149 of which were not used.
I fixed this by lazy loading some information about the object at request time solving the problem. However, the process
of loading in ~150 classes every time is a reasonably heavy operation. If you are loading 1000s of classes at once, this
could have a considerable overhead to do some seemingly simple things. So be careful about the scope of these plugins
and how they are being loaded in. Caching the results (serialization) from functions that do similar work
to def animals():
in this pattern can only get you so far and can cause some very undesirable behaviour.
Final thoughts
Plugin architecture is very flexible and allows you to abstract away a lot of complexity into very well-defined and easy to reason about compartments.