Introduction

While we were building the Cubed reporting dashboard, we always hit the same issue: how do you store static aggregated data for quick retrival but also have it being dynamic?

We wanted to be able to have users pull data from the DB but see it in different ways. You could achieve this by having a column for every metric and thus a table for every combination but this obviously gets out of hand instantly. If you have a field of "sales" you might want to show that as a sum() or even an avg() how about the min/max() of it?

How we dealt with this was by leveraging Django's Aggregate functions and building our own expressions. An expression would wrap the specific "field" in any math func/operator Django could offer. The next step was to build a generic way of utilising Django's annotate and aggregate functions - which are the foundations for building a sql group by statement.

This led us to building the CubedReportResource class which inherits from Tastypie's base ModelResource but then overrides a handful of functions to achieve what we want. Now any Django model we want to interact with and group by we inherit from CubedReportResource and then tell each field on the model/resource which expression they should use. We do this by using the install method found on the BaseFields class. Because we now know each field and the expression it will be using, we can add extra metrics to interact with them, this is done by the install_additional_metric . This function follows the exact same principals as install but will tell the resource (and thus Django) to "ignore this field" while validating the model and data. Both Tastypie and Django attempt to validate the fields on themselves - checking for types, indexes, and foreign keys.

We found that nearly all our report tables/classes were using the same default fields, and so we created BaseExpressions which would hold them. This class is initalized for every CubedReportResource, and can be added to by using the custom_expressions array. All fields can be overridden. If BaseExpressions has an expression for my_favourite_field and you add a new expression collections to custom_expressions which also has my_favourite_field, then that will be used when Django generates it's query.