Account

Account

The account object is the central tenet of the DCS config. It stores everything from an account's active status to which visitors should be blacklisted.

Data about the account itself (eg, maintenance mode, connection strings) are stored directly on this model. Any data linked to this account but not strictly part of it (eg, associated events, groups, patterns) are stored in dicts full of instances of the relevant information. For example, Account A may have five linked events. These will be stored in account.events, a dict of AccountEvent objects.

Where data is stored in a child model, the Account object itself stores a datetime recording when each dict was last updated. Using the example above, account.last_updated_events would store when the dict in accounts.events was last changed.

AccountList

Accounts are stored in an AccountList. This is an important object in it's own right as it stores all of the methods that are used for updating various parts of the account config. Within the AccountList class is the loop() function; the cornerstone of keeping account data up to date. AccountList extends the Runner class, which provides start() and stop() functions allowing workers to be individually controlled, in theory. In practice however this would rarely (if ever) happen - the ansible script brings up the pydcs service and the loop then runs in perpetuity.

The loop

    self.update_accounts()
    self.update_events()        
    self.update_patterns()
    self.update_clusters()
    self.update_client_cookie_list()
    self.update_path_replacements() 
    self.update_endpoints()
    self.update_domains()
    self.update_visscore_queues()
    self.update_blacklist_ips()
    self.update_blacklist_visitors()
    self.update_account_groups()

Above is the list of functions run once every 10 seconds by the AccountList runner. These all work in the same basic way; they connect to the relevant table in the attrib or attrib_client database, check to see if there are any entries that have updated more recently than the relevant last_updated_* variable, and if so refresh the dictionary of values.

Update accounts

Points at attrib.attrib_account. The meat of the entire process. Gets all accounts created after the last time the AccountList object was updated. Either updates existing Account objects with new data or creates new ones if they don't already exist in memory.

Update events

Points at attrib_client.attrib_event. Adds new events added since the last update.

Update patterns

Points at attrib_client.attrib_pattern. As above but for patterns.

Update clusters

Clusters are groups of events, held in attrib_client.attrib_autoevent_cluster and bound together via attrib_autoevent_cluster_lookup. These match urls to events, where visiting a given url would be equivalent to triggering an event.

Update client cookie list

If we are unable to set our own cookies on a client domain, we attempt to track visitors via information from the client's own cookies. Clients will usually have their own accounts system whereby a user will have a unique identifier we can convert to a vid to allow us to track them.

Update path replacements

Points at attrib_client.attrib_path_replacements. Holds unwanted path tokens and their replacement strings.

Update endpoints

Points at attrib_client.attrib_endpoints.

Update domains

Points at attrib_client.attrib_domains. Contains a list of all domains associated with this client account.

Update visscore queues

Slightly differently, this function deals with updating the name of the SQS queue associated with this account. If the value for visscore_queues has been updated on the Account object, the function checks SQS to see if the queue exists, and if so updates the value. If the queue does not exist, an exception is thrown.

Update blacklist IPs

Points at attrib_client.attrib_blacklist_ip. Updates a list of IP addresses from which hits should be either discarded entirely, or where the visit should be kept but no syncing should be done against it.

Update blacklist visitors

Points at attrib_client.attrib_blacklist_visitors. Updates a list of visitors for whom hits should be discarded or not synced.

Update account groups

Points at attrib.attrib_account_group. Updates a list of child accounts, where the Account being assessed is the parent.