Skip to content

Google Integration

There are three google services used as a part of the google integration, which are Google Ads, Google Search Console and Youtube. The services are added from https://console.cloud.google.com by selecting the products and specifying the scopes.

Google Cloud Console

To make any changes here a developer account is needed. After logging in to google cloud console, API & Services product should be selected. It might be accessible from the home page, or left navigation menu. There is also a dropdown to select a project whose product is accessed. The project responsible for Cubed integration is called API Project.

Changes in Links or Documentation contents.

API docs content and links are subject to constant change. So, this documentation should be read with a bit of caution. The process that was working at the time of writing this document is mentioned here and if anything has broken and require changes that are not present in this document, it is advised to check the official documentation.

At the time of writing, the documentation are as follows:

OAuth2 Flow with Google Integration:

OAuth2 flow is achieved with the help of frontend and backend. The backend provides the API to display data for the frontend but it also acts as a proxy between frontend and API Provider, in this case, google. Just looking at the frontend, the flow would look as described below:

Google Integration Frontend Flow

In account configuration Unified Google integration section is selected, create is clicked, services are selected, save and continue is clicked. It would take to sign in page in google, where Google Account should be logged in, the account should be set up with manager account for ads, it should have sites to use search console api, and it should have youtube videos to use youtube data API. After login it's redirected to google integration modal. Behind the scene backend does couple of processing of data as well as communicate with the google server. The process is described below:

Google Integration Frontend, Backend, Google Server Flow

The frontend makes two types of request to backend. If the request is data request from API, then the data is fetched from database and sent to frontend. It can also create a new API connection, for which a custom django view will be called that redirects to google login page. The view will have the information required by google server such as APP ID, Redirect URI. More information can be found here as well in the backend code of file unified_views.py.

After logging in, google server again redirects to the specified redirect uri that is caught by backend server as there is another view defined to catch the redirect URI. Google sends auth code which is used to get access token and refresh token by sending auth code as a post request to the API. Access token and refresh token provide us access to user data via API so they are saved to the api_google_oauth_connection table. Finally, backend also redirects to the frontend modal for google integration.

Testing in local environment:

To test the flow in local environment a google user email that has google ads running, google console set up with site urls and youtube video running, is required.

Service wise description:

There are connection detail that are common for all the services. They are user email, access tokens, cubed_user_id and so on. They are stored in api_google_oauth_connection. The list of services connected to the connection is stored in api_google_connection_service. If the services get error while pulling data, we store the failed connection and the specify which service ran into error in api_google_connection_deactivated table.

In frontend, on load the modal fetches the connection from api_google_oauth_connection where active=1 and deleted=0. The api_google_oauth_connection resource dehydrates and adds details such as service connection, service status in the response.

If any service is deactivated, there will be a warning message in the modal asking the account user to re-create the connection which is shown in the figure below.

Google Integration Frontend Disabled Connection

The table displays the connection name, user email of the cubed user that made the connection, and the status of each services.

There are accordions that can be opened for each connected services by selecting the connection and clicking the button to edit. If search console is connected Edit Site URL is clickable, similarly Edit Ad Accounts, Edit Youtube Connection are clickable when adwords and youtube are connected respectively.

Google Ads API has the concept of Manager accounts. As soon as OAuth is done all the manager accounts are pulled and saved to api_adwords_connection_detail. If user's google account does not have manager account then Google Ads API will not work. Which google have mentioned here in the prerequisite section.

To provide user full control of data, the sub accounts/customer accounts for which the ads api data is pulled, is not pulled during connection set up. But user can re-import them from the Edit Ads Account accordion. The customer id are stored in api_adwords_connection_account if it is not mcc account and in api_adwords_connection_mcc_account if it is a mcc account. The api_adwords_connection_account and api_adwords_connection_mcc_account are mapped in api_adwords_connection_account_lookup.

The import will save to these three table and while deleting or disabling, records from all these tables are deleted too.

Import Popup is shown below:

Google Integration Import Adwords

Adwords accordion is shown below where customer account can be disabled and Cubed wont pull any data for them.

Google Integration Import Adwords

Commands

update_adwords.py is the file that runs command for adwords. It looks at the connection in api_adwords_connection_detail table and pulls data using list of other commands prefixed with pull_adwords_*. These commands inherit GoogleAdsCommandWrapper class, specify in dictionary where to store data, the format of data from the API and query that is expected by the API. The returned data for specified query is saved to specified tables.

Google Search Console API

Similar to adwords, search console also has few prerequisite, mentioned here. After OAuth is setup with search console enabled, to pull data, site urls must be imported which is stored in api_googlesearch_site. So, similar to adwords site urls can be imported for search console API from the accordion.

Commands

There are two commands that pull data, which are update_seogd_googlesearch_console and update_seogd_googlesearch_console_index_status.py. First pulls the search results and performance for paths, associated with the site url and the second one pulls index status.

Youtube Data API

Youtube does not require any extra information to pull data, however to give user extra control, to enable or disable data pull like other services, an internal table is present api_youtube_connection_detail, which will be inactive when OAuth is done initially. Later, user can enable it and again disable it from the accordion for the youtube connected section.

Command

There is only one command that runs to pull youtube data which is update_youtube_api_data. It looks at channel details, video detail, channel analytics and video analytics.