Google Integrates Spectacles into Looker for Continuous Integration & Regression Testing

When you’re developing LookML views and models in Looker, it’s possible that changes you’ve made could end-up breaking downstream looks and dashboards that rely on that LookML content; you might have deleted or renamed a dimension or measure that’s used in one of your dashboards or you may have a syntax error in your SQL or LookML that prevents a look from returning results.

What’s worse, these issues often remain undetected until they end-up impacting your end users, and not-only do they take-up your time in finding and fixing what’s caused the issue, but they undermine the users’ confidence in your system.

In the world of software development these are called “regressions”, and regression testing is the process of ensures that previously validated functionality continues to operate as expected after new changes. This type of testing is distinct from functional testing, performance testing and user acceptance testing and in the past was performed in a Looker context either by creating hacky reports or explores that you ran after each release to see if they broke, or you used the open-source Spectacles tool or the hosted and fully-managed version of it that we covered on this blog back in 2021.

Google have now acquired Spectacles and have integrated its functionality into a new Looker feature called Continuous Integration, with Spectacles’ developers Dylan Baker and Josh Temple now joining the Looker product development team.

In this blog co-authored with my colleague Bailey Sharp-Ledger, we’ll explain what Looker Continuous Integration is, how it works and how it compares to the standalone Spectacles tool and managed service.

Note: As this feature is currently in Public Preview, it’s subject to change and has limited support.

What Exactly is Looker CI?

At its core, Looker Continuous Integration is about automating the validation of your LookML projects. Think of it as an automated quality assurance (QA) team for your LookML, constantly checking for issues before they affect your users.

CI allows you to run a series of comprehensive tests on your LookML code, catching critical errors and inconsistencies across various aspects of your Looker instance. The magic lies in its suite of specialised validators:

• SQL Validator: No more broken fields due to underlying database changes or mistyped SQL. It runs actual queries against your database, verifying that the dimensions in your Explores exist and execute correctly. You can also configure it to only check specific Explores or exclude certain fields. To avoid consuming resources, queries use LIMIT 0 and WHERE 1=2.
• Assert Validator: For those of you leveraging LookML data tests (and you should be!), this validator automatically runs them and flags any failures. This ensures your data transformations & assumptions remain valid.
• Content Validator: Ever had a Look or dashboard break because an underlying LookML field was renamed or removed? The Content Validator checks your existing Looks and dashboards for errors, ensuring that all published content remains functional.
• LookML Validator: This runs the standard LookML Validator that you’re familiar with from the IDE, checking for syntax errors, referencing issues, & other structural problems within your LookML code itself.

These validators are grouped into CI Suites, which are configurable sets of tests associated with your LookML project. You can define what it validates, how it validates (with specific options), and most importantly, when it validates.

Setting Up Your First CI Suite

Before you can build CI Suites, a Looker Administrator needs to perform a few one-time setup tasks

1. Admin Prerequisites (Initial Setup)

1. Check compatibility: Ensure your Looker instance is compatible (e.g., Looker-hosted, Public IP for Google Cloud core instances).
2. Enable Continuous Integration: From the Admin panel > Continuous Integration, underneath the toggle is a warning: “Don’t enable Continuous Integration on a Looker instance that requires your data to reside in one specific location. Continuous Integration currently stores certain data in the United States.”
3. Enter CI configuration details: With the enable toggled, an iFrame will appear on the same page. Create a dedicated Continuous Integration user; if this does not appear, check the format of your API Host URL under Admin > Platform > API. For example if the url of this page is “https://rittman.eu.looker.com/admin/api”, the API Host URL should be everything before the /admin.
4. Configure Github integration: Optional, but required for Pull Request trigger: install the CI GitHub app in your GitHub organisation.

2. Creating Your First CI Suite

Once the admin setup is done, Developers with manage_ci permission are able to create a CI suite directly within the Looker IDE.

1. Navigate to the CI Section: In the Looker IDE, under Develop > your project > click the new Continuous Integration icon

2. Go to Suites: Select the Suites tab. Here, you’ll see any existing CI suites, or you can create a new one.

3. Create a New Suite: Click the Create suite button.

4. Name & Configure:

• Suite name: Give your suite a descriptive name (e.g., “Full Project Validation,” “Feature Branch PR Checks”).
• Trigger on Pull Requests: This is the essential step for an automated workflow! Toggle “Trigger on pull requests from Looker” to enable automatic CI runs whenever a developer submits a PR to your repository. You can optionally limit this to specific target branches (e.g., only main and develop).
• Enable Validators: Now, select which validators you want to include in this suite. We highly recommend starting by enabling all four: SQL, Assert, Content, and LookML. You can also configure specific options for each validator (e.g., which Explores the SQL validator should test).

5. Save Your Suite: Click Create suite (or Update suite if editing).

3. Running Your CI Suite:
Your suite will now automatically run on pull requests if configured. However, you can also trigger a run manually:

1. From the Suites page, click the Run suite button next to your configured suite.
2. You’ll be prompted to choose a Workspace: Production (to validate your deployed project) or Dev Mode (to test a specific development branch), giving the ability to test changes before they even get to a PR.

Comparing Looker CI to Spectacles

The big advantage to Looker’s new CI feature compared to the old standalone Spectacles managed service is that it now becomes available to all Looker customers, at no extra cost and built directly into the Looker developer experience.

At the time of writing there are a couple of features that were in Spectacles, the managed service, that aren’t in Looker Continuous Integration (but this could change before the feature goes GA) and there are some differences in features that come from the integration into the Looker product.

• Spectacles did support integrating with dbt Cloud, so that dbt pull-request test pipelines in Github could then run a Spectacles CI Suite after the dbt build test completed in-order to check that any changes dbt introduced didn’t then break your downstream Looker explores, looks and other content; this feature isn’t in the current release of Looker Continuous Integration but may well be in the GA release later on
• Spectacles integrated with GitHub, GitLab, and Azure DevOps to test LookML pull requests automatically and had an API that you could use to build a custom workflow in the CI platform of your choice; Looker has now introduced dedicated CI endpoints in its Looker API that can be used to programmatically start an existing CI Suite run (create_ci_run), and check its results (get_ci_run). This makes it possible to integrate CI runs directly with orchestration tools such as Apache Airflow, dbt Cloud, or any scheduling system, and embed Looker CI in wider data workflows.
• Unlike Spectacles, Looker CI doesn’t include style validators for enforcing LookML coding standards such as consistent naming conventions, proper documentation, or standardised field descriptions; the open-source Look At Me Sideways (LAMS) is however available though this requires additional setup and isn’t integrated into the CI workflow.
• Notification options are limited compared to third-party tools that can integrate with Slack, email, or other communication platforms with customisable messaging; you can however leverage your git platform’s notification system for PR-based CI, or custom build a notification workflow using the Looker API.

INTERESTED? FIND OUT MORE!

Rittman Analytics is a boutique data analytics consultancy that helps ambitious, digital-native businesses scale-up their approach to data, analytics and generative AI.

We’re authorised delivery partners for Google Cloud along with Oracle, Segment, Cube, Dagster, Preset, dbt Labs and Fivetran and are experts at helping you design an analytics solution that’s right for your organisation’s needs, use-cases and budget and working with you and your data team to successfully implement it.

If you’re looking for some help and assistance with your Looker initiative or would just like to talk shop and share ideas and thoughts on what’s going on in your organisation and the wider data analytics world, contact us now to organise a 100%-free, no-obligation call — we’d love to hear from you!