How to get started with BigQuery Editions?

Since the announcement of pricing changes and the introduction of editions in Google Cloud’s BigQuery, companies with extensive workloads have been trying to figure out which edition best fits their use case. Since Google’s documentation still contains the legacy flex slot pricing parallel to the new one, it can be challenging to navigate the new pricing model. 

We have heard reports about data teams spending weeks trying to comb through the documentation and make sense of the new pricing, trying to figure out where and how to start.

In this post, we provide key information about BigQuery Editions to help you navigate the new pricing model.

Your guide to BigQuery editions

Let’s tackle the basics first: setting up a GCP project with Editions, understanding how this affects the performance as well as costs of your workloads, and deciding which projects to move to the new editions-based pricing and which ones to keep on on-demand. We’ll also cover how much reservation is needed and how much commitment should be done.

Here’s what you’ll need to know to understand how edition entities are related:

• First, you’ll need to decide in which region you want to try the new editions
• Select the project or projects you want to do the test in
• Decide which features are currently used or need to be used as a requirement for your BigQuery operation:
  • Do you need 99,9% or 99,99% SLA?
  • Do you need to be HIPAA compliant? Does FedRAMP apply to you?
  • Are you planning to use BigQueryML?
  • And so on. Please check the list of features for each edition.
• Next up, let’s think about reservation for slots:
  • Reservation can be shared among your already selected projects so that they can be maximized. It is recommended to set idle slot sharing.
  • If you don’t make a reservation manually, Google will set one up for you.
  • It is recommended to create a baseline reservation that way, your basic needs are always going to be covered. Basic reservation means that those slots are always going to be available for your workloads, no performance degradation should be expected.
  • Finding the maximum slot allocation for a reservation is far from trivial. It is recommended to set this because the autoscaler scales up the slots in 100-slot steps, so even if you’re just one slot above the limit, you’ll be required to pay for another set of 100 slots.  On the other hand, if you set up the maximum slot allocation to a lower level than your queries’ peak usage at runtime, performance might be degrading due to the lack of available slots.
• Commitments are optional but only available in Enterprise and Enterprise plus. It might be a good idea to commit to the baseline slots that were set up in the reservations.

It could take weeks to collect the necessary data and information that will enable you to make informed decisions about all the above. Rabbit’s BigQuery Edition planner lets you create scenarios instantly, and gives you a calculation for performance and costs right away.