In our last release notes we boasted that our recommendations for Compute Engine instance sizing are superior to Google’s native VM Machine Type Recommender’s rightsizing recommendations. At the time of writing that we were kind of giddy about our results, so it was just words based on some sum totals, but now we have some data, and decided to share it with you!

Little Big Picture

Okay, here’s an overview for you: a selected customer’s May 2024 spending on standalone instances and Managed Instance Groups, and what Rabbit and the Google Recommender engine would save them*:

As you can see, Rabbit’s recommendations far outperform the Google Recommender when it comes to dollars saved.

A Bigger Spreadsheet

But let’s drill down a bit, shall we? Let me present The Big Spreadsheet. It displays information about the standalone instances and Managed Instance Groups of said customer for which either Rabbit or the Google Recommender has a sizing recommendation for. We’ve annotated the table header, but let me offer a summary:

• The first four columns show the current setup of this virtual machine: its type, current number of virtual cores and mebibytes of RAM it has, and the cost it incurs every hour (without any discount factored in).
• Days_run is the number of days the instance has run during the examined period: notice that this is a fractional number, as you’re only charged for CPU and RAM when the machine is running. Cost is the (net) price the customers would pay for this instance in the examined period. Rabbit’s pricing calculation reflects reality closely, based not only on the machine type and the region the machine is located in, but also any global, flat discounts offered to the customer.
• CPU and memory utilization are the ratios of nominal and actually used resources, while “sufficient” CPU and memory are resources Rabbit believes necessary to run the workloads of these machines. We’ll get a bit deeper into this in the next section!
• The next four columns contain Rabbit’s proposed machine type for this machine and its cost, and the four columns after that contain info about the Google Recommender recommendation.
• The last two columns show you just how much money Rabbit or the Google Recommender would save off the cost of this instance.

So yeah, what’s more immediately obvious is just how few rightsizing recommendations we have: 31 vs Rabbit’s 84 recommendations that result in a saving, for these 50 machines. Rabbit saves you $8,249, while the Google Recommender saves you $2,413. Quite a difference, isn’t it?

$8,249 is quite a lot too. One could buy, uh, about 5 tons of carrots from that. Just saying.

How Rabbit recommends instance size

When you’re running a virtual machine, you’re reserving resources (CPU and memory) for use, and you’re paying for these whether you actually utilize them or not; whether they sit idly or do stuff for you. Rabbit is monitoring this utilization through the Cloud Monitoring API: sampled every minute, Rabbit then applies a fairly safe algorithm by default:

it selects the highest rate of utilization in the past month, then doubles it for safety, and says that must be enough for your workload even if your service goes viral

or something. This might sound kinda naive, but that’s how actual engineers operate: take a number, then double it for safety. But you can also customize this algorithm by selecting the aggregation function (you can use the mean utilization instead of maximum), the size of the safety buffer, even the window length.

Rabbit then selects the machine type that satisfies these requirements, and calculates the cost for it. If it’s lower than the price you currently pay, bam, you get a recommendation.

Let’s look at examples

This is going to be somewhat long. TL/DR: the Google Recommender recommendations are overly cautious, but make strange assumptions about what may be acceptable for your workloads.

The Google VM Machine Type Recommender are the little recommendations that pop up in the console from time to time. We have no idea how this thing works, probably similar to Rabbit. Actually, it’s a lot more limited than Rabbit: while Rabbit looks at the past 30 days for data (and makes it customizable!), Google Recommender looks only at the past 8 days worth of data, which is not enough to capture any meaningful seasonality (think weekend vs weekdays in some use-cases).

But enough of talking trash about Google Recommender, we’ll get back to that soon enough, let’s compare specific cases!

There’s the obvious couple of wins for Rabbit, the first three rows of the table, e2-highcpu-32 instances. Their CPUs are underutilized, hovering around 10%, obviously those 32 vCores are not really needed, yet only Rabbit provides recommendations: giving a e2-custom-6-32768 for the one with ~8.9% utilization, and e2-standard-8 for the other two. Custom machine types are more expensive compared to a preset machine of equal resources, but it can be worth it if you can save 2 CPUs. Note how the custom machine saves about 30 bucks a month more than the slightly larger preset ones: CPU is expensive!

Then there are rows 5-7: e2-highcpu-32 instances again, with a CPU utilization of around 1%. Talk about overprovisioning! Google Recommender offers e2-custom-12-32768 instances, while Rabbit suggests e2-highmem-4 preset machines, saving ~$40 more! Google Recommender seems really very cautious: even a single core would be enough to satisfy that level of utilization!

Why does Rabbit still offer a machine with 4 cores? Well, that’s because of memory, and this is related to a limitation we have: by default, if you’re just running a VM, the metrics about memory utilization aren’t available in Google’s Monitoring API (where Rabbit draws the data from), you need to install Google’s own Ops Agent for that (literally a single click on the Cloud Console). In these cases, Rabbit assumes the memory is fully utilized, and offers only machines capable of holding that much RAM.

This leads us to a surprising case where the Google Recommender outperforms Rabbit’s: look at row 13, a severely underutilized n1-standard-8 machine. Rabbit doesn’t get memory metrics, so it does its best and offers n2d-highmem-4 that has all that 32 gigs or RAM, and… what’s this? Google Recommender suggests a machine with 4 CPUs and only 10 GB RAM. We don’t really know what to make of this. One of our expert claims the Recommender must be getting some internal information from the hypervisor, anything else would be irresponsible; another of our experts says that’s a conspiracy theory, the hypervisor doesn’t know about the VM’s memory utilization, and the Recommender assumes if you have a CPU utilization of less than 1%, then you most likely won’t need THAT much of a RAM. Neither case looks very good on Google: either they make kinda-sound-but-potentially-unsafe-suggestions, or GCP withholds information from the customer. Either way, we here at Rabbit would rather our customer just installed Ops Agent.

There’s another case which we don’t really consider safe: row 52 is an e2-standard-2 machine, for which Rabbit doesn’t have a better idea, while the Google Recommender suggests e2-custom-medium-5632. What’s the difference? Well, the E2 medium is a shared core machine type: you nominally get one virtual CPU, but your workload may have access to less or more computing resources, depending on availability. It may be suitable for your workload, or it may not. We wouldn’t want to push this decision on our customers.

Finally, let’s talk about a case where Rabbit does something the native system doesn’t seem to: offering multiple different machine series options! Row 10 has an n1-standard-8: the native system offers a custom N1 machine somewhat better fitting its load, while Rabbit makes the leap** and offers an N2D machine! N2D machines feature AMD CPUs, and they are considerably cheaper than N1 or N2 machines equipped with Intel processors, while being even somewhat more performant.

Still with us?

If you got to this point, congratulations! Start using Rabbit now and the sizing recommendations for your own VMs, and share your results with us!

*  Note that the costs and savings may be in different currencies for different customers.

** Did you know rabbits can easily leap a meter high up in the air?