Autoscaling Cloud Spanner  |  Google Cloud (2023)

When you create a Cloud Spanner instance,you choose the number of compute capacity nodes or processing units to serveyour data. However, if the workload of an instance changes, Cloud Spannerdoesn't automatically adjust the size of the instance. This document introducesthe Autoscaler tool for Cloud Spanner(Autoscaler), an open source tool that you can use as a companion tool to Cloud Spanner.This tool lets you automatically increase or reduce the number of nodes orprocessing units in one or more Spanner instances based on howtheir capacity is being used.

This document is part of a series:

• Autoscaling Cloud Spanner (this document)
• Deploy a per-project or centralized Autoscaler tool for Cloud Spanner
• Deploy a distributed Autoscaler tool for Cloud Spanner

This document presents the features, architecture, configuration, anddeployment topologies of the Autoscaler. The documents that continue this seriesguide you through the deployment of Autoscaler in each of the differenttopologies.

This series is intended for IT, Operations, and Site Reliability Engineeringteams looking to reduce operational overheads and optimize the cost of theirCloud Spanner deployments. This series is also intended for people who haveworkloads with the following conditions:

• Periodic swings in user demand.
• Predicted to need increasing amounts of compute resources or storageover time.

Not all Cloud Spanner performance issues can be resolved by adding morenodes or processing units. Autoscaler can't solve problems that occur that areunrelated to the instance size such as lock contention and hot spotting.

Autoscaler

Autoscaler is useful for managing the utilization and performance of yourSpanner deployments. To help you to balance cost control withperformance needs, Autoscaler monitors your instances and automatically adds orremoves nodes or processing units to help ensure that they stay within the following parameters:

• The recommended maximums for CPU utilization.
• The recommended limit for storage per node,plus or minus a configurable margin.

Autoscaling Cloud Spanner deployments enables your infrastructure toautomatically adapt and scale to meet load requirements with little to nointervention. Autoscaling also right-sizes the provisioned infrastructure, whichcan help you to reduce costs.

Architecture

This section describes the components of Autoscaler and their respectivepurposes in more detail.

The Autoscaler architecture consists of Cloud Scheduler,two Pub/Sub topics, two Cloud Functions,and Firestore.The Cloud Monitoring API is used to obtain CPU utilization and storage metrics for Spannerinstances.

Cloud Scheduler

Using Cloud Scheduler,you define how often Autoscaler verifies your Spannerinstances scaling metrics thresholds. A Cloud Scheduler job can checksingle or multiple instances at the same time. You can define as many jobschedules as you require.

Poller Cloud Function

The Poller Cloud Function is responsible for collecting and processing the time-series metrics for one ormore Cloud Spanner instances. The Poller preprocesses the metrics data foreach Cloud Spanner instance so that only the most relevant data points areevaluated and sent to the Scaler Cloud Function. The preprocessingdone by the Poller Cloud Function also simplifies the process ofevaluating thresholds for regional and multi-regional Cloud Spannerinstances.

Scaler Cloud Function

The Scaler Cloud Function evaluates the data points received from the PollerCloud Function and determines whether you need to adjust the number ofnodes or processing units and if so, by how much. The Cloud Functioncompares the metricvalues to the threshold, plus or minus an allowed margin,and adjusts the number of nodes or processing units based on the configuredscaling method. For more details on scaling methods, see Autoscaler features.

Operational flow

This section details the operational model of Autoscaler, as shown in thefollowing architectural diagram.

1. You define the schedule, time, and frequency of your autoscaling jobsin Cloud Scheduler.
2. On the schedule that you define, Cloud Scheduler pushes amessage containing a JSON payload with the Autoscaler configurationparameters for one or more Spanner instances into thePolling Pub/Sub topic.
3. When the message is published into the Polling topic, an instance of thePoller Cloud Function is created to handle the message.
4. The Poller Cloud Function reads the message payload andqueries the Cloud Monitoring API to retrieve the utilization metrics for eachSpanner instance.
5. For each Spanner instance enumerated in the message, thePoller function pushes one message into the Scaling Pub/Subtopic, containing the metrics and configuration parameters to assess forthe specific Spanner instance.
6. For each message pushed into the Scaler topic, the Scaler Cloud Functiondoes the following:
   1. Compares the Spanner instance metrics againstthe configured thresholds, plus or minus a configurablemargin.You can configure the margin yourself, or use the default value.
   2. Determines whether the instance should be scaled.
   3. Calculates the number of nodes or processing units that the instanceshould be scaled to based on the chosen scaling method.
7. The Scaler Cloud Function retrieves the time when the instance was last scaledfrom Firestore and compares it with the current time, todetermine if scaling up or down is allowed based on the cooldown periods.
8. If the configured cooldown period has passed, the Scaler Cloud Function sendsa request to the Spanner Instance to scale up or down.

Throughout the flow, the Autoscaler writes a summary of itsrecommendations and actions to Cloud Logging for tracking and auditing.

Regardless of the deployment topology that you choose, the overall operation of Autoscaler remains the same.

Autoscaler features

This section describes the main features of Autoscaler.

Manage multiple instances

Autoscaler is able to manage multiple Cloud Spanner instances acrossmultiple projects. Multi-regional and regional instances also have differentutilization thresholds that are used when scaling. For example, multi-regionaldeployments are scaled at 45% high-priority CPU utilization, whereas regionaldeployments are scaled at 65% high-priority CPU utilization, both plus or minusan allowed margin.For more information on the different thresholds for scaling, see Alerts for high CPU utilization.

Independent configuration parameters

Each autoscaled Cloud Spanner instance can have one or more pollingschedules. Each polling schedule has its own set of configuration parameters.

These parameters determine the following factors:

• The minimum and maximum number of nodes or processing units that controlhow small or large your instance can be, helping you to control costs.
• The scaling method used to adjust your Cloud Spanner instance specific to your workload.
• The cooldown periods to let Cloud Spanner manage data splits.

Different scaling methods for different workloads

Autoscaler provides three different scaling methods for up and down scalingyour Cloud Spanner instances: stepwise, linear, and direct. Each method isdesigned to support different types of workloads. You can apply one or more methodsto each Cloud Spanner instance being autoscaled when you createindependent polling schedules.

Stepwise

Stepwise scaling is useful for workloads that have small or multiplepeaks. It provisions capacity to smooth them all out with a single autoscalingevent.

The following chart shows a load pattern with multiple load plateaus or steps,where each step has multiple small peaks. This pattern is well suited for thestepwise method.

When the load threshold is crossed, this method provisions and removes nodesor processing units using a fixed but configurable number.For example, three nodes areadded or removed for each scaling action. By changing the configuration, you canallow for larger increments of capacity to be added or removed at any time.

Linear

Linear scaling is best used with load patterns that change more gradually orhave a few large peaks. The method calculates the minimum number of nodes orprocessing units required to keep utilization below the scaling threshold. The number of nodes or processing unitsadded or removed in each scaling event is not limited to a fixed step amount.

The sample load pattern in the following chart shows larger sudden increases anddecreases in load. These fluctuations are not grouped in discernible steps asthey are in the previous chart. This pattern is more easily handled using linearscaling.

Autoscaler uses the ratio of the currently observed utilization over theutilization threshold to calculate whether to add or subtract nodes orprocessing units from the current total number.

The formula to calculate the new number of nodes or processing units is as follows:

newSize = currentSize * currentUtilization / utilizationThreshold

Direct

Direct scaling provides an immediate increase in capacity. This method isintended to support batch workloads where a predetermined higher node count isperiodically required on a schedule with a known start time. This method scalesthe instance up to the maximum number of nodes or processing units specified in the schedule, and is intended to be used in addition to a linear or stepwisemethod.

The following chart depicts the large planned increase in load, which Autoscalerpre-provisioned capacity for using the direct method.

Once the batch workload has completed and utilization returns to normal levels,depending on your configuration, either linear or stepwise scaling is applied toscale the instance down automatically.

Deployment methods

Autoscaler can be deployed either in an individual project or alongside theCloud Spanner instances it manages. Autoscaler is designed to allow forflexibility and it can accommodate the existing separation of responsibilitiesbetween your operation and application teams. The responsibility to configurethe autoscaling of Spanner instances can be centralized with asingle operations team, or it can be distributed to the teams closer to theapplications served by those Spanner instances.

The different deployment models are discussed in more detail in Deployment topologies.

Serverless for ease of deployment and management

Autoscaler is built using only serverless and low management Google Cloudtools, such as Cloud Functions, Pub/Sub, Cloud Scheduler,and Firestore. This approach minimizes the cost and operationaloverhead of running Autoscaler.

By using built-in Google Cloud tools, Autoscaler can take full advantageof IAM (IAM) for authentication and authorization.

Configuration

Autoscaler has different configuration options that you can use to manage thescaling of your Cloud Spanner deployments. The next sections describe thebase configuration options and more advanced configuration options.

Base configuration

Autoscaler manages Cloud Spanner instances through the configuration definedin Cloud Scheduler. If multiple Cloud Spanner instances need to bepolled with the same interval, we recommend that you configure them in the sameCloud Scheduler job. The configuration of each instance is representedas a JSON object. The following is an example of a configuration where twoCloud Spanner instances are managed with one Cloud Scheduler job:

[ { "projectId": "my-spanner-project", "instanceId": "spanner1", "scalerPubSubTopic": "projects/my-spanner-project/topics/spanner-scaling", "units": "NODES", "minSize": 1, "maxSize": 3 },{ "projectId": "different-project", "instanceId": "another-spanner1", "scalerPubSubTopic": "projects/my-spanner-project/topics/spanner-scaling", "units": "PROCESSING_UNITS", "minSize": 500, "maxSize": 3000, "scalingMethod": "DIRECT" }]

Cloud Spanner instances can have multiple configurations on differentCloud Scheduler jobs. For example, an instance can have one Autoscalerconfiguration with the linear method for normal operations, but also haveanother Autoscaler configuration with the direct method for planned batchworkloads.

When the Cloud Scheduler job runs, it sends a Pub/Submessage to the Polling Pub/Sub topic. The payload of this messageis the JSON array of the configuration objects for all the instances configuredin the same job. See the complete list of configuration options in thePoller README file.

Advanced configuration

Autoscaler has advanced configuration options that let you more finely controlwhen and how your Cloud Spanner instances are managed. The following sectionsintroduce a selection of these controls.

Custom thresholds

Autoscaler determines the number of nodes or processing units to be added or subtracted to an instance using the recommended Spanner thresholds for the following load metrics:

• High priority CPU
• 24-hour rolling average CPU
• Storage utilization

We recommend that you use the default thresholds as described inCreating alerts for Cloud Spanner metrics. However, in some cases you might want to modify the thresholds used by Autoscaler.For example, you could use lower thresholds to make Autoscaler react morequickly than for higher thresholds. This modification helps to preventalerts being triggered at higher thresholds.

Custom metrics

While the default metrics in Autoscaler address most performance and scalingscenarios, there are some instances when you might need to specify your ownmetrics used for determining when to scale in and out. For these scenarios, youdefine custom metrics in the configuration using the metrics property.

Margins

A margin defines an upper and a lower limit around the threshold. Autoscaleronly triggers an autoscaling event if the value of the metric is more thanthe upper limit or less than the lower limit.

The objective of this parameter is to avoid autoscaling events being triggeredfor small workload fluctuations around the threshold, reducing the amount offluctuation in Autoscaler actions. The threshold and margin together define thefollowing range, according to what you want the metric value to be:

[threshold - margin, threshold + margin]

.The smaller the margin, the narrower the range, resulting in a higher probabilitythat an autoscaling event is triggered.

Specifying a margin parameter for a metric is optional, and it defaults to fivepercentage points both preceding and below the parameter.

Deployment topologies

To deploy Autoscaler, decide which of the following topologies is best tofulfill your technical and operational needs:

• Per-project topology: The Autoscaler infrastructure is deployedin the same project as Cloud Spanner that needs to be autoscaled.
• Centralized topology: Autoscaler is deployed in one project andmanages one or more Cloud Spanner instances in different projects.
• Distributed topology:: Most of the Autoscaler infrastructure isdeployed in one project but some infrastructure components are deployedwith the Cloud Spanner instances being autoscaled in different projects.

Per-project topology

In a per-project topology deployment, each project with a Spannerinstance needing to be autoscaled also has its own independent deployment of theAutoscaler components. We recommend this topology for independent teams who wantto manage their own Autoscaler configuration and infrastructure. It's also a goodstarting point for testing the capabilities of Autoscaler.

The following diagram shows a high-level conceptual view of a per-projectdeployment.

The per-project deployments depicted in the preceding diagram havethese characteristics:

• Two applications, Application 1 and Application 2, each use their ownCloud Spanner instances.
• Spanner instances (A) live in respective Application 1and Application 2 projects.
• An independent Autoscaler (B) is deployed into each project to controlthe autoscaling of the instances within a project.

For a more detailed diagram of a per-project deployment, see theArchitecture section.

A per-project deployment has the following advantages and disadvantages.

Advantages:

• Simplest design: The per-project topology is the simplest design ofthe three topologies since all the Autoscaler components are deployedalongside the Cloud Spanner instances that are being autoscaled.
• Configuration: The control over scheduler parameters belongs to theteam that owns the Spanner instance, which gives the teammore freedom to adapt Autoscaler to its needs than a centralized ordistributed topology.
• Clear boundary of infrastructure responsibility: The design of aper-project topology establishes a clear boundary of responsibility andsecurity over the Autoscaler infrastructure because the team owner of theSpanner instances is also the owner of the Autoscalerinfrastructure.

Disadvantages:

• More overall maintenance: Each team is responsible for theAutoscaler configuration and infrastructure so it might become difficult tomake sure that all of the Autoscaler tools across the company follow thesame update guidelines.
• More complex audit: Because each team has a high level of control, acentralized audit may become more complex.

To learn how to set up Autoscaler using a per-project topology, seeDeploy a per-project or centralized Autoscaler tool for Cloud Spanner.

Centralized topology

As in the per-project topology, in a centralized topology deployment all of thecomponents of Autoscaler reside in the same project. However, theSpanner instances are located in different projects. Thisdeployment is suited for a team managing the configuration and infrastructure ofseveral Cloud Spanner instances from a single deployment of Autoscaler in acentral place.

The following diagram shows a high-level conceptual view of acentralized-project deployment:

The centralized deployment shown in the preceding diagram has the followingcharacteristics:

• Two applications, Application 1 and Application 2, each use their ownCloud Spanner instances.
• Spanner instances (A) are in respectiveApplication 1 and Application 2 projects.
• Autoscaler (B) is deployed into a separate project to control theautoscaling of the Cloud Spanner instances in both the Application 1 andApplication 2 projects.

For a more detailed diagram of a centralized-project deployment, seeDeploy a per-project or centralized Autoscaler tool for Cloud Spanner.

A centralized deployment has the following advantages and disadvantages.

Advantages:

• Centralized configuration and infrastructure: A single teamcontrols the scheduler parameters and the Autoscaler infrastructure. Thisapproach can be useful in heavily regulated industries.
• Less overall maintenance: Maintenance and setup are generally less effortto maintain compared to a per-project deployment.
• Centralized policies and audit: Best practices across teams might beeasier to specify and enact. Audits might be easier to execute.

Disadvantages:

• Centralized configuration: Any change to the Autoscaler parametersneeds to go through the centralized team, even though the team requestingthe change owns the Spanner instance.
• Potential for additional risk: The centralized team itself mightbecome a single point of failure even if the Autoscaler infrastructure isdesigned with high availability in mind.

For a step-by-step tutorial to set up Autoscaler using this option, see theDeploy a per-project or centralized Autoscaler tool for Cloud Spanner.

Distributed topology

In a distributed topology deployment, the Cloud Scheduler andCloud Spanner instances that need to be autoscaled reside in the same project. Theremaining components of Autoscaler reside in a centrally managed project. Thisdeployment is a hybrid deployment. Teams that own the Spannerinstances manage only the Autoscaler configuration parameters for theirinstances, and a central team manages the remaining Autoscaler infrastructure.

The following diagram shows a high-level conceptual view of a distributed-projectdeployment.

The hybrid deployment depicted in the preceding diagram has the followingcharacteristics:

• Two applications, Application 1 and Application 2, use their ownCloud Spanner instances.
• The Spanner instances (A) are in both Application 1 andApplication 2 projects.
• An independent Cloud Scheduler component (C) is deployed intoeach project: Application 1 and Application 2.
• The remaining Autoscaler components (B) are deployed into a separateproject.
• Autoscaler autoscales the Cloud Spanner instances in both theApplication 1 and Application 2 projects using the configurations sent bythe independent Cloud Scheduler components in each project.

For a more detailed diagram of the centralized-project deployment, seeDeploy a distributed Autoscaler tool for Cloud Spanner.

A distributed deployment has the following advantages and disadvantages.

Advantages:

• Application teams control configuration and schedules:Cloud Scheduler is deployed alongside the Cloud Spannerinstances that are being autoscaled, giving application teams more controlover configuration and scheduling.
• Operations team controls infrastructure: Core components ofAutoscaler are centrally deployed giving operations teams control over theAutoscaler infrastructure.
• Centralized maintenance: Scaler infrastructure is centralized,reducing up-keep overhead.

Disadvantages:

• More complex configuration: Application teams need to provideservice accounts to write to the polling topic.
• Potential for additional risk: The shared infrastructure might becomea single point of failure even if the infrastructure is designed with highavailability in mind.

To learn how to set up Autoscaler in a distributed deployment, seeDeploy a distributed Autoscaler tool for Cloud Spanner.

Data splits

Cloud Spanner assigns ranges of data called splits to nodes or subdivisionsof a node called processing units. The node or processing unitsindependently manage and serve the data in the apportioned splits. Data splitsare created based on several factors, including data volume and access patterns.For more details, see Cloud Spanner - schema and data model.

Data is organized into splits and Cloud Spanner automatically manages thesplits. So, when Autoscaler adds or removes nodes or processing units, it needsto allow the Cloud Spanner backend sufficient time to reassign and reorganize the splits as new capacity is added or removed from instances.

Autoscaler uses cooldown periods on both scale-up and scale-down events tocontrol how quickly it can add or remove nodes or processing units from an instance. This method allows the instance the necessary time to reorganize the relationships between compute notes or processing units and data splits. By default, the scale-up and scale-down cooldown periods are set to the following minimum values:

• Scale-up value: 5 minutes
• Scale-down value: 30 minutes

For more information about scaling recommendations and cooldown periods, seeScaling Cloud Spanner Instances.

Costs

Autoscaler resource consumption is minimal so for most use cases, costs arenegligible. There is zero cost when Autoscaler is used on Google Cloud. For example, running an Autoscalerto manage 3 Spanner instances with a polling interval of 5 minutesfor each instance is free of cost. This estimate includes the following:

• 3 Cloud Scheduler Jobs
• 0.15 GB of Pub/Sub messages
• 51840 Cloud Function 500ms invocations
• Less than 10 MB of data in Firestore

The estimate does not include the Cloud Spanner database operation costs. Usethe Pricing Calculator to generate a cost estimate based on your projected usage.

What's next

• Learn how to deploy Autoscaler in per-project or centralized topology.
• Learn how to deploy Autoscaler in distributed topology.
• Read more about Cloud Spanner recommended thresholds.
• Read more about Cloud Spanner CPU utilization metrics.and latency metrics.
• Learn about best practices for Cloud Spanner schema design.to avoid hotspots and for loading data into Cloud Spanner.
• Explore reference architectures, diagrams, and best practices about Google Cloud.Take a look at ourCloud Architecture Center.