When databases fail and network paths falter, you still need your mission-critical cloud services to stay online. Yet guaranteeing high availability has become increasingly difficult because of the complexity of modern distributed systems. 

To help you maintain availability and reliability during adverse events, we’re announcing Fault Injection Testing in preview. Fault Injection Testing is designed to help developers and architects automate failure testing to ensure predictable behavior during disruptions. 

By deliberately introducing faults into your environment, you can verify your safety mechanisms before an actual outage impacts your customers.

Why native resilience testing matters

Unlike in self-hosted data centers, cloud applications offer less direct access to underlying infrastructure to facilitate failover testing.

Without native tools to prove your application can survive a failure, you risk a critical gap in your reliability strategy that exposes you to several risks:

• Damaged trust and reputation: Frequent failures or poor performance lead to customer dissatisfaction and long-term damage to your brand's image.

• Compliance and regulatory penalties: For many industries, particularly financial institutions, failing to prove disaster recovery capabilities can lead to non-compliance, audits, and fines.

• Migration delays: Large-scale migrations often stop when teams cannot verify that critical applications will remain stable during a zone failure.

How Fault Injection Testing works

Fault Injection Testing allows you to run experiments by creating experiment templates. These templates act as blueprints, defining the specific fault to be injected and the resources that will be targeted for the experiment.

In this public preview, you can test two primary failure scenarios:

• Failover Cloud SQL: This fault triggers a failover of a high availability Cloud SQL instance from the primary zone to a standby zone.

• Degrade application traffic: This allows you to selectively add latency and HTTP error codes through a Layer 7 load balancer.

Before any fault is injected, Fault Injection Testing performs an automated dry run. This read-only simulation checks your permissions and provides an up-to-date list of every resource that will be affected. 

Once you verify the scope, you can manually start the injection. The duration you defined in the template will run its course, and the faults will be reverted at the expiration of the timer.  

During the experiment, you can verify that your application is behaving as you planned.  If things do not go as planned, you can use the stop and revert capability to immediately halt the experiment and begin restoring resources to their normal state.

During preview, we recommend as a best practice to use FIT in a non-production environment. Preview is an opportunity to get early access to learn how the service fits and complements your existing testing practices, and to provide us with your feedback to improve the product as well!

Built for the enterprise

Partners like KeyBank and Servier are already using Fault Injection Testing to validate their deployments. By using native fault injection, these organizations can approximate demanding failure scenarios — such as zonal outages — to help ensure their services remain stable.

Get started with Fault Injection Testing

Fault Injection Testing is available through the Google Cloud console, the gcloud CLI, and REST APIs.

1. Request preview access: Talk to your Google Cloud Account Team to add your project to the preview.

2. Enable the API: Search for "Fault Testing API" in your Google Cloud console and select enable.

3. Assign roles: Ensure your team has the roles/faulttesting.operator role to configure and run experiments.

4. Run your first dry run: Create a template for a Cloud SQL or load balancer resource in a non-production environment and execute a dry run to see the potential impact.

For more details on implementation, talk to your account team, or view the User Guide for Fault Injection Testing.