Replication Modes

Trilio Site Recovery supports two replication modes that govern how your VM data is mirrored from the primary site to the secondary (DR) site: synchronous (sync) and asynchronous (async). The mode you choose determines your Recovery Point Objective (RPO) — sync replication provides RPO=0 by writing to both arrays before acknowledging I/O, while async replication uses periodic Protection Group snapshots and tolerates a configurable RPO measured in minutes. This guide explains the architectural differences between the two modes, how each maps to Pure Storage FlashArray constructs, and what constraints each imposes on your Cinder volume types and Protection Group configuration — so you can make the right choice for each workload tier.

Before choosing and configuring a replication mode, ensure the following are in place:

• Two independent OpenStack sites are deployed and registered with Trilio Site Recovery — each with its own Nova, Cinder, Neutron, and Keystone endpoints.
• Trilio Site Recovery services (protector-api and protector-engine) are running independently on both sites.
• Pure Storage FlashArray arrays are deployed at both sites. For async mode, an async replication connection must be established between the arrays. For sync mode, an ActiveCluster Pod connection is required.
• Cinder volume types with the correct replication_enabled and replication_type extra specs exist on both sites (detailed in Configuration below).
• The OSC CLI plugin (protectorclient) or the Horizon dashboard is installed and configured with credentials for both sites (via clouds.yaml).
• You have confirmed which workloads require RPO=0 (sync) versus a configurable RPO window (async), as this decision is made at Protection Group creation time and cannot be changed later.

Replication mode is not a separately installed component — it is activated through the combination of correctly configured Cinder volume types and the --replication-type flag when you create a Protection Group. The steps below walk you through preparing the storage layer for each mode.

Step 1: Verify your FlashArray replication topology

For async mode, confirm that an async replication agreement exists between FlashArray A and FlashArray B and that the Protection Group (Pure PG) name you intend to use is available:

# On FlashArray A — verify async replication target exists
# (Run via Pure Storage CLI or GUI; this step is outside OpenStack)
pureadmin list --replicationarray

For sync mode, confirm that an ActiveCluster Pod is configured and stretched between both arrays before proceeding.

Step 2: Create the replicated Cinder volume type on the primary site

# Source primary site credentials
source ~/site-a-openrc

# Create the volume type
openstack volume type create replicated-ssd \
  --description "SSD volumes with async replication to Site B"

# Set required extra specs for async mode
openstack volume type set replicated-ssd \
  --property volume_backend_name=pure@backend-a \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> async'

For sync mode, substitute async with sync:

openstack volume type set replicated-ssd-sync \
  --property volume_backend_name=pure@backend-a \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> sync'

Step 3: Create the matching volume type on the secondary site

The volume type name and extra specs must match on both sites. Protector validates this during Protection Group creation.

# Source secondary site credentials
source ~/site-b-openrc

# Async
openstack volume type create replicated-ssd \
  --description "SSD volumes replicated from Site A"

openstack volume type set replicated-ssd \
  --property volume_backend_name=pure@backend-b \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> async'

Step 4: Validate volume type replication readiness

Use the Protector site introspection command to confirm the volume type is visible and replication-capable on both sites:

openstack protector site list-volume-types site-a
openstack protector site list-volume-types site-b

Confirm that replication_enabled shows True and replication_type matches your intended mode in the output for both sites before proceeding to create a Protection Group.

Replication mode is determined by two interacting layers of configuration: Cinder volume type extra specs (storage-layer) and the Protection Group replication_type field (service-layer). Both must be consistent.

Cinder Volume Type Extra Specs

Protection Group replication_type Field

When you create a Protection Group, you specify the replication mode with --replication-type. Valid values:

This field is set at creation time and is immutable — you cannot change the replication type of an existing Protection Group.

Replication Policy Parameters (Async Only)

For async Protection Groups, a replication policy must be attached. The policy provides Pure Storage array credentials and controls the replication schedule:

openstack protector protection-group policy-create <pg-name> \
  --primary-fa-url https://flasharray-a.example.com \
  --primary-fa-token "T-..." \
  --secondary-fa-url https://flasharray-b.example.com \
  --secondary-fa-token "T-..." \
  --pure-pg-name "pg-prod-web-app" \
  --replication-interval 300 \
  --rpo-minutes 15

For sync mode Protection Groups, no replication policy is required — the ActiveCluster Pod handles continuous replication at the storage layer without a configurable interval.

All Volumes in a Protection Group Must Use the Same Mode

All Cinder volumes belonging to a Protection Group are placed in a single Cinder Consistency Group, and all volumes in that Consistency Group must share the same volume type. This means you cannot mix async and sync volumes within a single Protection Group. Create separate Protection Groups for workloads with different RPO requirements.

Choosing Between Async and Sync

Use async replication for the majority of workloads. It has no impact on primary-site write latency, scales to large volumes, and lets you tune the RPO window (typically 5–60 minutes) to balance storage bandwidth against recovery objectives. Async is the default choice for application tiers that can tolerate a small data loss window — web servers, batch jobs, development environments, and most database replicas.

Use sync replication when your workload cannot tolerate any data loss. Because every write must be committed on both arrays before it is acknowledged to the guest OS, sync replication adds write latency proportional to the round-trip time between your two sites. Pure Storage ActiveCluster is designed for stretched metropolitan deployments with low inter-site latency (typically under 5 ms RTT). Use sync for Tier-1 databases, financial transaction systems, or any application where RPO=0 is a hard regulatory or SLA requirement.

Creating a Protection Group with Async Replication

# Source primary site credentials
source ~/site-a-openrc

openstack protector protection-group create \
  --name prod-web-app \
  --description "Production web application" \
  --replication-type async \
  --primary-site site-a \
  --secondary-site site-b \
  --volume-type replicated-ssd

After creation, attach a replication policy to configure the async schedule:

openstack protector protection-group policy-create prod-web-app \
  --primary-fa-url https://flasharray-a.example.com \
  --primary-fa-token "T-12345678-abcd-..." \
  --secondary-fa-url https://flasharray-b.example.com \
  --secondary-fa-token "T-87654321-dcba-..." \
  --pure-pg-name "pg-prod-web-app" \
  --replication-interval 300 \
  --rpo-minutes 15

Creating a Protection Group with Sync Replication

openstack protector protection-group create \
  --name prod-db-tier \
  --description "Tier-1 database — RPO=0" \
  --replication-type sync \
  --primary-site site-a \
  --secondary-site site-b \
  --volume-type replicated-ssd-sync

No replication policy is needed for sync Protection Groups. The ActiveCluster Pod provides continuous zero-RPO replication transparently.

Checking Replication Mode on an Existing Protection Group

openstack protector protection-group show prod-web-app

The replication_type field in the output confirms the mode. The consistency_group_id field links you to the underlying Cinder Consistency Group, which in turn maps 1:1 to a Pure Storage Protection Group (async) or Pod (sync).

Forcing a Manual Sync (Async Only)

If you need to trigger an out-of-schedule replication before a planned failover:

openstack protector consistency-group sync prod-web-app

This is particularly useful before a planned failover to minimize the recovery point gap. It is not applicable to sync Protection Groups, where replication is continuous.

Example 1: End-to-end async Protection Group setup

Create a volume type on both sites, create an async Protection Group, attach a policy, and verify replication readiness.

# --- Site A ---
source ~/site-a-openrc

openstack volume type create replicated-ssd \
  --property volume_backend_name=pure@backend-a \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> async'

openstack protector protection-group create \
  --name prod-web-app \
  --replication-type async \
  --primary-site site-a \
  --secondary-site site-b \
  --volume-type replicated-ssd

openstack protector protection-group policy-create prod-web-app \
  --primary-fa-url https://flasharray-a.example.com \
  --primary-fa-token "T-12345678-abcd-efgh-ijkl-mnopqrstuvwx" \
  --secondary-fa-url https://flasharray-b.example.com \
  --secondary-fa-token "T-87654321-dcba-hgfe-lkji-xwvutsrqponm" \
  --pure-pg-name "pg-prod-web-app" \
  --replication-interval 300 \
  --rpo-minutes 15

Expected output from protection-group create:

+------------------------+--------------------------------------+
| Field                  | Value                                |
+------------------------+--------------------------------------+
| id                     | pg-12345678-1234-1234-1234-12345678 |
| name                   | prod-web-app                         |
| status                 | active                               |
| replication_type       | async                                |
| consistency_group_id   | cg-87654321-4321-4321-4321-87654321 |
| primary_site           | site-a                               |
| secondary_site         | site-b                               |
+------------------------+--------------------------------------+

Example 2: Sync Protection Group for a database tier

source ~/site-a-openrc

# Volume type (both sites must have this)
openstack volume type create replicated-ssd-sync \
  --property volume_backend_name=pure@backend-a \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> sync'

# Protection Group — no policy needed for sync
openstack protector protection-group create \
  --name prod-db-tier \
  --description "Tier-1 database — RPO=0" \
  --replication-type sync \
  --primary-site site-a \
  --secondary-site site-b \
  --volume-type replicated-ssd-sync

Expected output:

+------------------------+--------------------------------------+
| Field                  | Value                                |
+------------------------+--------------------------------------+
| id                     | pg-abcdefab-cdef-abcd-efab-cdefabcd |
| name                   | prod-db-tier                         |
| status                 | active                               |
| replication_type       | sync                                 |
| consistency_group_id   | cg-fedcbafe-dcba-fedc-baFE-dcbafedcba|
| primary_site           | site-a                               |
| secondary_site         | site-b                               |
+------------------------+--------------------------------------+

Note that no replication policy appears under openstack protector protection-group policy show prod-db-tier — this is expected for sync Protection Groups.

Example 3: Validating volume type replication eligibility before adding a VM

Attempting to add a VM whose volumes use a non-replicated volume type to a Protection Group will fail. Check eligibility first:

# List volume types and check replication properties
openstack protector site list-volume-types site-a

Expected output (showing eligible and ineligible types):

+------------------+---------------------+-----------------+--------------------+
| Name             | Backend             | Repl Enabled    | Replication Type   |
+------------------+---------------------+-----------------+--------------------+
| replicated-ssd   | pure@backend-a      | True            | async              |
| replicated-ssd-s | pure@backend-a      | True            | sync               |
| local-ssd        | pure@backend-local  | False           | N/A                |
+------------------+---------------------+-----------------+--------------------+

Only volume types with Repl Enabled = True and a matching Replication Type are eligible for use with a Protection Group.

Example 4: Forcing a manual sync before planned failover (async)

# Trigger an out-of-schedule replication snapshot
openstack protector consistency-group sync prod-web-app

Expected output:

Forcing consistency group sync for prod-web-app...
  Consistency Group: cg-87654321-4321-4321-4321-87654321
  Backend: pure@backend-a

✅ Sync initiated successfully.
Monitor replication lag on FlashArray B to confirm snapshot arrival
before executing planned failover.

Issue: Protection Group creation fails with "volume type does not support replication"

Symptom: openstack protector protection-group create returns an error indicating the specified volume type is not replication-eligible.

Likely cause: The Cinder volume type is missing one or both required extra specs: replication_enabled='<is> True' and replication_type='<in> async' (or sync).

Fix:

# Check current extra specs
openstack volume type show replicated-ssd

# Add or correct the missing specs
openstack volume type set replicated-ssd \
  --property replication_enabled='<is> True' \
  --property replication_type='<in> async'

Repeat this on both sites — Protector validates the volume type on the secondary site as well.

Issue: Adding a VM to a Protection Group fails with "volume type mismatch"

Symptom: openstack protector protection-group member-add fails, indicating that one or more of the VM's volumes do not use the Protection Group's configured volume type.

Likely cause: The VM has volumes of mixed types, or at least one volume uses a non-replicated volume type (e.g., local-ssd).

Fix: All volumes attached to a VM that is added to a Protection Group must use the same replicated volume type as the Protection Group. Migrate non-compliant volumes to the correct type before adding the VM:

# Check volumes attached to the VM
openstack volume list --server <instance-id>

# Check each volume's type
openstack volume show <volume-id> -c volume_type

If a volume uses the wrong type, create a new volume of the correct type and migrate the data, or retype the volume if the backend supports it.

Issue: Sync Protection Group has unexpectedly high write latency

Symptom: After adding VMs to a sync Protection Group, application write latency increases significantly compared to the async baseline.

Likely cause: This is expected behavior for sync replication — every write must be acknowledged by both FlashArrays before returning to the guest. Latency scales with the round-trip time between sites. If latency is higher than expected, the inter-site network RTT may be too high for ActiveCluster.

Fix: Measure the network RTT between the two FlashArray management and data interfaces. Pure Storage ActiveCluster is designed for metropolitan distances (typically under 5 ms RTT). If your sites exceed this threshold, switch to async replication for the affected workloads by creating a new async Protection Group with an appropriate RPO target.

Issue: replication_type mismatch between Protection Group and volume type

Symptom: A Protection Group created with --replication-type sync cannot accept volumes from a volume type configured with replication_type='<in> async', or vice versa.

Likely cause: The volume type's replication_type extra spec does not match the Protection Group's replication_type. Protector enforces consistency between these two layers.

Fix: Use a volume type whose replication_type extra spec matches the Protection Group mode. You need separate volume types for async and sync workloads — create them with distinct names (e.g., replicated-ssd-async and replicated-ssd-sync) and use the appropriate type when creating each Protection Group.

Issue: consistency-group sync fails for an async Protection Group

Symptom: openstack protector consistency-group sync prod-web-app returns an error or reports that the secondary site is unreachable.

Likely cause: The secondary site's Protector API or FlashArray is unreachable, or the Pure Storage replication connection between the two arrays has been interrupted.

Fix:

1. Validate site connectivity: openstack protector site validate site-b
2. Check the replication policy is correctly configured: openstack protector protection-group policy show prod-web-app
3. Verify the FlashArray replication connection is healthy from the Pure Storage management console.
4. If the secondary site is confirmed unreachable, note that Protector will block modifications to the Protection Group until connectivity is restored — this is by design to prevent metadata divergence. Restore connectivity, then retry.

Issue: No replication policy needed message is confusing for sync groups

Symptom: After creating a sync Protection Group, running openstack protector protection-group policy show <name> returns "not found" or an empty result, and you are unsure whether setup is complete.

Likely cause: This is correct and expected. Sync Protection Groups use Pure Storage ActiveCluster Pods for continuous zero-RPO replication — no snapshot schedule or policy is required. The absence of a replication policy is not an error for replication_type: sync.

Fix: No action required. Confirm the Protection Group status is active via openstack protector protection-group show <name> and proceed to add VM members.