Authentication and Microversioning

This page explains how Trilio Site Recovery authenticates API requests and how clients declare which API capabilities they expect. Every request to the protector-api service must carry a valid Keystone token, and every request that uses features introduced after the base API version must include an OpenStack-API-Version header. Understanding both mechanisms matters because the protector service runs independently on each OpenStack site — your client must authenticate to each site's Keystone separately and negotiate the microversion independently with each site's protector-api endpoint. This page also covers version discovery, which lets you inspect what microversion range a given deployment supports before committing to a header value.

Before working with authentication and microversioning, you need:

• Two independent OpenStack deployments — a primary site and a secondary (DR) site, each with its own Keystone, Nova, Cinder, and Neutron endpoints.
• A Keystone service account (protector) with the admin role in the service project on each site. This account is created during deployment (see the deployment guide).
• A tenant (project) user with the member role in the project you intend to protect. The protector service will acquire a Keystone trust from this user on first use.
• The protectorclient OSC plugin installed and configured with a clouds.yaml that defines credentials for both sites. The CLI is the coordination layer — it authenticates to both Keystones and orchestrates metadata sync across them.
• Network reachability: your workstation or automation host must be able to reach the Keystone endpoint (port 5000) and the protector-api endpoint (port 8788, by default) on both sites.
• OpenStack Victoria or later on both sites.

The protectorclient OSC plugin is the recommended way to interact with the protector API. It handles token acquisition, microversion header injection, and cross-site coordination for you.

Step 1 — Install the plugin

pip install protectorclient

Step 2 — Configure clouds.yaml

Create or update ~/.config/openstack/clouds.yaml with an entry for each site. The CLI uses these entries to authenticate independently to each Keystone:

clouds:
  site-a:
    auth:
      auth_url: http://site-a-controller:5000/v3
      project_name: myproject
      username: myuser
      password: mypassword
      user_domain_name: Default
      project_domain_name: Default
    region_name: RegionOne

  site-b:
    auth:
      auth_url: http://site-b-controller:5000/v3
      project_name: myproject
      username: myuser
      password: mypassword
      user_domain_name: Default
      project_domain_name: Default
    region_name: RegionOne

Step 3 — Verify the plugin is recognized

openstack --os-cloud site-a protector protection-group list

If the plugin is installed correctly you will see either an empty table or a list of existing protection groups. A No such command error means the plugin is not on the Python path used by the openstack binary.

Step 4 — Confirm you can reach both sites

openstack --os-cloud site-a token issue
openstack --os-cloud site-b token issue

Both commands should return a token ID and expiry. Any authentication error here will prevent the CLI from coordinating cross-site operations.

Keystone token authentication

The protector-api service validates every incoming request through keystonemiddleware.auth_token. Configuration lives in /etc/protector/protector.conf under [keystone_authtoken]:

Example [keystone_authtoken] stanza:

[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = protector
password = PROTECTOR_PASS

Each site has its own protector.conf pointing at its own Keystone. There is no shared token validation between sites.

Microversioning

The protector API uses the OpenStack-API-Version header for microversioning, following the standard OpenStack microversion contract:

The header format is:

OpenStack-API-Version: protector <version>

If you omit the header, the API responds using the base version (1.0) semantics. If you send a version higher than the server supports, the server returns HTTP 406 Not Acceptable with a JSON body indicating the supported range. If you send a malformed header the server returns HTTP 400 Bad Request.

The protectorclient OSC plugin automatically injects the highest mutually supported microversion. When making raw HTTP calls (cURL, Python requests), you must set the header explicitly.

RBAC policy

The file /etc/protector/policy.yaml controls which Keystone roles can call which API operations. The default policy grants all protection-group, member, operation, and policy endpoints to admin_or_owner (either an admin user, or the user whose project matches the URL's {tenant_id}). Admin-only endpoints such as site management (/v1/admin/sites) enforce context_is_admin. Do not relax the admin-only restriction on site endpoints — site records contain service credentials for both clusters.

Authenticating as a tenant user

All tenant-scoped protector endpoints are under /v1/{tenant_id}/. You need a Keystone token scoped to the correct project:

# Source tenant credentials or use --os-cloud
source ~/myproject-openrc

# Obtain a token and project ID
TOKEN=$(openstack token issue -f value -c id)
TENANT_ID=$(openstack token issue -f value -c project_id)

Pass the token as the X-Auth-Token header on every request. Tokens are short-lived (default 1 hour in Keystone). Your automation should detect a 401 Unauthorized response and re-issue a token rather than caching tokens indefinitely.

Sending the microversion header

Include OpenStack-API-Version: protector 1.1 (or the version you require) on every request:

curl -s \
  -H "X-Auth-Token: $TOKEN" \
  -H "OpenStack-API-Version: protector 1.1" \
  http://site-a-controller:8788/v1/$TENANT_ID/protection-groups

Omitting the header is valid but restricts you to base-version (1.0) behavior. It is good practice to pin to the highest version your integration has been tested against rather than using latest, so that future server upgrades do not change behavior unexpectedly.

Authenticating to both sites

Because the two protector-api services are completely independent, cross-site CLI operations require credentials for both sites. The protectorclient plugin reads --source-cloud and --target-cloud (or their clouds.yaml equivalents) and acquires separate tokens from each Keystone:

openstack protector protection-group create \
  --name prod-web-app \
  --replication-type async \
  --primary-site site-a \
  --secondary-site site-b \
  --volume-type replicated-ssd \
  --os-cloud site-a

The plugin authenticates to site-a for the API call and authenticates to site-b to validate that the secondary site is reachable and that metadata can be synced. Metadata sync is blocked if the peer site is unreachable — this is intentional to prevent the two sites from diverging.

Version discovery

Before pinning a microversion, query the version endpoint to confirm what the server supports. This is especially important in mixed-version deployments where site-a and site-b may be on different patch levels:

curl -s http://site-a-controller:8788/

Example 1 — Version discovery

Query the root URL to retrieve the API version document. No authentication is required for this endpoint.

curl -s http://site-a-controller:8788/

Expected response:

{
  "versions": [
    {
      "id": "v1.0",
      "status": "CURRENT",
      "min_version": "1.0",
      "max_version": "1.2",
      "links": [
        {
          "href": "http://site-a-controller:8788/v1/",
          "rel": "self"
        }
      ]
    }
  ]
}

The min_version and max_version fields tell you the range your client can use in OpenStack-API-Version.

Example 2 — Obtain a Keystone token and call the protector API

This pattern is the baseline for any scripted interaction with the protector API.

# Step 1: source credentials for the primary site
export OS_AUTH_URL=http://site-a-controller:5000/v3
export OS_PROJECT_NAME=myproject
export OS_USERNAME=myuser
export OS_PASSWORD=mypassword
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default

# Step 2: obtain token and project ID
TOKEN=$(openstack token issue -f value -c id)
TENANT_ID=$(openstack token issue -f value -c project_id)

# Step 3: list protection groups with microversion 1.1
curl -s \
  -H "X-Auth-Token: $TOKEN" \
  -H "OpenStack-API-Version: protector 1.1" \
  http://site-a-controller:8788/v1/$TENANT_ID/protection-groups

Expected response (empty project):

{
  "protection_groups": []
}

Example 3 — Request with an unsupported microversion

Sending a version number above the server's max_version produces a 406 error that tells you the acceptable range:

curl -s -i \
  -H "X-Auth-Token: $TOKEN" \
  -H "OpenStack-API-Version: protector 9.9" \
  http://site-a-controller:8788/v1/$TENANT_ID/protection-groups

Expected response:

HTTP/1.1 406 Not Acceptable
Content-Type: application/json

{
  "error": {
    "code": 406,
    "message": "Version 9.9 is not supported by the API. Minimum is 1.0, maximum is 1.2.",
    "title": "Not Acceptable"
  }
}

Example 4 — Python script authenticating to both sites independently

This pattern mirrors how the protectorclient plugin coordinates across sites.

import requests
from keystoneauth1 import session
from keystoneauth1.identity import v3

def get_token_and_project(auth_url, username, password, project_name,
                           user_domain='Default', project_domain='Default'):
    auth = v3.Password(
        auth_url=auth_url,
        username=username,
        password=password,
        project_name=project_name,
        user_domain_name=user_domain,
        project_domain_name=project_domain
    )
    sess = session.Session(auth=auth)
    return sess.get_token(), sess.get_project_id()

# Authenticate to site-a
site_a_token, site_a_project = get_token_and_project(
    auth_url='http://site-a-controller:5000/v3',
    username='myuser',
    password='mypassword',
    project_name='myproject'
)

# Authenticate to site-b (separate Keystone — token from site-a is not valid here)
site_b_token, _ = get_token_and_project(
    auth_url='http://site-b-controller:5000/v3',
    username='myuser',
    password='mypassword',
    project_name='myproject'
)

HEADERS_A = {
    'X-Auth-Token': site_a_token,
    'OpenStack-API-Version': 'protector 1.1',
    'Content-Type': 'application/json'
}

# Use site-a token for site-a API calls
response = requests.get(
    f'http://site-a-controller:8788/v1/{site_a_project}/protection-groups',
    headers=HEADERS_A
)
print(response.json())

Expected output:

{"protection_groups": []}

401 Unauthorized on every request

Symptom: Every API call returns HTTP 401 regardless of the operation.

Likely cause: The X-Auth-Token header is missing, the token has expired, or the token was issued by the wrong Keystone (for example, a site-b token sent to site-a's protector-api).

Fix:

1. Re-issue a token with openstack token issue.
2. Confirm OS_AUTH_URL points to the same site as the protector endpoint you are calling. Site-a tokens are not valid on site-b's Keystone.
3. Check the keystonemiddleware configuration in /etc/protector/protector.conf: www_authenticate_uri and auth_url must both point to this site's Keystone.

403 Forbidden on tenant-scoped endpoints

Symptom: You can list protection groups but cannot create or modify them. Or you receive 403 even though you own the resource.

Likely cause: The {tenant_id} in the URL does not match the project scope of your token, or the RBAC policy is more restrictive than the default.

Fix:

1. Verify the {tenant_id} you are using matches openstack token issue -f value -c project_id.
2. Check /etc/protector/policy.yaml on the server. The default admin_or_owner rule should permit project members; a misconfigured custom policy may be blocking access.
3. Confirm the protector service was reloaded after any policy file change (systemctl restart protector-api).

406 Not Acceptable — version negotiation failure

Symptom: The API returns HTTP 406 with a message indicating your requested version is out of range.

Likely cause: The OpenStack-API-Version header specifies a version higher than the server's max_version (currently 1.2) or lower than min_version (1.0).

Fix:

1. Query the version discovery endpoint (curl http://<host>:8788/) to confirm the server's supported range.
2. Adjust the header to a value within the reported [min_version, max_version] range.
3. If you are using the protectorclient plugin and still see this error, confirm the plugin version is compatible with the server version.

Metadata sync blocked — peer site unreachable

Symptom: Protection group create, update, or member-add operations fail with an error indicating the secondary site cannot be reached.

Likely cause: The protector service enforces strict metadata consistency — modifications to a Protection Group are blocked when the peer site is unreachable. This prevents the two sites from diverging. The error is expected behavior, not a bug.

Fix:

1. Confirm network connectivity from the primary site's protector-engine to the secondary site's Keystone (port 5000) and protector-api (port 8788).
2. Verify the secondary site's service credentials stored in the site record are still valid: openstack --os-auth-url <site-b-keystone> --os-username protector-service --os-password <password> --os-project-name service token issue.
3. Check that protector-api is running on the secondary site: systemctl status protector-api on the secondary controller.

Authentication works from CLI but fails from automation

Symptom: openstack token issue works interactively but a script or CI job receives 401.

Likely cause: The script is using environment variables or a clouds.yaml entry that points to a different auth_url, project, or credentials than the interactive session.

Fix:

1. Add -v or --debug to the openstack command in the script to print the resolved auth URL and project.
2. Confirm OS_AUTH_URL, OS_PROJECT_NAME, OS_USERNAME, and OS_PASSWORD are set correctly in the automation environment — they are not inherited from an interactive source openrc in the same shell.
3. If using clouds.yaml, confirm the --os-cloud value matches the key in the YAML file and that the file is readable by the process user.