Configure rack awareness in Aiven for Apache Kafka® MirrorMaker 2

Configure rack awareness in Aiven for Apache Kafka® MirrorMaker 2 to reduce cross-availability zone (AZ) network traffic by directing MirrorMaker to read from local follower replicas instead of remote partition leaders.

About rack awareness in MirrorMaker 2

Rack awareness in MirrorMaker 2 depends on follower fetching being enabled on both the source Aiven for Apache Kafka® service and the MirrorMaker 2 service.

When enabled, MirrorMaker prefers reading from in-sync follower replicas in the same availability zone as the MirrorMaker node. This reduces cross-availability zone (AZ) network traffic and associated costs.

If follower fetching is disabled on either service, MirrorMaker reads only from partition leaders and rack awareness has no effect.

Rack awareness is supported only for integrations with Aiven-hosted Kafka services. It is automatically disabled for external Kafka clusters because availability zones cannot be reliably mapped across cloud providers.

For details on follower fetching, see Follower fetching in Aiven for Apache Kafka®.

When to use rack awareness

Use rack awareness when:

• MirrorMaker 2 and the source Kafka service run across multiple availability zones.
• Cross-AZ network costs or latency are a concern.
• The source Kafka service is hosted on Aiven.

Rack awareness does not provide benefits when the Kafka cluster runs in a single availability zone.

Configuration hierarchy

By default, MirrorMaker 2 assigns the node availability zone as the rack ID and uses follower replicas in the same availability zone.

Rack awareness behavior follows this order:

1. Replication flow setting: If follower_fetching_enabled is set to false for a replication flow, rack awareness is disabled for that flow.
2. MirrorMaker 2 service setting: If kafka_mirrormaker.follower_fetching_enabled is set to false, rack awareness is disabled for all replication flows.
3. Integration type: Rack awareness is disabled for integrations with external Kafka clusters.
4. Rack ID selection for Aiven-hosted Kafka services:
   • If rack_id is set for the integration, that value is used.
   • Otherwise, the node availability zone is used as the rack ID.

Prerequisites

• A running Aiven for Apache Kafka® MirrorMaker 2 service.
• Follower fetching enabled on:
  • The source Aiven for Apache Kafka® service
  • The MirrorMaker 2 service
• Availability zone (AZ) information for Kafka brokers if you plan to configure a custom rack_id value:
  • AWS: Map AZ IDs to names
  • Google Cloud: Regions and zones

Enable or disable rack awareness for a replication flow

Rack awareness is controlled by the follower fetching setting and can be enabled or disabled per replication flow.

1. In the Aiven Console, open the MirrorMaker 2 service.
2. Click Replication flows.
3. Create a replication flow or edit an existing one.
4. Set Follower fetching enabled to on or off.
5. Click Create or Save.

Enable or disable rack awareness for the MirrorMaker 2 service

Disabling follower fetching at the service level disables rack awareness for all replication flows.

Disable this setting only if MirrorMaker 2 must always read from partition leaders.

Console

1. In the Aiven Console, open the MirrorMaker 2 service.
2. Click Service settings.
3. In Advanced configuration, click Configure.
4. Click Add configuration options.
5. Set kafka_mirrormaker.follower_fetching_enabled to Disabled.
6. Click Save configurations.

CLI

avn service update SERVICE_NAME \
  -c kafka_mirrormaker.follower_fetching_enabled=false

API

curl --request PUT \
  --url https://api.aiven.io/v1/project/PROJECT_NAME/service/SERVICE_NAME \
  --header 'Authorization: Bearer TOKEN' \
  --header 'content-type: application/json' \
  --data '{
    "user_config": {
      "kafka_mirrormaker": {
        "follower_fetching_enabled": false
      }
    }
  }'

Terraform

resource "aiven_kafka_mirrormaker" "mm2" {
  project      = var.project_name
  service_name = "my-mm2-service"
  cloud_name   = "google-us-east1"
  plan         = "business-4"

  kafka_mirrormaker_user_config {
    kafka_mirrormaker {
      follower_fetching_enabled = false
    }
  }
}

Override the rack ID per integration

To override the default AZ-based rack assignment, set a custom rack_id in the MirrorMaker 2 integration configuration.

Console

1. In the Aiven Console, open your MirrorMaker 2 service.
2. Click Integrations.
3. Select the integration to update.
4. In Advanced configuration, set the rack_id to your custom value (for example, use1-az1).
5. Click Save changes.

CLI

avn service integration-update \
  --project PROJECT_NAME \
  --user-config-json '{"rack_id": "use1-az1"}' \
  INTEGRATION_ID

Replace PROJECT_NAME, INTEGRATION_ID, and use1-az1.

API

curl --request PUT \
  --url https://api.aiven.io/v1/project/PROJECT_NAME/integration/INTEGRATION_ID \
  --header 'Authorization: Bearer TOKEN' \
  --header 'content-type: application/json' \
  --data '{
    "user_config": {
      "rack_id": "use1-az1"
    }
  }'

Replace the placeholders with your values.

Terraform

resource "aiven_service_integration" "mm2_integration" {
  project                  = var.project_name
  integration_type         = "kafka_mirrormaker"
  source_service_name      = aiven_kafka.source.service_name
  destination_service_name = aiven_mirrormaker.mm2.service_name

  mirrormaker_user_config {
    cluster_alias = "source-cluster"
    rack_id       = "use1-az1"
  }
}

Example configurations

For a deeper explanation of how rack awareness works with follower fetching, see Enable follower fetching in Aiven for Apache Kafka®.

Below are examples specific to MirrorMaker 2 integrations.

AZ-based rack ID (default)

Node AZ	Rack ID used	Behavior
use1-az1	use1-az1	Reads and writes to brokers in use1-az1
europe-west1-b	europe-west1-b	Reads and writes to brokers in the same AZ

Custom rack ID per integration

{
  "cluster_alias": "kafka-source",
  "rack_id": "custom-rack-1"
}

This overrides the node's AZ. MirrorMaker uses custom-rack-1 for that integration.

Mixed setup

Integration	rack_id value	Rack ID used
Aiven-hosted service A	Not set	Node’s AZ
Aiven-hosted service B	custom-rack	custom-rack
External Kafka service	Any value	Rack awareness disabled

Related pages

• Follower fetching in Aiven for Apache Kafka®
• Enable follower fetching in Aiven for Apache Kafka®
• Aiven for Apache Kafka® MirrorMaker 2