In this article, we will walk through the step-by-step process to migrate Apache Kafka, the Confluent Operator, and their associated Custom Resources (CRs) and Custom Resource Definitions (CRDs) using SWIFT. 


When setting up a Stage sync with a Disaster Recovery (DR) policy using SWIFT, it's critical to handle dependencies in the correct order. This ensures a smooth migration process without breaking components or causing failures due to unresolved resource dependencies.


#Why Dependency Resolution is Important

Kafka and the Confluent ecosystem rely heavily on CRDs and associated resources. If these are not properly applied in the correct sequence during the migration, the Kafka components may fail to initialize or synchronize correctly.


1. Go to the SWIFT dashboard and login. From the left-hand menu, go to Business Continuity & DR , Click on DR Policies to create a stage1+2 policy.


 2. Once you are on the DR Policies page, follow the steps below to create a new policy:

  1. Click on 'New'

    • This will open a prompt to configure the new DR policy.

  2. Provide Required Details

    • DR Policy Name: Enter a meaningful name for your policy.

    • Sync Type: Choose the appropriate sync type (e.g., by-schedule, by-frequency, once, or continuous).

    • Periodicity: Specify the sync periodicity for Stage 1 and Stage 2.

  3. Create the Policy

    • After entering the required details, click on 'Create'.

    • The new DR policy will be created and listed under the DR Policies section.



 



3. To apply the newly created DR Policy, follow the steps below. The first step involves syncing the Confluent Operator deployment along with the Sync Webhook. 


  • Sync should be done at Order Level 0 to establish the foundational components.




  • Once you have selected Order Level 0, proceed with the following steps to configure the source and target environments for the sync operation:

    1. Select Source and Target Clusters

      • Choose the source cluster where your Kafka and Confluent Operator are currently running.

      • Select the target cluster where you want to replicate these components.

    2. Define Namespaces

      • In the source namespace field, select the namespace where Kafka is currently deployed.

      • In the target namespace field, specify the namespace in the target cluster where the resources should be created.

    3. Application Selection

      • Under the Application section, select 'Selective' to choose specific components for synchronization.

    4. Enable Sync Webhook

      • Under the 'Sync Webhook'  section, click on 'All' to ensure all webhook configurations are included in the sync.



4.  Since you have chosen the Selective option in the Application section, follow the steps below to select and apply the required resource:

  1. Select the Deployment Object

    • From the list of available resources, select only the 'Deployment' object named confluent-operator.

    • This ensures that only the 'Confluent Operator' is synced at this stage.

  2. Apply the Sync

    • After selecting the confluent-operator deployment, click on the 'Apply' button to start the sync process.






5. Now in the Order Level 1, you need to do selective sync for only CR and CRD's. Please find below steps for that.


#click on DR Policy and apply the same and select Order Level-1 as below.



#Now choose both clusters and select 'Selective' option as below.



# Select the 'Namespace' in CRD scope under Customer resource configuration section.




# Now Apply the policy.



# Last order level-2 for all apps. You just select the order-level-2 as below. No need to select anything, its all basic things will be needed and then click on 'Apply' button to start the sync.


# Select below option for all apps.




6. Once you have added all the order levels, you can expand the DR Policy. It will look as shown below, displaying all the order levels you selected. 




7. Once sync completed, then it will looks like below.


8 .Once all objects, including CRs and CRDs, have been migrated, you can check on the target cluster to verify whether all the pods have come up. In the snippet below, you can see that all the pods, services, CRs, and CRDs are up and running.




9. Also we were able to access the Kafka UI.