Delete a live node

To delete a live node that you have access to, do the following:

  1. Login to the live node and perform the following steps with administrative rights. This local node must be alive, and the Cassandra service must be in Up and Normal status.
    You can check the local Cassandra node status by running the following command:
    $ "<Install folder>\Shared Services\Cassandra\bin\nodetool.bat" status
    The JAVA_HOME environment variable must be set to "<Install folder>\Shared Services\Cassandra\JRE" before running "nodetool status".
  2. Choose the SFS/Cassandra helper ‘supervisor’ node where monitoring and verifying live node removal and unregistering SFS base services are located. Temporarily disable "CassandraScheduledTaskRunner" in Windows Task Scheduler on the ‘supervisor’ node to avoid any unexpected Cassandra maintenance operations during live node removal.
    The scheduled task can be disabled through the Task Scheduler application or by running the following command:
    $ schtasks /Change /TN "CassandraScheduledTaskRunner" /Disable
  3. Continue with the live node removal steps only if "nodetool status" shows all nodes are 'Up' in the Cassandra cluster. If there are any dead nodes, remove them first. (see Delete a dead node). If any node are 'Down' due to a cluster communication issues, fix them first.
  4. Remove the CassandraScheduledTaskRunner scheduled task from the Windows Task Scheduler on the live decommissioning node in order to avoid any unexpected Cassandra access or operation during the node removal.
    The scheduled task can be removed through the Task Scheduler application or by running the following command:
    $ schtasks /Delete /TN "CassandraScheduledTaskRunner" /F
  5. Check if the “CassandraScheduledTaskRunner.exe” process is running by using Task Manager and wait for it to complete.
  6. Run the following command on the live decommissioning node:
    $ "<Install folder>\SharedServices\SecurityFrameworkService\CassandraSetup\CassandraSetup.exe" uninstall
    This operation will also delete the Cassandra data folder on the local host.
    The Quorum consistency should also be achieved after node removal. Check the other nodes status before invoking CassandraSetup uninstall in order for it to work correctly.

    Uninstall will perform the following operations automatically:

    • Deletes keyspace secrets of local node from controlsuite_ddb.ddb_key_by_public_key
      • Deletes the local node record from controlsuite_ddb.ddbnode_by_host_id and controlsuite_ddb.ddbnode_by_datacenter_id
      • If needed it deletes the empty datacenter record from controlsuite_configuration.datacenter_by_id
      • Updates keyspaces replication factors and runs repair
      • Runs the nodetool.bat decommission
      • Stops the Cassandra service
    • Removes DDBService encryption certificate from the store
      • Deletes the Cassandra data directory
      • Deletes the Cassandra ssl directory
      • Deletes the following files from c:\<Install folder>\Shared Services\Cassandra\conf: cassandra.yaml, cassandra-rackdc.properties, logback.xml
      • Deletes %ProgramData%\Nuance\SecurityFrameworkService\Settings\ndiddbstore.xml
  7. Run "nodetool status" on the ‘supervisor’ node and verify that the live decommissioning node is no longer part of the Cassandra cluster. Once verified, continue with the next steps.
  8. Targeting to the remote live ‘supervisor’ node SSDSURL (https://<Hostname of the remote ‘supervisor’ node>:8181/SecurityFrameworkService), the locally installed services and client components need to be unenrolled using <Install folder>\Shared Services\SecurityFrameworkService\NDISecTool\NDISecTool.exe.
    • Point the SecurityFramework to a remote live node:
    $ NDISecTool.exe -ssdscert /ssdsurl https://<Hostname of the remote node>:8181/SecurityFrameworkService -tofu -o
    • Unenroll the local v1-auth, v1-service, v1-ddbmanagement service running 3 times NDISecTool command for each service in sequence:
    $ NDISecTool.exe -unenrollservice -n SecurityFrameworkService -servicename v1-auth (v1-service, v1-ddbmanagement)
  9. Delete the C:\ProgramData\Nuance\NDISecurity\common.xml on decommissioned node file.
  10. Reenable the "CassandraScheduledTaskRunner" task in Windows Task Scheduler on the ‘supervisor’ node. using Task Scheduler application or by running the following command:
    $ schtasks /Change /TN "CassandraScheduledTaskRunner" /Enable
  11. If you need to clean the decommissioned node completely, follow the uninstall/cleanup ControlSuite procedure described in different document(s). If the live node is removed from the cluster and needs to be pointed to a different datacenter, use Configuration Assistant.