Delete a live node manually
To delete a live node that you have access to, do the following:
-
Login to the live node and perform the following steps with administrative
rights. This local node must be live, and the Cassandra service must be in Up
and Normal status.
You can check the local Cassandra node status by running the following command:$ "<Install folder>\Shared Services\Cassandra\bin\nodetool.bat" statusThe JAVA_HOME environment variable must be set to "<Install folder>\Shared Services\Cassandra\JRE" before running "nodetool status".
-
Choose the SFS/Cassandra helper ‘supervisor’ node where monitoring and
verifying live node removal and unregistering SFS base services are located.
Temporarily disable "CassandraScheduledTaskRunner" in Windows Task
Scheduler on the ‘supervisor’ node to avoid any unexpected Cassandra maintenance
operations during live node removal.
The scheduled task can be disabled through the Task Scheduler application or by running the following command:$ schtasks /Change /TN "CassandraScheduledTaskRunner" /Disable
- Continue with the live node removal steps only if "nodetool status" shows all nodes are 'Up' in the Cassandra cluster. If there are any dead nodes, remove them first. (see Delete a dead node manually). If any node are 'Down' due to a cluster communication issues, fix them first.
-
Remove the CassandraScheduledTaskRunner scheduled task
from the Windows Task Scheduler on the live decommissioning node in order to
avoid any unexpected Cassandra access or operation during the node
removal.
The scheduled task can be removed through the Task Scheduler application or by running the following command:$ schtasks /Delete /TN "CassandraScheduledTaskRunner" /F
- Check if the “CassandraScheduledTaskRunner.exe” process is running by using Task Manager and wait for it to complete.
-
Run the following command on the live decommissioning node:
$ "<Install folder>\SharedServices\SecurityFrameworkService\CassandraSetup\CassandraSetup.exe" uninstallThis operation will also delete the Cassandra data folder on the local host.The Quorum consistency should also be achieved after node removal. Check the other nodes status before invoking CassandraSetup uninstall in order for it to work correctly.
Uninstall will perform the following operations automatically:
- Deletes keyspace secrets of local node from
controlsuite_ddb.ddb_key_by_public_key
- Deletes the local node record from controlsuite_ddb.ddbnode_by_host_id and controlsuite_ddb.ddbnode_by_datacenter_id
- If needed it deletes the empty datacenter record from controlsuite_configuration.datacenter_by_id
- Updates keyspaces replication factors and runs repair
- Runs the nodetool.bat decommission
- Stops the Cassandra service
- Removes DDBService encryption certificate from the store
- Deletes the Cassandra data directory
- Deletes the Cassandra ssl directory
- Deletes the following files from c:\<Install folder>\Shared Services\Cassandra\conf: cassandra.yaml, cassandra-rackdc.properties, logback.xml
- Deletes %ProgramData%\Nuance\SecurityFrameworkService\Settings\ndiddbstore.xml
- Deletes keyspace secrets of local node from
controlsuite_ddb.ddb_key_by_public_key
- Run "nodetool status" on the ‘supervisor’ node and verify that the live decommissioning node is no longer part of the Cassandra cluster. Once verified, continue with the next steps.
-
Targeting to the remote live ‘supervisor’ node SSDSURL (https://<Hostname of
the remote ‘supervisor’ node>:8181/SecurityFrameworkService), the locally
installed services and client components need to be unenrolled using
<Install folder>\Shared
Services\SecurityFrameworkService\NDISecTool\NDISecTool.exe.
- Point the SecurityFramework to a remote live node:
$ NDISecTool.exe -ssdscert /ssdsurl https://<Hostname of the remote node>:8181/SecurityFrameworkService -tofu -o- Unenroll the local v1-auth, v1-service, v1-ddbmanagement service running 3 times NDISecTool command for each service in sequence:
$ NDISecTool.exe -unenrollservice -n SecurityFrameworkService -servicename v1-auth (v1-service, v1-ddbmanagement) - Delete the C:\ProgramData\Nuance\NDISecurity\common.xml on decommissioned node file.
-
Reenable the "CassandraScheduledTaskRunner" task in Windows Task
Scheduler on the ‘supervisor’ node. using Task Scheduler application or by
running the following command:
$ schtasks /Change /TN "CassandraScheduledTaskRunner" /Enable
- If you need to clean the decommissioned node completely, follow the uninstall/cleanup ControlSuite procedure described in different document(s). If the live node is removed from the cluster and needs to be pointed to a different datacenter, use Configuration Assistant.