Delete a dead node

To delete a dead node that you cannot access, do the following:

  1. Login to any of the other live nodes (referred as local node) and perform the following steps with administrative rights.
  2. Make sure that local Cassandra service is in Up and Normal status.

    Make sure the remote dead Cassandra node is not in "Up" status. It must be "down" or its server needs to be shut down.

    Note: The remote dead Cassandra node must not be started again after removing it.

    You can check the local and remote Cassandra node status by running the following command:

    $ "<Install folder>\Shared Services\Cassandra\bin\nodetool.bat" status

    Note: The JAVA_HOME environment variable must be set to "<Install folder>\Shared Services\Cassandra\JRE" before running "nodetool status"
  3. Run the following command:

    $ "<Install folder>\SharedServices\SecurityFrameworkService\CassandraSetup\CassandraSetup.exe" removenode -hn <Hostname of the remote node>

    This will perform the following operations automatically:
    • Runs nodetool removenode <HostId of the remote node>
    • Runs nodetool assassinate <IpAddress of the remote node>
    • Deletes keyspace secrets of remote node from controlsuite_ddb.ddb_key_by_public_key
    • Deletes remote node record from controlsuite_ddb.ddbnode_by_host_id and controlsuite_ddb.ddbnode_by_datacenter_id
    • If needed it deletes the empty datacenter from controlsuite_configuration.datacenter_by_id
    • Updates keyspaces replication factors and run repair
  4. Connect to the local node SSDSURL (https://<Hostname of the local node>:8181/SecurityFrameworkService). Before connecting, the installed services on the remote host need to be unenrolled using <Install folder>\Shared Services\SecurityFrameworkService\NDISecTool\NDISecTool.exe:
    1. Run the following command to point SecurityFramework to the local node:
      $ NDISecTool.exe -ssdscert /ssdsurl https://<Hostname of the local node>:8181/SecurityFrameworkService -tofu -o
    2. Run the following command to unenroll the remote v1-auth service:

      To find the exact ID of the service, you can display the service record by running:

      $ NDISecTool.exe -findservices -n SecurityFrameworkService -servicename v1-auth -endpointurl "https://<Hostname of the remote node>:8181/SecurityFrameworkService"

      Once the service ID is found, run the following command:

      $ NDISecTool.exe -unenrollservice -n SecurityFrameworkService -serviceid <the previously saved v1-auth service ID>
    3. Run the following command to unenroll the remote v1-service service:

      To find the exact ID of the service, you can display the service record by running:

      $ NDISecTool.exe -findservices -n SecurityFrameworkService -servicename v1-service -endpointurl "https://<Hostname of the remote node>:8181/SecurityFrameworkService"

      Once the service ID is found, run the following command:

      $ NDISecTool.exe -unenrollservice -n SecurityFrameworkService -serviceid <the previously saved v1-service service ID>

    4. Run the following command to unenroll the remote v1-ddbmanagement service:

      To find the exact ID of the service, you can display the service record by running:

      $ NDISecTool.exe -findservices -n SecurityFrameworkService -servicename v1-ddbmanagement -endpointurl "https://<Hostname of the remote node>:8181/SecurityFrameworkService"

      Once the service ID is found, run the following command:

      $ NDISecTool.exe -unenrollservice -n SecurityFrameworkService -serviceid <the previously saved v1-ddbmanagement service ID>

    5. Unenroll all the other services installed on the remote host in the way described above.
  5. The client components on the remote host cannot be unenrolled remotely. These components will remain in the database and will not cause any problem.