In order to remove specific hosts from Altaro VM Backup that are added with the vCenter installation, you can simply click on [Edit] under one of the hosts:

Then you will be taken to a vCenter details screen, showing you the hosts with the hosts within the environment too. At this point you can click on the [Remove Host] button: