(also, Replacing Expired STS Certificates, Replacing Expired VMWare Service Certificates)

If your VMWare vCenter Server Appliance (VCSA) (at least versions 6.0, 6.5, 6.7, 7.0) web interface is suddenly telling you that “User name and password are required” no matter what credentials you fill out, then it’s quite likely your STS certificates are expired.

Check out the VMWare KB articles on checking and replacing these certificates, which, IMO – given they are self-signed in nearly every environment – should have always been replacing themselves since the invention of VCSA but for some reason, have not.

The TL;DR is this; (NB that these instructions are from the above linked articles, which at time of writing explicitly state they are for 6.0, 6.5, 6.7 and 7.0 – do not blindly attempt this stuff on newer VCSA versions)

1. SSH to your VCSA as root
   (you did make a note of that password, right?)
2. If you’re presented with the appliance shell (The prompt says “Command>“) rather than a bash prompt, type ‘shell‘ and press enter to get into bash.
   If you got a bash prompt right away, you can skip to step 4
3. In order to SCP scripts to your VCSA, you’ll need to change the shell to regular bash instead of the appliance shell;
   root@VCSA# chsh -s /bin/bash root
4. from your device, scp checksts.py to your VCSA’s tmp dir;
   you@YourPC$ scp checksts.py root@YOURVCSA:/tmp
5. on the VCSA, execute the checksts script to see if your STS certificates are expired;
   root@VCSA# python /tmp/checksts.py
6. If your STS certs are expired, SCP fixsts.sh from your device to your VCSA’s tmp dir;
   you@YourPC$ scp fixsts.sh root@YOURVCSA:/tmp
7. on the VCSA, execute the fixsts.sh script to generate new self-signed STS certificates and install them;
   root@VCSA# bash /tmp/fixsts.sh
   (you will be prompted for the password for administrator@YOURSSODOMAIN – NB that if you make a typo when entering it in this script you will need to re-run the script, backspace does not work here)
8. Restart the many services on your VCSA (each of these will likely take a while to complete, be patient);
   root@VCSA# service-control --stop --all
root@VCSA# service-control --start --all
   If you do NOT get errors about services failing to start, skip to step 12.
9. If you get errors about services not being able to start (timed out, crashed on start, etc), it is likely that your appliance / service certificates have also expired. You can run the following one-liner to output all of the certificates and their expiry dates to see if their expiry dates are in the past;
   root@VCSA# for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done
10. If you have expired service certificates, run the VCSA certificate manager;
    root@VCSA# /usr/lib/vmware-vmca/bin/certificate-manager
11. Choose option 4 or 8 – they both do the same thing, except in the event the services won’t start after replacing the certs, option 4 will roll back certificates afterwards (which will of course, under the current circumstances, likely still leave you with a broken state anyway).
12. If you had to chsh to /bin/bash and you now want to switch it back to the ‘factory supplied’ appliance shell;
    root@VCSA# chsh -s /bin/appliancesh root

You should now be able to access and log in to your VCSA as normal – NB that if you’ve pinned the old service / CA certificates anywhere you may need to remove/update those pins.