The correct order and tests needed to verify that services and applications are working correctly after a power outage or server shutdown.  Boot up a tier at a time and complete test before proceeding to next tier.


Notes - IPs listed might not be accurate.  Reference server documentation for latest notes.



Tier Server Test / Notes IP Address
1 ESXi Nodes
Power on
Verify you can access each node via https://<serverip>
172.16.10.241
172.16.10.242
172.16.10.243
1
TrueNAS Power on
Verify you can access each node via https://<serverip>

1 vCenter Power on via vSphere Client
https://vcenter.royalelect.com/vsphere-client/?csp and verify you can log in or connect.
172.16.10.240
2
Domain Controller Power on via vSphere Client
Ping and verify all Automatic Services started
Double check internal and external name resolution is working
(if planning on bringing the Umbrella DNS servers up, leave DNS as is.  If not, set ethernet adapters DNS settings to point to itself and then on the DNS Service make sure the forwarders points to 208.67.222.222 and 208.67.220.220)
SAC-DC-01 or SAC-DC-03
2
Umbrella DNS
Power on via vSphere Client
Verify on the console everything is green.  LocalDNS might be yellow if only one domain controller was brought back online.
REC-UVA-01 or REC-UVA-02
2 INTERNET TEST At this point all networking - dhcp, dns, etc. should be up so test if you can access external sites via IP and DNS.
3 IT Jump Box Power on at the host
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
SAC-IT-01
3
AD-CONNECT
Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Try and run a Sync with Microsoft365
10.0.1.73
3 TEST
At this point we should have the capacity to process payroll and perform mass password resets if needed

4
SAC-DFS-02 Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Verify you can access the I drive
10.0.1.195
4
SAC-SQL-01 Power on at the host
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Connect via SQL Management Studio and verify all the databases are online
10.0.0.94
4
ACCUBID Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Open Accubid Classic and LiveCount on the server to verify the licensing service is working correctly
10.0.1.179
4
APPS Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Open Sharpe from \\accubid\sharpe\estimator.exe

4
HEAVYJOB Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Open HeavyJob Manager
Run A DIS Job
10.0.1.199
4
SAC-RDS-01 Power on via vSphere Client
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Navigate to https://sac-rds-01.royalelect.com/rdweb and try to run Accubid, Sharpe, and HeavyBid
10.0.0.189
4 TEST At this point we core line of business apps are up and running (if pulling from backups - sac-dfs-02 will be last do to size/time of restore)
5 WIP All servers listed below will be turned on not in any specific order
6 PRIMAVERA Power on via vSphere Client
Ping and verify all Automatic Services started
10.0.0.95
6 LMBIND (physical server) Power on at the host
Ping and verify all Automatic Services started
Verify all the OST Licenses are configured per https://royalelect.freshservice.com/solution/articles/4000018737-on-screen-takeoff-licensing-down
10.0.1.190
6 WASP Power on via vSphere Client
Ping and verify all Automatic Services started
Go to http://wasp.royalelect.com to login or verify you see a SilverLight screen
10.0.0.69
6 SQL2012 Power on at the host
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Connect via SQL Management Studio and verify all the databases are online
Test OnScreenTakeoff
172.16.10.10
6 SAC-PRX-01 Power on via vSphere Client
Ping and verify all Automatic Services started
172.16.10.64
6
DATA GATEWAY
Power on at the host
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Verify the On-premises data gateway service  has started
Go to powerbi.apps.com and try to sync a dataset that is connected to the Date Gateway
172.16.10.55
6 SOLARWINDS Power on at the host
Ping and verify all Automatic Services started
Run command nltest /sc_query:royalelect.com to test domain connectivity
Verify you can open https://solarwinds.royalelect.com:8787

172.16.10.30