PowerShell: Cleanup Reboot script

cleanup-reboot

Intro

If your environment leverages Provisioning Services and is fairly large, you may run into issues with servers coming up but not allowing any users to login. The fix we implemented is to reboot those servers after it’s clear that they will not allow users to login (usually sometime mid-morning when most of our users have logged in). This was  a manual process and I wanted to find a way to automate this until we could change our reboot schedules or find a resolution. In this post we’ll explore using a script to catch these unresponsive servers.

Scenario

Your reboot script ran last night and you find that the next day there are a handful of servers that report to the farm, but do not have any users attached. You’ve checked the obvious problems/issues and the event log isn’t any help. The only solution seems to be another reboot. Once the system comes up, users are able to connect.

In order to automate the reboots, we have to determine the criteria that identifies a server as having a problem and not just an idle server.

  1. It is mid-morning and most of our users are logged in and using Citrix. The server has no users connected even though the load evaluators and login behavior should allow users to connect.
  2. The server is marked as ‘Online’ when using the PowerShell Cmdlet get-xaserver.
  3. The server is a not in our excluded list (more on this below)
  4. The server has a high load even though it is hosting no users.

I got one two words for you…Worker Groups

Param(
[parameter(Position = 0, Mandatory=$False )]
[ValidateNotNullOrEmpty()]
$DeliveryControllers="YOURDDC.DOMAIN.LOCAL" # Change to hardcode a default value for your Delivery Controller. Can be a list separated by commas
)
#Constants
$PSSnapins = ("Citrix*")
$WORKERGROUPS = "Zone Data Collectors,Productivity Apps" #Define which worker groups should be processed. Comma seperated list, spaces acceptable, case insensitive (for example "Zone Data Collectors,Productivity Apps"). Leaving blank will process all servers in the farm as in previous revisions
$EXCLUDESERVERS = "CORPCTX01,CORPCTX02,CORPCTX05" #Define which servers should be excluded from processing. Comma seperated list, short names only, case insensitive (for example "CORPCTX01,CORPCTX02,CORPCTX05")

view raw
cleanup-reboota.ps1
hosted with ❤ by GitHub

You will have to edit the $WORKERGROUPS variable to list the Citrix Worker Groups (WG) you wish to check for zero-user servers. The list can be separated by commas. Optionally you can also edit $EXCLUDESERVERS to have the script ignore servers you do not want checked for zero-users.

The script will iterate though the servers in each WG. This is done with 2 nested foreach loops.

foreach ($workergroup in $workergroups){ # Iterate through workergroups
$checkworkergroup = @(get-xaworkergroup ComputerName $DeliveryController | where-object {$_.WorkerGroupName -eq $workergroup})
if ($checkworkergroup.count -eq 0){
$finalout += "$workergroup is invalid. Confirm names in worker group list and try again.<br>"
write-verbose "$workergroup is invalid. Confirm names in worker group list and try again."
} else {
$workergroupservers = @(get-xaworkergroupserver ComputerName $DeliveryController workergroupname $workergroup | sort-object property ServerName) # Create a query to pull the Worker Group membership
$finalout += "Checking servers in Worker Group: $WORKERGROUP<br>"
write-verbose "Checking servers in Worker Group: $WORKERGROUP"
foreach ($workergroupserver in $workergroupservers){ # Iterate through workergroup servers
$server = $workergroupserver.ServerName
if (($excludedservers -notcontains $server) -and ($OnlineServers -contains $server)) { # Check that server is not excluded and is online
if ("$server" -eq "$env:COMPUTERNAME") { # Bypass local server
} else {
$sessions = $xasessions | Where {$_.ServerName -eq $server} # Create a query against server passed through as first variable where protocol is Ica. Disregard listening sessions
if ($sessions.count -eq 0) { #Server has no users.
$wgServerLoad = Get-XAServerLoad computername $DeliveryController servername $server #Check server load
if ($wgServerLoad.Load -ge 3500) {
set-XAServerLogOnMode ServerName $server LogOnMode ProhibitNewLogOnsUntilRestart #Set ProhibitNewLogOnsUntilRestart
$finalout += "$server is online, but is hosting no users with load of $wgServerLoad. LogOnMode set to ProhibitNewLogOnsUntilRestart.<br>"
write-verbose "$server is online, but is hosting no users with load of $wgServerLoad. LogOnMode set to ProhibitNewLogOnsUntilRestart."
}
}
}
}
}
}
}

view raw
cleanup-rebootb.ps1
hosted with ❤ by GitHub

The script confirms the Worker Group is valid and then iterates through the servers in the WG. It checks that the server is considered Online and not a member of the $EXCLUDESERVERS list. It then confirms the server has no users connected and then checks to see if the load is greater than or equal to 3500. In our farm, this was a common characteristic for these zero-user servers. You may need to change this to better reflect your environment. I setup a scheduled task to run this script every day around 10Am and it has automated an administrative task freeing me up to write blogs about PowerShell.

NOTE that this script does not actually reboot the server. We have another script that runs all day looking for servers that are set to ProhibitNewLogOnsUntilRestart. This secondary script checks that no users are logged into a server set to ProhibitNewLogOnsUntilRestart and then reboots it. You could edit the script above to add this.

What about reporting?

### Un-comment lines below to allow of an emailed report ###
#Assign e-mail(s) to $sendto variable and SMTP server to $SMTPsrv
#$sendto = "first.last@domain.com#Can add additional emails separated by commas
#$from = "" #Set appropriate from address
#$SMTPsrv = "" #Set appropriate SMTP server
#$title = "Online servers with zero users"
#foreach ($email in $sendto) {
# $smtpTo = $email
# $smtpServer = $SMTPsrv
# $smtpFrom = $from
# $messageSubject = "Report: Online servers with zero users"
# $date = get-date -UFormat "%d.%m.%y – %H.%M.%S"
# $relayServer = (test-connection $smtpServer -count 1).IPV4Address.tostring()
# $message = New-Object System.Net.Mail.MailMessage $smtpfrom, $smtpto
# $message.Subject = $messageSubject
# $message.IsBodyHTML = $true
# $message.Body = $finalout
# $smtp = New-Object Net.Mail.SmtpClient($smtpServer)
# $smtp.Send($message)
#}

view raw
cleanup-rebootc.ps1
hosted with ❤ by GitHub

If you wish to receive an email report of which servers were set to ProhibitNewLogOnsUntilRestart uncomment the above lines in the script. You will have to provide an SMTP server and a single or list of email addresses. Here’s an example of the email report:cleanup-reboot

The Script

You can download the script from Github.

Thanks for reading,
Alain

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.