Res software is probably best know for their PowerFuse product which provides powerful and granular control of a system and a user’s environment. They also have a terrific product called Wisdom which they describe as “Run Book Automation for Windows.” We utilize Wisdom every day to manage our XenApp farm and related servers. I could spend pages and pages gushing about Wisdom, but I’m going to use this post to show how I use Wisdom to accomplish certain tasks. Naturally, the solution I present can be accomplished in a variety of ways, but I find Wisdom to be elegant and have a very short learning curve. On top of that, it provides extensive,detailed change management and reporting which many products to not.
Typically, a XenApp environment has many, many variables at work that can compromise the stability of a system. At times, this results in a crash dump and reviewing these dumps can give insight to what caused the problem.
NOTE: If you have a affinity for punishing yourself and want to actually dive into dumps, I highly recommend Crash Dump Analysis by Dmitry Vostokov. Check out his minidump analysis series to get started.
- Minidump Analysis (Part 1)
- Minidump Analysis (Part 2)
- Minidump Analysis (Part 3)
- Minidump Analysis (Part 4)
In our environment, we have many servers and occasionally one will crash, reboot, and come back into production before we get an e-mail alert and we would not know if a dump was generated unless we connected to the server to find it. I will describe how I created a Wisdom module to detect dump files on a server, copy them to a central location, and send an e-mail alert to the team.
Step 1 – Determine if a dump file exists
You can set up your server to create full and mini dumps by going into Computer properties, clicking the Advanced tab, and selecting Startup and Recovery.
This window will show you where the dump files are being created. Full memory dumps are written to: %SystemRoot%\MEMORY.DMP and minidumps are written to: %SystemRoot%\Minidump\
Here is the Wisdom Module that we’ll dive into:
The Execute Command task:
The command line is: if not exist %WINDIR%\Minidump\*.dmp EXIT /B 1
This is a conditional statement that looks for any file with a *.DMP extension in the Minidump directory. If the file exists, then the command will successfully end with an error code of 0 (as set by Wisdom) otherwise it will fail and exit with an error code of 1.
Step 2 – E-mail someone that a memory dump file was found
The send e-mail task
I’m highlighting the Condition portion of this task because this is where the conditional logic of the previous task (and its exit codes comes into play). The condition is whether the previous task was successfully completed. If so, the we set a DUMPEXISTS parameter to 1 if true or 0 if false.
Step 3 – Copy the memory dump to a network share and off the server
The perform file operations task
The task creates a directory on a share (based on the server name – more on this later), copies the memory dump file to that location and then deletes it from the server. The condition on this task is the value of DUMPEXISTS, which we set in the previous task. If the e-mail task ran, then DUMPEXISTS is set to one, so this task will run and move the dump to a network share.
The remaining 3 tasks for this module repeat the previous 3 for the other memory dump type.
The SERVERNAME parameter (which is used when we copy the memory dump to the network) is simply formed by the %COMPUTERNAME% variable. Wisdom, luckily, has access to the environment variables that are set on the machine the task is run on.
Finally, you should set this module to run on every reboot, then you’ll get e-mail alerts that memory dumps were generated if a server crashes.
Wisdom allows you to export any resource, module, project or run book as an XML file that can be imorted to another Wisdom database. Another wonderful feature. I’ve sanitized a building block of this module for the community. Due to WordPress file extension restrictions, I’ve renamed the building block with a .DOC extension. Change it to .XML and you should be able to import it.