Slashdot Mirror
Ask Slashdot: Moving From *nix To Windows Automation?
Zubinix writes "I have a background in doing automation in a Unix/Linux environment using scripting languages such as perl and bash shell, as well as ssh for remote scripting. My next project will be in the Windows environment so what approach and methodology is best for developing, say, the automation required for a test system? I don't want to use things like Cygwin, as I need to integrate with Windows applications such as Exchange and Sharepoint. Is there a list of should and should not dos when it comes to Windows automation?"
Your answer to everything is WMI. Just start command prompt, type :> wmic And voila, you are ready
And why not? Powershell is perfect for the job.
So, been down this EXACT road. I was doing Linux automation, was so effective was brought in to do Windows too - yea!
Anyway, tried the cygwin route... It went OK, but just not quite it.
Went the vbscript route for a while (pure hell), but could work with wmi and had the windows objects available.
Soon, I started writing console apps in C# to make the trickier stuff happen. The .Net framework made it all so easy.
My final set of tools came to be powershell (access to the .net framework, you can do almost anything), Systernals psexec (for running processes on remote machines), and basic vbscript .bat. I had it set up with a web interface so I could enter a dos command into a web interface and point it a machine. It would build the bat and run it on the remote machine and return the standard out. This allowed me to add IIS sites and app pools, install com components, install apps, run msunit tests, and basically do whatever I wanted to any machine on the domain. Took me a quarter to build, but worked well. I've moved on in the company, but my replacement is still using it.
ymmv
"I am a one trick pony who cannot learn to do things in other operating systems because my ego gets in the way."
Go elsewhere to seek advice. I'd suggest stackoverflow.com
I'd also recommend having a Linux box; you can work in a familiar environment, then share out batch scripts you write via Samba -- read-only for binaries dirs that don't mind being unable to write out config files; writeable (but perhaps not listable) for a centralized location for saving off output from your various scripts, and so forth.
Use a decent automation engine, will save you years of headaches and maintenance. There are pros and cons though.
Ones such as BMC Bladelogic or HP Server Automation (opsware) are probably the best out there (enterprise-grade).
Be forewarned: they are not cheap. Best for enterprises 100+ systems. The more, the better ROI.
Don't get your hopes up with Sharepoint. This is quite possibly the single worst piece of crap ware I've encountered in the past 2 years.
Powershell. The only tool that knows how to talk to all the different frameworks in Windows is Powershell. No other tool can talk to .NET, COM, WMI, native APIs (via P/Invoke), and external stdio based tools. If you can't do the automation you want using something in one of the above frameworks, you've got bigger problems than finding a good automation tool.
Since the test guy usually has to be a part time sysadmin too, you should be aware of these tools:
System update readiness tool: http://support.microsoft.com/kb/947821/en-us
WMI diagnostic utility: http://www.microsoft.com/downloads/en/details.aspx?familyid=d7ba3cd6-18d1-4d05-b11e-4c64192ae97d&displaylang=en
gplogview: http://www.microsoft.com/downloads/en/confirmation.aspx?familyId=BCFB1955-CA1D-4F00-9CFF-6F541BAD4563
Windows SDK (including debugging tools for windows): http://www.microsoft.com/downloads/en/details.aspx?FamilyID=35AEDA01-421D-4BA5-B44B-543DC8C33A20
ollydbg: http://www.ollydbg.de/
sysinternals suite: http://technet.microsoft.com/en-us/sysinternals/bb842062
Windows Management Framework: http://support.microsoft.com/kb/968929
WDK: http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=36a2630f-5d56-43b5-b996-7633f2ec14ff
WAIK: http://www.microsoft.com/downloads/en/details.aspx?familyid=696DD665-9F76-4177-A811-39C26D3B3B34&displaylang=en
Windows 7 SP1 WAIK supplement: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=0AEE2B4B-494B-4ADC-B174-33BC62F02C5D
If XP is involved, check out Windows SteadyState. It's like deepfreeze, if you've ever used that. qemu is also a great way to boot test machines and capture output at scale; using CoW disks you can have fresh machines every time you boot regardless if the test machines are XP or not.
-- thalakan
Mouse click away you say? AutoIt
I prefer *nix too but I have a lot of Microsoft shops that I take care of. AutoIt has saved me a lot of time and effort.
Essentially, for newer versions of Exchange and SharePoint, Powershell is the only scripting option, and is excellent. For older versions, you don't have a lot of options, but you can probably call COM APIs using PowerShell as well, but the effort is a lot higher. The APIs exposed by Exchange (e.g.: MAPI) are hideous. SharePoint can be managed via direct SQL database queries from anything, with some care.
For Windows servers in general, stick to PowerShell. It's by far the best shell/scripting language by a wide margin. There's a learning curve, but it's absolutely worth it. To get set up to an equivalent level to Bash+SSH:
- Install PS2 + WinRM2 on pre-2008 R2 servers. It's not installed by default, and 2008 comes with PS1. There's a hotfix that updates PowerShell and all associated components to v2 on XP, 2003, Vista, and 2008. .ps1 script files from running by default is a shit workaround for not having an execute bit, and is totally useless. Just turn it off.
- Enable unsigned scripts to run. Preventing
- Enable WinRM. This is the equivalent of SSH, but has to be enabled on both servers (which is obvious), and the clients (for no discernible reason).
- Enable CredSSP. This improves WinRM by allowing delegated credentials to be used remotely. Unlike SSH, WinRM is an RPC protocol, and due to limitations of Windows authentication, the remote server cannot use network resources with the client's credentials (one hop security). The workaround is credentials delegation. This requires a server-side setting, and two client-side settings.
- Install plugins. There are about a dozen that ship with Windows 2008 R2, a plugin for Exchange 2007 and 2010, and a plugin for Sharepoint. To set up the AD plugin on pre-2008 R2 domains, there's a module that has to be deployed, it's basically a web service that the PS plugin uses. The SQL plugin is an optional download that is particularly handy for SharePoint.
That seems like a lot, but can be done in seconds from the command-line, and even better, can be done completely automatically using Group Policy. For new server deployments, I ask for my servers to be created under an AD container, and I just set those options up in a common policy configured on that container, and then everything just works.
The equivalent of a remote SSH session in: Enter-PSSession 'computername'
However, it gets better: You can create and store sessions in variables by creating them using New-PSSession. You can pipe objects into remote sessions and invoke commands on them, or arrays of them. In other words, a single line of code lets you invoke commands in bulk across farms of servers.
For example, as a Citrix XenApp admin, I often have to update machine policy across a farm. This is just two lines with PowerShell:
$xaservers = Get-XAServer | Select -ExpandProperty ServerName
Invoke-Command -CN $xaservers { gpupdate }
That will query the list of XenApp servers, extract the server names into an array, and invoke "gpupdate" in parallel across all of them using a temporary secure shell session.
There might be equivalent capabilities on Linux, but nothing on the Windows platform comes close to what you can do with PowerShell.
All Windows automation that you might want to do is exposed through a COM interface and there is no programming language targeting the Windows platform that doesn't include support for COM
What about languages that aren't so much targeting Windows but just barely support it?
On a personal note, when decided to switch from command scripts to a programming language, I was afraid that it would be hard. But it turned out that most things are actually easier (if at times a bit more tedious) and it's a lot harder to shoot yourself in the foot in most of them. At the very least you'll never get errors caused by the unpredictable script syntax again.
BASH is basically turing complete. Also, "[...]at times more tedious[...]" is the issue. I do 95% of my work in a Windows environment and a lot of times I long for the simplicity of *nix. Most of the time in Windows I feel like I'm fighting a huge and complex system, trying desperately to make it behave in a sane and rational way in order to produce the desired output. In the *nix world at least it's just my code I'm battling with, not the entire OS and ecosystem. But hey, maybe some people actually like a world where all systems communicate using XML and where character encoding is "enicode" yet in practice can be pretty much anything as long as the marketing division can claim it's all unicode because technically the operating system uses unicode internally...
Greylisting is to SMTP as NAT is to IPv4
You bring up a good point, but I'm going to address this to the OP.
As previously noted by the other commenter, Powershell is useful if your environment is full of Vista computers or newer or if you have more control over the environment so Powershell can be installed (there is an XP installer). It's major draw is the usefulness to systems admins that want to do real-time execution.
Otherwise, you're looking at VBScript or batch scripts. Neither are nowhere near as elegant or smooth as some other languages, but tapping WMI and other Windows objects is actually really simple using VBScript. There are loads of examples here. Also, a Google search in the form of "vbscript %thing%" will usually get you pointed in the right direction.
Do yourself a huge favor, though - get a decent editor. While Windows has a simple notepad app, there is no context highlighting, in-line completion, or other helpful tools for looking at script code. Personally, I prefer PrimalScript because it's useful for other things like SQL, C, Perl, etc (including PowerShell). It also has a built-in debugging engine. However, that app can be expensive. One good free alternative is Notepad++ with the appropriate plug-ins for the files you want to create/edit. There are plenty of others out there, so grab whatever you feel works best for you.
If you're planning on using a database and are familiar with MySQL, you should feel nearly at home with Microsoft's version of SQL. There are some minor syntax differences (for example, the update query is slightly different, if my memory is correct) but it shouldn't take long to get used to them.
Lastly, ideally, you're going to want to an account that is a local administrator on all of the systems you will be touching. Yes, you can do some serious damage with such an account and is typically not the way of things in the *nix world, but many aspects of a Windows system are inaccessible unless you have an Administrator level account - especially remote access to certain things like the registry and WMI. If getting a local administrator account on all the systems you're responsible for isn't an option, have someone that does have one (or a domain admin account) on speed-dial in case your scripts fail due to permissions errors.
Good luck!
My sources are unreliable, but their information is fascinating. -- Ashleigh Brilliant
This is false. Powershell is available as an addon for Win XP and Win 2k3. The poster above said preinstalled fo a reason. Any decent Window environment should have a software distribution system which makes the update for PS 2.0 a non issue.
I can do the same thing in 3 lines of code using only SSH and virsh in our KVM environment (1 line to set sh as shell, 1 to run SSH and use virsh to clone / create a VM, and 1 to start it).
And I can do that with 1 line of PowerShell code (PowerShell has a consistent way to short parameters, use positional parameters and pipe full objects like virtual machines):
new-vm ESX01 -name NewVM_01 -templ TemplateName | start-vm
Your point? The OP was asking what to use to set up and administer test systems in a *Windows* environment. PowerShell is a valid option for that.
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*
How exactly do you see Windows updates for all packages again? Oh, right, it's not a feature. Yawn.
Ignorance can be cured by Google (or bing). It took me 30 seconds to find out that Windows Update does indeed have a COM API which can be readily used from PowerShell. Here a way to list *all* available packages from Microsoft Update:
PS C:\Windows\system32> $mus = new-object -com "Microsoft.Update.Session"
PS C:\Windows\system32> $updates = $mus.CreateUpdateSearcher().Search("") | select -exp updates
PS C:\Windows\system32> $updates|select title
Title
-----
Latvian Language Pack - Windows 7 Service Pack 1 (KB2483139)
Slovenian Language Pack - Windows 7 Service Pack 1 (KB2483139)
( other language packs )
Definition Update for Microsoft Security Essentials - KB2310138 (Definition 1.103.1197.0)
Note how the list was produced by selecting just the title. The updates returned by the update searcher are actually objects which not only expose a lot of extra information such as disk/cpu/memory capacity recommendations, severity indicators from the security response team, knowledge base article references, license text etc, but also allows you to directly proceed to install by piping them.
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*