Configuration options for Remote PowerShell and WS-Management

Here’s the want list:
• to be able to run WCF and workflow services in IIS that use a basicHttpBinding.
• to scale out services in an application farm using the network load balancing service in Windows Server 2008.
• to authenticate users using Kerberos to flow the Windows Identity.
• to administer servers remotely using PowerShell.

It’s not exactly an exotic or out there set of needs, however it has been over three weeks now that I’ve been working through various attempts to get this up are running reliably.

The crux of the issue is around the use of HTTP and kerberos. To get the services to work in a load balanced environment with kerberos, a set of SPNs needed to be added to the Active Directory for the domain.The web applications hosting the service needed to run under a domain identity (e.g. MyDomain\service.expert) so they are mapped to an application pool with this identity. SPNs are then added to map the HTTP protocol to this user, rather than the machine account. In our case, four SPNs are added to the service.expert user – one for the network load balancers virtual host name and one for each server in the application farm:

HTTP/SVNLB301.ap.aderant.com
HTTP/SVEXPGG302.ap.aderant.com
HTTP/SVEXPGG303.ap.aderant.com
HTTP/SVEXPGG304.ap.aderant.com

Doing this breaks the default WinRM service configuration as the WinRM HTTP listener is running under a machine account not service.expert and so the SPN is incorrect and Kerberos negotiation fails. This is pretty much where we left off on the last posting and since then I have been looking at using HTTPS as the transport for the PowerShell remoting calls and other authentication mechanisms.

There are two options for hosting the WinRM service:

1. as a Windows Service (this is the default)
2. in IIS using a WinRM v2 features called ‘WinRM IIS Extensions’. This is an optional install in Windows Server 2008 to support the ‘fan-in’ model for PowerShell remoting which is targeted at the cloud.

Hosting the WinRM service using HTTPS is meant to be simple so long as you have an appropriate certificate installed on the server for SSL. The command is:

> winrm quickconfig -transport:HTTPS

I have never been able to get this to work. Before explaining how I did get a WinRM HTTPS endpoint working, let’s cover off the certificate.

Windows Server 2008 has a role which allows a server to act as a certificate authority (CA) for a domain. This role includes a self-service website from which any machine on the domain can request a certificate. I used this to request certificates created using the web server template with the common name (CN) set to the fully qualified domain name of the server in my application farm. The self-service website is pretty straight forward but note that the certificated is installed in the current user path, not the local machine so so you need to move it. The easiest way to see this is to use the certificate provider within PowerShell:

> cd cert:\CurrentUser\My
> ls
> cd cert:\LocalMachine\My
> ls

This will show you all of the certificates installed in the current user\my and the local machine\my stores. You can also use the management console (MMC) and add in the certificate plug-in for both the current user and local computer.

The WSMAN provider allows you to configure the WinRM service from within Powershell.

> cd WSMAN:\localhost\Listener
> new-item . -Address * -Transport HTTPS -CertificateThumbprint “XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX”

You need the 40 character certificate thumbprint which can be easily found by listing the certificates in cert:\LocalMachine\My. With the real thumbprint replacing the Xs, the above command will create an HTTPS listener that is hosted in the WinRM service.

To connect to the machine from a remote client, using kerberos to authenticate as the current user:

> icm -ComputerName targetServer -UseSSL -Authentication NegotiateWithImplicitCredential -ScriptBlock {get-host}

The script block is executed on the remote machine. If a test certificate has been used to set-up the HTTPS channel, then the remote call will fail. The certificate must have been issued by the domain CA, the CN must match the machine name and the revocation list is checked. It is possible to switch off these checks by adding the following parameter to the call:

> icm ... -SessionOption (new-PSSessionOption -SkipCNCheck -SkipCACheck - SkipRevocationCheck)

Any combination of the three skips can be used.

This again proved somewhat unreliable for me, due to the use of Kerberos over HTTPS to authorize the user. There are other authentication options available such as basic, which is secure over an HTTPS channel since the channel is encrypted.

The change in identity of the HTTP SPN just seemed to keep tripping me up, which made me wonder why not host the management service in IIS and then set it to run in an application pool with the same identity as our other services? Finding out how to do this took me some time and led me to the fan-in model for PowerShell mentioned earlier.

Fan-In Model
Within WinRM v2 there comes a plug-in model to allow ISVs to supply a module that allows their software to be managed via WS-Management. The PowerShell team ships such a module pwrshplugin.dll which can be found in %windir%\system32. To be able to host such a module in IIS, you need to ensure that you have the WinRM IIS Extensions option installed, I have only seen it available on Windows Server 2008 and not Windows 7.

[ On Windows Server 2008 R2, you can use the ServerManager module to check the installed features:

> Import-Module ServerManager
> Get-WindowsFeatures
]

With this option enabled, you can create a new web application and drop in a web.config file similar to the following which is discussed here:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <system.management.wsmanagement.config>
      <PluginModules>
        <OperationsPlugins>
          <Plugin Name="PowerShellplugin" Filename="%windir%\system32\pwrshplugin.dll" SDKVersion="1" XmlRenderingType="text">
           <InitializationParameters>
                <Param Name="PSVersion" Value="2.0" />
            </InitializationParameters>
            <Resources>
                <Resource ResourceUri="http://schemas.microsoft.com/powershell/Microsoft.PowerShell" SupportsOptions="true">
                    <Capability Type="Shell" />
                </Resource>
            </Resources>
          </Plugin>
        </OperationsPlugins>
      </PluginModules>
    </system.management.wsmanagement.config>
        <security>
            <access sslFlags="Ssl" />
            <authentication>
                <anonymousAuthentication enabled="false" />
                <basicAuthentication enabled="true" />
                <windowsAuthentication enabled="true" />
            </authentication>
        </security>
        <modules>
            <add name="WSMan" />
        </modules>
  </system.webServer>
</configuration>

The web application is configured to use SSL and Basic or Windows authentication is accepted. You might need to edit your applicationhost.config file to unlock the section of the section. The web application can be mapped to an application pool that has the same identity as the other services, in our case MyDomain\service.expert so the SPNs should work.

[Note: do not set-up an HTTPS listener in both IIS and WinRM at the same time on the same certificate, if you do recycling the app pool will drop the HTTPS binding from IIS – the Windows Service WinRM gets precedence.]

To connect to the machine from a remote client (using basic authentication), the following is required:

> $secpasswd = ConvertTo-SecureString "myPassword" -AsPlainText -Force
> $mycreds = New-Object System.Management.Automation.PSCredential ("MyDomain\MyUsername", $secpasswd)
> icm -ConnectionUri https://svexpgg303.ap.aderant.com/Powershell -Authentication Basic -Credential $mycreds -ScriptBlock {get-host}

The password is captured in a secure string and then a new PSCredential object is created to contain the username and password. This is passed to the invoke-command cmdlet using the -Credential parameter. Note that we are also using the -ConnectionUri parameter.

UPDATE [2nd October 2010]: I finally got to the bottom of the 1300 error I saw in the Windows Remote Management event log thanks to this post: http://blogs.msdn.com/b/wmi/archive/2010/02/25/winrm-hosted-in-iis-fails-to-start-with-error-1300-in-event-log.aspx

The account that the application pool is using must have the ‘Generate security audits’ right granted. Also when testing, it is important to reset IIS after each change to ensure that you are running against the correct set-up.

Retesting with security set-up correctly proved that any app pool can be used and the web application path could contain subfolders.

Having managed to establish a secure connection for remote PowerShell via IIS using basic auth and HTTPS, I’ve pretty much given up on getting it to work over Kerberos. I might try just once more to do Kerberos over HTTP when the management service is hosted in IIS but I’ve already been fighting with this for way too long. I hope the above saves someone the pain I went through…