Be Careful Installing .NET Updates on Exchange Servers

Windows Update is now offering the the .NET Framework 4.5.2 update as an "Important" update to Windows computers.

• Microsoft .NET Framework 4.5.2 for Windows 8.1 and Windows Server 2012 R2 for x64-based Systems (KB2934520)
• Microsoft .NET Framework 4.5.2 for Windows Server 2008 R2 for x64-based Systems (KB2901983)

Both of these updates require a restart. Note that .NET Framework 4.5.2 is only supported and recommended for Exchange 2013 CU7+, but it is being offered as an Important update to all Windows servers. If your servers or patching processes use Windows Update you will still see these updates are being pushed to them. Personally, I have not experienced any issues with .NET Framework 4.5.2 installed on pre-CU7 Exchange servers.

Windows Update on Windows Server 2012 R2

Windows Update on Windows Server 2008 R2

When the .NET Framework update is installed on your Windows servers it will re-optimize all .NET assemblies on the server when it restarts. Perfmon shows ~99% of CPU resources are in use for about 15-20 minutes while this occurs.

98% CPU Utilization After Restart

.NET Runtime Optimization Service Racing

To be fair, this behavior happens with any .NET Framework update, not just this version. It also usually happens with Exchange CUs and security updates.

The main culprits are the mscorsvw.exe process (The .NET Runtime Optimization Service), TiWorker.exe process (Windows Modules Installer Worker), and Ngen.exe (Microsoft Common Language Runtime native compiler), as shown above. Exchange uses .NET assemblies extensively in its own code, so this optimization will affect the server's ability to function properly until this process completes. It will take significantly longer for the server to restart and system performance will be very very poor.

Once the the re-optimization process completes the Exchange server performance will eventually return to normal. This may take some time because other processes such as the IIS Worker processes and Exchange services were starved for resources and need to "warm up". In some cases I have seen Exchange services, such as the Microsoft Exchange Transport service, fail to start. Make sure all your services are running and performance returns to normal before moving on to patch the next server. I even suggest restarting the patched server one more time just to make sure it restarts normally and all services start properly first.

You should also be aware that if the Exchange server is load balanced using "least connections" the load balancer will probably drive all future connections to the server that is recompiling and those users will have a less than stellar experience. I recommend putting servers into maintenance mode on the load balancer prior to updating them and re-enabling them once optimization completes.

Tip: If you need to update .NET Framework on several servers it can take quite a bit of time for all this optimization. The mscorswv.exe process only uses one process thread by default. You can use the DrainNGenQueue.wsf script from the Microsoft .NET Framework Team to improve the performance of this process by allowing it to use multiple threads and up to 6 cores. For more information, see Wondering why mscorsvw.exe has high CPU usage? You can speed it up.