Windows Server Reboot Loop After Installing January 2022 Security Updates

It seems all my blog posts are about Microsoft update failures lately: (

I've seen several reports of Windows Server 2012 R2, 2019, and 2022 getting stuck in a reboot loop after installing the January Windows Updates. Specifically, these updates:

• KB5009624 for Windows Server 2012 R2
• KB5009557 for Windows Server 2019
• KB5009555 for Windows Server 2022

Microsoft is currently aware of the issue.

To fix the issue, restart the computer in Safe Mode which will allow you to login and remove the offending update from Windows Update. You can normally get into Safe Mode by pressing F8 immediately after the server starts.

Domain Controllers are a little more tricky, since there isn't a local user account to login with. For DCs you should restart in Safe Mode with Networking. This will allow you to login with a Domain Admin account.

To remove the update from the command line, run the the appropriate command for your operating system:

Windows Server 2012 R2:

wusa /uninstall /kb:5009624

Windows Server 2019:

wusa /uninstall /kb:5009557

Windows Server 2022:

wusa /uninstall /kb:5009555

I found that if the server is configured to automatically download and install updates it will reinstall the errant update all over again. Grrrr. To prevent this, you can hide the update from reinstalling.

• Uninstall the update and then run run Check for Updates from Windows Update in the server.
• Right-click the update and select Hide Update to prevent it from being reinstalled.

I, for one, am really getting tired of poor quality of updates coming from Microsoft these days. There's simply no excuse for this.

UPDATE - January 17, 2022

Microsoft is releasing Out-of-band (OOB) updates today, January 17, 2022, for some versions of Windows. This update addresses issues related to VPN connectivity, Windows Server Domain Controllers restarting, Virtual Machines start failures, and ReFS-formatted removable media failing to mount. All updates are available on the Microsoft Update Catalog, and some are also available on Windows Update as an optional update. Check the release notes for your version of Windows for more information. Updates for the following Windows versions are available on Windows Update as an optional update. For instructions, see the KB for your OS listed below: ·         Windows 11, version 21H1 (original release): KB5010795 ·         Windows Server 2022: KB5010796 ·         Windows 10, version 21H2: KB5010793 ·         Windows 10, version 21H1: KB5010793 ·         Windows 10, version 20H2, Windows Server, version 20H2: KB5010793 ·         Windows 10, version 20H1, Windows Server, version 20H1: KB5010793 ·         Windows 10, version 1909, Windows Server, version 1909: KB5010792 ·         Windows 10, version 1607, Windows Server 2016: KB5010790 ·         Windows 10, version 1507: KB5010789 ·         Windows 7 SP1: KB5010798 ·         Windows Server 2008 SP2: KB5010799 Updates for the following Windows versions are available only on Microsoft Update Catalog. For instructions, see the KB for your OS listed below: ·         Windows 8.1, Windows Server 2012 R2: KB5010794 ·         Windows Server 2012: KB5010797

 

Read more ...

AAD Connect 2.0.88.0 breaks Shared Mailboxes for Exchange hybrid customers

Microsoft released Azure AD Connect version 2.0.88.0 as a download-only version on 12/16/2021. This update includes several bug fixes and introduces support for syncing AD objects from a single forest to multiple tenants.

However, this version contains a new potentially devastating bug that removes disabled user accounts in AD from Azure AD. 

Because shared mailboxes use disabled user accounts this means those mail users are also deleted from Exchange Online. Cloud users will no longer see on-prem shared mailboxes in the GAL or be able to access them. Inbound mail flow will also be affected for these mailboxes since they no longer exist from an Exchange Online Protection perspective.

Luckily, version 2.0.88.0 is not being pushed as an auto upgrade version, so only customers who download and install it are affected.

AAD Connect version 2.0.89.0 has been released. If you are affected by this bug, you should update to the latest version. See my other updates below.

The workaround is to remove AAD Connect 2.0.88.0 and reinstall the previous AAD Connect version 2.0.28.0. Since Microsoft removes all but the most current version, I've made AAD Connect 2.0.28.0 available on my blog here.

I recommend exporting your current AAD Connect configuration first, then importing it when installing version 2.0.28.0. Be sure to uncheck the "Enable staging mode" when completing the installation. 

During the first sync you will see that the disabled accounts in AD are being synced again to Azure AD.

Update #1 - Dec 22, 2021

The Azure AD Connect Version History website was updated yesterday after my blog post to say that version 2.0.89.0 has been released which addresses this issue. However, at this time only 2.0.88.0 is still available from the AAD Connect download website.

Update #2 - Dec 22, 2021

AAD Connect version 2.0.89.0 is now available for download. Strangely, this new version is no longer listed in the version history. :-/ 

I  have confirmed that the bug has been squashed.

Read more ...

November 2021 Windows Security Updates break OWA published with Azure App Proxy

If you use Azure App Proxy to publish Outlook Web App (OWA) your may find that it suddenly stopped working. This is due to a bug in recent Windows security updates that affects Kerberos delegation.

Microsoft quietly announced this in the Microsoft 365 Message Center as announcement #2750 - Take action: Out-of-band update to address authentication issues on DCs relating to Kerberos delegation scenarios.

There are separate out-of-band updates for all versions of Windows Server from Windows Server 2008 SP2 through Windows Server 2019. Make sure to download the correct update for your version of Windows Server.

At a minimum, your should apply these updates to all the Domain Controllers that reside in the same AD site as your Exchange Servers. The OOB update requires a restart of the DCs where it is applied.

Once installed, OWA published through AAD App Proxy will start working again.

Publishing OWA through Azure App Proxy allows your organization to use Conditional Access and MFA for OWA access. If you would like help with this for your organization, please contact EXPTA Consulting.

Read more ...

Azure AD Connect V1.X versions no longer support the V2 endpoint

Microsoft introduced the Azure AD Connect sync V2 endpoint with version 1.6.4.0 in March 2021. Among the improvements, the V2 endpoint includes performance improvements and allows for synchronization of groups with up to 250K members. Enterprise customers with groups of 50K or more were encouraged to move to the new V2 endpoint.

AAD Connect version 2.0.3.0 was released in July 2021 and was a major upgrade. It supports the V2 endpoint by default, but requires Windows Server 2016 or 2019 due to it's dependency on SQL Server Express 2019 for localDB. There are still many customers running AADC V1.x for this reason.

Today, Microsoft updated the AADC version history to say that the V2 endpoint is no longer available for V1.x versions. 

UPDATE - 11/10/2021: Microsoft just added the following information to the AAD Connect version history:

Known Issues

There is an issue where customers who have the V2 endpoint running with an older version and try to upgrade to a newer V1.6 release will see that the 50K limitation on group membership is reinstated. We will not fix this issue in V1.6 and require customers to upgrade to AADConnect V2.0 if this is an issue for them.

Azure AD Connect V1.x customers are strongly encouraged to update to V2.x, keeping in mind that this may require installing AADC V2.x on a new Windows 2016 or Windows 2019 server. I wrote a step-by-step article on upgrading here.

In the meantime, if you are still using Azure AD Connect 1.x you should make sure you're using the V1 endpoint using the following steps.

First, check to see which sync endpoint you're using with these cmdlets, run from the server where Azure AD Connect 1.x is running:

• Import-Module 'C:\Program Files\Microsoft Azure AD Sync\Extensions\AADConnector.psm1'
• Get-ADSyncAADConnectorExportApiVersion
• Get-ADSyncAADConnectorImportApiVersion

If both Get-* cmdlets return the value "1", you're using the V1 endpoint. Nothing more to do here, except plan to upgrade to AAD Connect V2.x as soon as reasonable.

If the values returned are "2", you're using the V2 endpoint and need to change it back to V1.

• Import-Module 'C:\Program Files\Microsoft Azure AD Sync\Extensions\AADConnector.psm1'
• Set-ADSyncScheduler -SyncCycleEnabled $false
• Set-ADSyncAADConnectorExportApiVersion 1
• Set-ADSyncAADConnectorImportApiVersion 1
• Set-ADSyncScheduler -SyncCycleEnabled $true

Be aware that the V1 endpoint cannot sync groups with 50K+ members. You should plan to upgrade to AAD Connect V2.x as soon as possible.

Read more ...

CHANGE LOG: View Quarantine add-in for Outlook

View Quarantine add-in for Outlook Change Log

---

Please see How to install an Outlook add-in to view the Microsoft 365 End-User Quarantine for a full description and installation instructions.

---

UPDATE #1: Microsoft won't certify my add-in because they say it "does not provide significant value or benefits to commercial marketplace customers". I think most of you will disagree. I'll keep trying to get it certified, but in the meantime you can always install it from my website using the procedures from my blog article.

UPDATE #2 [version 1.0.1.0]: I added automatic localization for 37 languages. Reinstall the add-in to if you need one of these languages. Please let me know if my translations need adjustments.

Read more ...

How to install an Outlook add-in to view the Microsoft 365 End-User Quarantine

This article explains how to install an Outlook add-in that will open the Microsoft 365 end-user quarantine in a browser with a single click. This makes it really easy to access the quarantine directly from Outlook. And since this is a true Office add-in, it also displays and works in Outlook mobile and Outlook on the web!

View the add-in change log for status and feature updates.

The add-in displays in the Outlook ribbon when viewing any folder that contains mail items. 

The View Quarantine add-in for Outlook

The Microsoft 365 end-user quarantine

I originally built this add-in using the Build your first Outlook add-in - Office Add-ins documentation. This gave me a good head start to build and customize the add-in.

The View Quarantine add-in source files are available for free on my website here. The add-in consists of three files plus icons in various sizes for the different platforms.

Source File	Description
commands.html	An HTML "wrapper" that calls the JavaScript used by the add-in when the button is clicked.
commands.js	The JavaScript functions that provide status and opens the end-user quarantine in a browser.
manifest.xml	The real heart of the add-in. It defines the unique ID for the add-in and describes when to display the View Quarantine button and how the add-in functions.
assets folder	Contains six icon files of different sizes and opacity for Outlook, OWA, and Outlook mobile.

I'm currently in the process of publishing this add-in to the AppSource marketplace via the Microsoft Partner Center, which may take a week or so to certify. In the meantime, there are two ways you can try out the View Quarantine add-in now.

Option 1 -- Install via the Web

You can install the add-in from my website until Microsoft publishes it on AppSource.

• Open Outlook and click the Get-Add-ins button in the ribbon.
• Click My add-ins in the top left.
• Click the + Add a custom add-in dropdown at the bottom of the window under Custom add-ins, then select Add from URL...
• Enter the following URL: https://www.expta.com/quarantine/manifest.xml and click OK.

• You will see a warning before installation. Click Install to install the add-in.

• The add-in will now be listed under Custom add-ins. Note: To remove the View Quarantine add-in at any time, click the ellipses (...) and select Remove.

• Close the Add-ins window to add it to the Outlook ribbon.

Option 2 -- Install from Source Files

You can also install the add-in using the manifest.xml file in my source files.

• Download the View Quarantine source files from my website.
• Extract the ZIP file to a local drive or network share.

• Open Outlook and click the Get-Add-ins button in the ribbon (shown above).
• Click My add-ins in the top left.
• Click the + Add a custom add-in dropdown at the bottom of the window under Custom add-ins, then select Add from file...
• Browse to the manifest.xml file and click Open.
• You will see a warning before installation. Click Install to install the add-in.

• The add-in will now be listed under Custom add-ins.

• Close the Add-ins window to add it to the Outlook ribbon.

Deploying to All Users in the Organization

Once you're satisfied that the add-in is installed and working properly, you can deploy it to all users in your organization. Here's how to do that:

• Open the Exchange Admin Center and navigate to Organization > add-ins.
• Install the manifest file by clicking the + drop down button and do one of the following:
• To install from the web, select Add from URL, enter https://www.expta.com/quarantine/manifest.xml, and click OK
• To install from the source files, select Add from File, browse to the manifest.xml file you downloaded above, and click OK
• After the add-in is installed, double-click the View Quarantine add-in to select how you want it deployed to users:
• Optional, enabled by default (users can choose to remove it)
• Optional, disabled by default (users can choose to enable it)
• Mandatory, always enabled. Users can't disable this add-in.

I hope you enjoy this free add-in and you find it useful!

Read more ...

Notes and details on the eradication of Basic Authentication in Exchange Online

Unless you've been living under a rock, or are just blissfully unaware, Microsoft has been making a concerted push to remove Basic authentication from Exchange Online for some time.

There's a very good reason for this. Basic auth is a single factor authentication method (username/password), which is just too easy for the bad guys to guess and exploit. Modern Authentication, on the other hand, supports MFA and is much more secure. Disabling Basic auth in your tenant requires you to use Modern Auth for all authentication requests.

The trouble is that some legacy apps and clients still only use Basic auth. Fortunately, that list is getting shorter. As you may have read in the Microsoft Message Center or the Exchange Team Blog, Microsoft is currently disabling Basic auth in tenants that they've determined are not using it. I applaud this endeavor.

At a recent MVP meeting we discussed how this effort is being undertaken. Here are some notes and details on certain aspects that you might find useful or interesting.

• Microsoft is examining tenants for actual Basic auth usage. They are not checking to see if the tenant has an Authentication Policy set or is using Conditional Access to block Basic authentication.
• Basic auth is being disabled in the tenant configuration for all protocols except Autodiscover. Basic auth is required by Autodiscover for legacy (read, old) Outlook clients like Outlook 2013 and earlier. This alone is one of the best reasons to get off these old clients ASAP. See New minimum Outlook for Windows version requirements for Microsoft 365 starting November 1, 2021.
• Basic auth for SMTP is being disabled for customers that don't use it by using the Set-TransportConfig -SmtpClientAuthenticationDisabled:$true command. Admins can reenable it by setting the value to $false. This setting can also be configured as a per-user setting, which is recommended. The user setting overrides the tenant setting.
• Authentication Policies are the preferred way to disable Basic auth, rather than Conditional Access policies. CA policies only apply AFTER the user has already signed in.
• You can use Authentication Policies to disable Basic auth for Autodiscover (and all other protocols). That means that if you may have two areas to check if you need to reenable Basic auth for a protocol -- the Auth Policy and the tenant configuration settings that Microsoft is using.
• For a limited time, tenant admins can use the Basic Auth troubleshooter to run diagnostics and provide self-service options to reenable Basic auth for Exchange Online protocols such as POP3, IMAP4, Exchange ActiveSync, Exchange Web Services, Offline Address Book, MAPI, RPC and Remote PowerShell. Simply click the Help & Support button on any O365 portal and type Diag: Enable Basic Auth in EXO.

• So far, they have disabled Basic auth in thousands of tenants since they started. Only 0.06% of tenants have reenabled Basic auth for a specific protocol, and all of them using the self-help troubleshooter.
• Tenant admins can tell if Basic auth has been disabled in their tenant by connecting to Exchange Online PowerShell and running Get-OrganizationConfig | fl basic*. The BasicAuthBlockedApps property value will be 0 if Basic auth is still enabled or 255 if it's been fully disabled. This value is a bit mask for each of the following protocol values, totaling 255. Thanks to Greg Taylor for the secret decoder ring. 😊

Protocol	Action	Value
ActiveSync	Block Basic for Exchange ActiveSync	1
WebServices	Block Basic for Exchange Web Services	2
POP	Block Basic for POP3 Clients	4
IMAP	Block Basic for IMAP4 Clients	8
PowerShell	Block Basic for PowerShell	16
MAPI	Block Basic for MAPI Protocol	32
OAB	Block Basic for Offline Address Book	64
RPC	Block Basic for RPC Protocol	128

• Be aware that if you've configured a client to connect using Basic auth (Outlook for Mac, for example), it will likely require you to reconfigure the client profile to use Modern Auth after Basic is disabled.

This information should be helpful in your "Death to Basic Auth" journey.

Read more ...