Exchange Online will Throttle and Block Email from Persistently Vulnerable Exchange Servers

Microsoft announced that they will begin Throttling and Blocking Email from Persistently Vulnerable Exchange Servers to Exchange Online.

They will begin slowly by blocking OnPremises (hybrid) connections from Exchange 2007 servers to Exchange Online, but plan to include all persistently vulnerable servers soon. A “persistently vulnerable server” is any Exchange server that has reached end of life (e.g., Exchange 2007, Exchange 2010, and very soon, Exchange 2013), or remains unpatched for known vulnerabilities.

If you have any Exchange 2007, 2010, or 2013 servers in your organization that are used for hybrid or SMTP relay, you must make plans to replace or remove them ASAP. You must also keep any Exchange 2016 and 2019 servers up to date with the latest Cumulative and Security Updates, as they are released.

I support this initiative. Organizations using Exchange Servers with known and unpatched vulnerabilities put themselves and their partners at risk. I look forward to seeing it include additional unsupported Exchange Server versions, and I understand their approach of rolling this out slowly and carefully at first.

There are millions of mailboxes in Exchange Online and they have a duty to protect those mailboxes. They know there are 16-year old Exchange 2007 servers currently connecting to their service that haven’t been patched in years and have dozens of known vulnerabilities. 

Microsoft is not forcing anyone to move to Exchange Online. They’re just not going to allow Exchange servers with known vulnerabilities to send hybrid (trusted) emails into their service.

It’s only 2007 for now, but eventually it will be any Exchange Server that is not running the latest or second to latest updates. It’s too risky for everyone.

The table below details the stages of progressive enforcement over time of a persistently vulnerable server:

Read more ...

Do Not Install the February 2023 Exchange Server Security Updates via Windows Update

Microsoft released the February 2023 Exchange Server Security Updates yesterday, February 14, via Windows Update and on their website. The website links to the correct files for the February 2023 Security Updates, but Windows Update currently is deploying the January 2023 Security Update CAB file for Exchange Server 2013, 2016, and 2019. 

For this reason, install the February updates from the hyperlinks listed in the EHLO Blog post, not Windows Update. If you did already install the Exchange Security Updates via Windows Update, download the files directly and reinstall them on all your servers. Then check the version with the HealthChecker.ps1 script.

UPDATE: Microsoft just said, "We've fixed the issue, so it's safe to install the SU from MU/WU again. Apologies for the inconvenience." According to Microsoft, customers who already installed the February SU via Windows Update can either install it again via Windows Update or download the update from the web and install it.

For further details, keep reading.

Several of my customers installed the Security Update For Exchange Server CU23 SU6 (KB5023038) on multiple servers via Windows Update. These servers include Exchange Server 2013, 2016, and 2019.

Windows Update history shows the February 2023 KB5023038 update has been installed, but HealthChecker.ps1 reports the servers are still on the January 2023 KB5022143 SU. 

Checking the registry,

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Microsoft Exchange v15\DisplayVersion says the Exchange build is 15.1.2507.18 (not listed here, but should be 15.1.2507.21). 

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Patch\DisplayName says “Security Update for Exchange Server 2016 Cumulative Update 23 (KB5022143)” 

When you run Windows Update again, it says no updates are pending.

When I view the C:\Windows\SoftwareDistribution\Download folder I see a folder named f664b5c67010fa70659624355017ed43 with the date and time when Windows Updates were applied.

Inside that folder is the Exchange2016-KB5022143-x64-en.cab which is the January 2023 Exchange Security Update, proving that Windows Update is pushing the wrong SU.

Installing the February 2023 Exchange Security Updates from the published download pages installs the correct SU. Verify with the HealthChecker script.

Read more ...

How to Remove Internet Explorer from Windows Computers and Servers

Support for Internet Explorer 11 ended on June 15, 2022 and IE 11 will no longer be accessible after February 14, 2023.

Even so, Internet Explorer 11 is still installed as the default web browser with every version of Windows and Windows Server. To make matters worse, Windows 10 also shipped with non-Chromium Edge (now called Legacy Edge). You'll need to download and install a "modern" web browser, such as Microsoft Edge, Google Chrome, FireFox, etc. Keep in mind that you can reload Internet Explorer sites with IE mode in Microsoft Edge.

Even with ChrEdge (my name for Chromium Edge) installed as the default web browser, I've heard from a few customers that IE is still being used for some web work flows. For example, a customer recently showed me this MFA security confirmation prompt that is opening in IE for some reason.

All this leads me to the task at hand. How do you uninstall Internet Explorer 11 from Windows?

This is not straight forward since IE 11 is part of the Windows installation. It cannot be removed as a Windows application or feature. The answer is to use DISM (Deployment Imaging Servicing Management) installed with all versions of Windows and Windows Server.

Run the following command from an elevated CMD or PowerShell prompt:

C:\Windows\System32\dism.exe /online /Disable-Feature /FeatureName:Internet-Explorer-Optional-amd64

Removal only takes a few seconds and the output looks like this:

DISM will prompt you to restart the computer to complete the operation. Until you do, Internet Explorer will still show as an available web browser. Optionally, you can add the /Quiet switch which will not show DISM uninstall progress and will automatically restart the computer when it's done.

Read more ...

ProTip: Scheduled Task to Start Stopped Services in Exchange Server

As a best practice, I've always created a scheduled task on my Exchange Servers that starts all stopped services 1 minute after server startup. This is useful for virtualized environments where servers restart much faster than physical environments. Often the VM comes back up before the network has fully initialized. When this happens, some of the Exchange services do not start properly.

This scheduled task is becoming important with the release of the January 2023 Exchange Server Security Updates. There is a known issue that Microsoft Exchange AD Topology service may not start properly on Exchange 2016 servers running on Windows Server 2012 R2. Microsoft is investigating.

I've seen this happen on several of my customers' environments. Since almost all Exchange services rely on the AD Topology service, none of the Exchange services will start. You are required to start all the services manually after every reboot. Instead, you can install the scheduled task that does this for you.

Run the following from an elevated PowerShell or EMS prompt:

$TaskProgram =  "$env:SystemRoot\System32\WindowsPowerShell\v1.0\powershell.exe"

$TaskArguments = '-NoProfile -command "gwmi win32_service | where {$_.StartMode -eq ''Auto'' -and $_.State -eq ''Stopped''} | Start-Service"'

$TaskAction = New-ScheduledTaskAction -Execute $TaskProgram -Argument $TaskArguments

$TaskTrigger = New-ScheduledTaskTrigger -AtStartup -RandomDelay 00:01:00

Register-ScheduledTask -TaskName "Start Stopped Services" -Action $TaskAction -Trigger $TaskTrigger -RunLevel Highest -User "system" -Description "Starts all stopped services 1 minute after startup due to slow network initialization."

It's safe to install this on any Exchange server.

Read more ...

Turning off Basic Authentication for Autodiscover in Exchange Online

Much has been said and written about disabling Basic Authentication in Exchange Online, and for good reason. Basic Auth is insecure and makes it easy for bad guys to hack your accounts and access your organization's data.

Microsoft disabled Basic Auth for most Exchange Online protocols in October of this year. Those protocols include Outlook, EWS, RPS, POP, IMAP, and EAS. SMTP Auth was also be disabled in your tenant if it is not being used. Modern Authentication is the secure way to authenticate for these protocols. Congratulations to Microsoft for pulling off such a monumental achievement to help keep customers' data safe!

Read Deprecation of Basic authentication in Exchange Online | Microsoft Learn for more information.

Next up, Microsoft is going to disable Basic Auth for the Autodiscover protocol. I would argue this is one of the most significant changes to deprecating Basic Auth, since it is continuously used by Outlook and whenever you configure a mail profile on a mobile device that uses ActiveSync. Because of this, it's easy to ignore this traffic when monitoring for Basic Auth usage. It's also a fairly easy protocol for the bad guys to use for password guessing or dictionary attacks.

As with all the other impacted protocols, Microsoft is not turning off the protocol itself, only the ability to authenticate to the protocol using nothing more than a username and password.

Read more ...

Azure AD Connect version 2.1.20.0 released with a new ability you probably can't use yet

UPDATE: Hot on the heels of the last update, Microsoft released Azure AD Connect version 2.1.20 (6 days later) which apparently fixes a bug in the new sync feature described below. If you implemented the custom sync rule described earlier, you'll need to undo it and do it again. <facepalm>

Microsoft updated Azure AD Connect to version 2.1.19.0 today. 

According to the release notes, "We added a new attribute 'employeeLeaveDateTime' for syncing to Azure AD. To learn more about how to use this attribute to manage your users' life cycles, please refer to this article".

Let's break this down.

First, the support article the release notes refer to is for Azure AD Connect cloud sync, not Azure AD Connect. At this time, syncing the employeeLeaveDateTime is still not supported in Azure AD Connect.

Since AAD cloud sync doesn't support Exchange hybrid at this time, AAD cloud sync is of no value for hybrid customers. However, the AD Connect team is working on adding Exchange hybrid support in the future. 

The support table shows that syncing the employeeHireDate attribute is already supported in Azure AD Connect, so I expect that suspect the configuration is the same. I could not find a similar support article for Azure AD Connect.

Second, the employeeHireDate and EmployeeLeaveDateTime attributes only exist in Azure AD. The on-premises Active Directory schema is not extended to add these two attributes, so the recommendation is to use one of the existing extensionAttribute* attributes an AD to hold these values.

At this time, Microsoft recommends using the extensionAttribute1 attribute for employeeHireDate, but the documentation makes no mention of how to handle the EmployeeLeaveDateTime attribute. See How to synchronize attributes for Lifecycle workflows - Microsoft Entra | Microsoft Learn. I hope this documentation will be updated soon with the missing info and how to handle it if this attribute is already in use in your organization.

Last, if you want to update the employeeLeaveDateTime attribute directly in Azure AD using the Graph API, please see Set employeeLeaveDateTime - Microsoft Graph | Microsoft Learn.

Read more ...

Fix for "Online - Data retrieval failures occurred" on Exchange DAG members

You may find that when you add an Exchange server to a DAG that Server Manager shows multiple errors.

The Notification flag at the top indicates "Refresh failed" and Manageability for All Servers shows an error for the remote DAG member saying, "Online - Data retrieval failures occurred".

You may also see errors that say, "Configuration refresh failed with the following error: The WS-Management service cannot process the request. The computed response packet size (517916) exceeds the maximum envelope size that is allowed (512000)."

These errors occur when the Failover Clustering feature is installed on the DAG member. I've usually only seen this for Exchange 2019 installed in Windows Server 2019 or Windows Server 2022. This is a Windows Server issue, not an Exchange issue, so this fix should also apply to any Windows cluster experiencing this problem.

The fix is to increase the WSMAN maxEnvelopeSize in the registry on all DAG members.

1. On the DAG member, launch regedit.exe and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WSMAN\Client.
2. Create a new DWORD (32-bit) value named maxEnvelopeSize, or modify it if it already exists.
3. Set the value data to 2000 hexadecimal (8192 decimal).

Finally, restart the Windows Remote Management (WS-Management), aka WinRM, service on the server.

When you refresh Server Manager, the error should go away.

Read more ...