How to confirm if your IP address is blocked by Exchange Online Protection (EOP)

Exchange Online Protection uses several mechanisms to protect the service and its customers from receiving spam from known bad or suspicious IP addresses. This article explains how to check if the public IP address used by your messaging server is on the EOP Anti-Spam IP List, and how to remove it if it is.

This is especially important to do for Office 365 hybrid customers, since the on-premises Exchange servers will need to send SMTP emails to cloud users via EOP. I always do this before I begin a hybrid deployment.

From a web browser go to the Office 365 Anti-Spam IP Delist Portal (https://sender.office.com).

Step 1: Enter a valid email address to receive a verification email to complete the check, and enter the public IP address of the messaging server you want to check. You can use a service like WhatIsMyIPAddress.com from the messaging server to determine your public IP. Enter the captcha characters and then click Submit.

Office 365 AntiSpam IP Delist Portal

You will receive an email with the subject Microsoft Office 365 Delisting Service where you need to click a link to confirm your email address. This link is valid for about 10 minutes. If you wait longer than that, you'll need to start over. The email is sent almost immediately, so if you don't see it in your Inbox within a minute, check your junk mail.

Click to confirm your email address

Step 2: A new browser window will open to allow you to delist the IP address you entered in step 1. Click the Delist IP Address button.

Step 2: Delist IP Address

If the IP address is not on the blocked list, you'll see a message similar to the one below:

Step 3: The IP address is not currently blocked by EOP

If the IP address is blocked it will be submitted to be automatically delisted. The website says this takes about 30 minutes, but in practice I've seen it take up to 24 hours. You will not get a confirmation email when the IP address has been delisted. You'll need to re-run the test occasionally until you see that it's not blocked.

Step 3: The IP address is blocked, but will be delisted

The quick geek way to test if your IP address is blocked is to use the telnet client. From the messaging server, telnet to the EOP endpoint using port 25 and try to send an SMTP email. You can find your EOP endpoint using www.mxtoolbox.com. If your IP address is on the EOP block list you will see it after the RCPT TO command .  Here's an example of the telnet output:

Access denied

Here, you can see that the IP address is banned by EOP and instructions for removal.

550 5.7.606 Access denied, banned sending IP [x.x.x.x]. To request removal from this list please visit https://sender.office.com/ and follow the directions. For more information please go to http://go.microsoft.com/fwlink/?LinkID=526655 (AS16012609)

Read more ...

How to Add Access to the Office 365 EOP Quarantine in Outlook and OWA

Exchange Online Protection (EOP) in Office 365 offers a quarantine feature for administrators and end-users. Administrators can view and release messages that were caught as spam for all users. End-users can view and release their own messages.

The administrative quarantine is accessed from the Office 365 Exchange Admin Center > Protection > Quarantine. Here you will see all the quarantined items for all users in the org.

Administrative Quarantine in the Office 365 Portal

The administrative quarantine is always available to EOP admins. The end user quarantine must be enabled and configured by the admin from Office 365 Exchange Admin Center > Protection > Content Filter, then click the Configure end-user spam notifications link on the right.

When enabled, end-users will receive an email from EOP listing all the new messages held in quarantine since the last notification.

The URL to the end-user quarantine is https://admin.protection.outlook.com/quarantine. In a hybrid scenario, even users without Office 365 licenses can access the end-user quarantine.

End-User Quarantine

That's all pretty cool, but wouldn't it be nice to access the quarantine directly from Outlook? Here's how you configure it:

• Open Outlook as the end-user.
• Right-click the user mailbox and select New Folder.

• Name the folder Quarantine or * Quarantine to have it placed higher in the folder list.
• Right-click the new folder and select Properties.
• Click the Home Page tab.
• Enter https://admin.protection.outlook.com/quarantine in the Address field and then click the checkbox for Show home page by default for this folder. Click OK.

• You will then see the sign on page for EOP within Outlook. After you sign in you can access your end-user archive.

End-User Quarantine within Outlook

Unfortunately you can't view the end-user quarantine using this method from OWA, but I have a work-around. 

• Send yourself an email with the subject of Quarantine and the link to https://admin.protection.outlook.com/quarantine in the body of the message.
• Drag the message from your Inbox to the Quarantine folder you created.

Now you have easy access to the end-user Quarantine from Outlook, OWA, and even mobile devices!

Read more ...

Pending Messages in Exchange Online Will Not Be Delivered

UPDATE: I'm happy to say that the bug described in this article has been squashed. Deferred emails will automatically be recategorized in Office 365 transport within a few hours and will then be delivered.

There’s a pretty big unpublicized bug in Office 365 transport that you need to be aware of.

If emails cannot be delivered due to a problem with TLS those emails will queue in Exchange Online Protection (EOP) and will be marked as Pending or Deferred. If you change the Outbound Connector in EOP to work around the problem these messages will NEVER be delivered, even after the transport issue is resolved. This happens because the outbound messages in Office 365 are stamped with the old TLS configuration and are not reevaluated when the Outbound Connector configuration is changed.

The following example shows a message that was sent at 2:46PM UTC and the status shows as Pending. The last event for this message shows DEFER at 3:29PM UTC with the detail, "The last attempt to deliver the message encountered an error".

Sample Message Trace from Office 365

For example, say the TLS certificate expires on your hybrid server. Inbound messages from EOP to the hybrid server will queue because the Outbound Connector is using Forced TLS, but the certificate is invalid. If you resolve the problem by reconfiguring the Office 365 Outbound Connector to use Opportunistic TLS the problem is solved for new emails - they get delivered right away, but the Pending messages will never get resubmitted and eventually expire after 48 hours.

This same behavior would occur if you have a custom Outbound Connector that forces TLS with a business partner. If their TLS certificate expires or they reconfigure their server to not use TLS. The messages will not be resubmitted to use the new configuration.

Incredibly, there currently is no way for Microsoft to resubmit these messages like there is for on-prem Exchange. After opening a high-priority case with Microsoft Online, the only "solution" they could give is to contact all the senders and ask them to resend their emails.

To work around this, you may want to ensure that your Outbound Connectors that use TLS are configured to use opportunistic TLS. That way if something changes in the TLS configuration, such as the receiving server's cert expires or TLS is disabled, emails can still be delivered.

Read more ...

Troubleshooting TLS SMTP Connections to Exchange Online Protection

By default Office 365 uses Transport Layer Security (TLS) to send encrypted SMTP emails between Exchange Online and Exchange on-prem. This provides end-to-end encryption of emails between your on-prem Exchange Hybrid Server and Exchange Online Protection (EOP), just like they were the same organization.

TLS utilizes x.509 (SSL) certificates to encrypt the email in transport. Exchange Online and Exchange 2013's implementation of TLS differs from previous implementations of TLS in several important ways:

• The certificates used for TLS must be issued by a trusted third-party CA (Digicert, Godaddy, Verisign, etc.)
• The CRL for the certification authority must be available. If Exchange or O365 can't read the CRL it will not trust the certificate.
• The FQDN that the Receive Connector provides in response to EHLO must match the subject name or a subject alternative name on the certificate. SAN certificates and wildcard certificates are both valid for TLS use. If you use a wildcard cert the FQDN used on the connector can be any name that is valid for the wildcard cert's domain.
• In previous versions of TLS any certificate could be used to encrypt SMTP traffic, even expired or self-signed certificates. This is not the case with Office 365 and Exchange 2013.

As previously mentioned, TLS is required by default for SMTP communication between Hybrid servers and Exchange Online Protection (EOP). When you run the Hybrid Configuration Wizard it will configure forced TLS for all Send and Receive Connectors. Note that this configuration is updated whenever you run the HCW, so if you reconfigure the connectors for opportunistic TLS or turn it off completely, the HCW will reconfigure them to use forced TLS again.

The following options are available for TLS encryption in Office 365:

• Force TLS - Email sent across this connector must use TLS. If TLS is unavailable messages will queue until they are delivered or expired.
• Opportunistic TLS - The connector tries to setup a TLS connection using the STARTTLS verb. If a TLS connection cannot be made the connector falls back to regular ESMTP or SMTP. This is functionally equivalent to turning TLS off.

Normally the use of TLS is configured on Receive or Inbound Connector. You do this to control how email is accepted by your domain and it can be easily configured using the Exchange Admin Console. 

Office 365's Inbound Connector from Hybrid Server

Send Connectors can also be configured to require TLS. This is used to enforce that email is sent by this connector must only use TLS, and is configured in the shell using the RequireTLS property:

Outbound Connector TLS Settings to Office 365

Using this configuration if the corresponding Receive Connector does not offer or require TLS, messages will queue on the sending server until they are finally delivered or expire.

You can easily tell if a receiving SMTP server is configured to use TLS using Telnet. Install the Telnet Client feature, Telnet to the server using TCP port 25, and look for the STARTTLS verb after issuing the EHLO domain.com command. For example:

Telnet servername 25

If you see that the STARTTLS verb is missing, the server is not offering TLS. If your Send Connector is configured to require TLS, the messages will queue on the sending server with the error,

451 4.4.0 Primary target IP address responded with: "451 5.7.3 STARTTLS is required to send mail."

There are several reasons that the STARTTLS verb might be missing:

• The receiving server is not configured to Force TLS or use Opportunistic TLS.
• The sending server's IP is on an SMTP block list (aka SMTP blacklist or SMTP blocklist). Office 365 will not attempt to send TLS traffic with a server it can't trust.
• The receiving server is configured to only respond to SMTP (not ESMTP) commands. TLS is part of the ESMTP protocol.
• Your firewall is doing some form of email inspection and is filtering "unsafe" verbs from the SMTP conversation. Some examples of firewalls that do this are:
• Cisco ASA
• Dell SonicWall
• Barracuda

Office 365 uses internal and external SMTP blocklists to protect the network. If the sending server is on one of these black lists the EOP servers will not offer TLS. You can test this by sending a non-TLS email to EOP using Telnet. See Brian Reid's article, "Cannot Send Emails to Office 365 or Exchange Online Protection Using TLS". The response from the EOP server will tell you which block list you're on and how to request removal.

If the receiving server requires TLS, but the sending server is not configured to use a TLS certificate messages will queue on the sending server with the error,

451 4.4.0 Primary target IP address responded with: "451 5.7.3 Must issue a STARTTLS command first."

The fix here is to configure the Send Connector to use TLS and a valid certificate using the Enable-ExchangeCertificate cmdlet, such as:

Enable-ExchangeCertificate -Thumbprint 5DC5902752816FD2FC51D5564C363F68D8F7FFC4 -Services SMTP

Make sure to use Get-ExchangeCertificate to get the correct certificate's thumbprint for the command above.

Hopefully this information will help you understand TLS and will assist you with troubleshooting.

Read more ...

How to Configure an Internal SMTP Relay Server for Office 365

Most organizations have internal application servers and appliances that send emails to users or groups. Examples include copier/scanners and application servers, such as backup servers that notify admins of a completed or failed backup job.

If the organization has Exchange on-prem you would normally configure an internal relay receive connector in Exchange and configure the internal resources to send emails to Exchange. But what do you do when you've migrated all your mailboxes to Office 365 and have decommissioned your Exchange servers?

The solution is to install an IIS SMTP relay server in your internal network, configure it to accept email from specific IP addresses, and forward emails to Office 365. You can also configure the SMTP relay for external domains, if necessary.

Here's how to do it:

• Install the SMTP Server feature and its dependencies to a new or existing Windows server. This will be your relay server and your firewall needs to allow it to send SMTP traffic (TCP port 25) outbound to the Internet. I typically use the DirSync server, if there is one.

Adding the SMTP Server feature and its dependencies to Windows Server 2012

• Open Internet Information Services (IIS) 6.0 Manager to configure the SMTP relay.
• Configure the properties of [SMTP Virtual Server #1] as follows:
• On the Access tab:
• Authentication: Only Anonymous access is checked.
• Relay: Only the list below. Add IP addresses or ranges of servers allowed to relay.
• Note - It's important to only allow IP addresses you trust to relay through this server. Any IP address you enter here will be allowed to send emails on behalf of your domain.
• On the Messages tab:
• Adjust message size limits. The default message size limit is 2048 KB (2 MB).  You may want to change it to 10240 KB (10 MB) or more to allow for larger messages from copier/scanners, etc.
• On the Delivery Tab:
• Outbound Security: Anonymous access only and no TLS encryption.
• Outbound Connections: Port 25
• Advanced: Leave the Smart Host field blank
• Add new remote domains:
• Right-click Domains > New > Domain and add the domain(s) hosted in Exchange Online.
• If the relay server is allowed to relay emails to other external domains add a new *.com remote domain. Repeat for *.org, *.net, etc. as necessary.

Add Office 365 and other remote domains if required for external relay

• For properties of each domain hosted in Exchange Online:
• Use www.mxtoolbox.com to get the EOP endpoint (MX record) for your tenant. For example, contoso-com.mail.protection.outlook.com.
• Check Allow incoming mail to be relayed to this domain
• Forward all mail to this smart host: The EOP endpoint you identified above (i.e., contoso-com.mail.protection.outlook.com)
• Outbound Security: Check Anonymous access and TLS encryption
• For properties of all other remote domains (if any):
• Check Allow incoming mail to be relayed to this domain
• Outbound Security: Check Anonymous access and do not check TLS encryption.
• Restart IIS.  Be aware that whenever you restart IIS, the SMTP virtual server usually stays stopped – start it.

Now we need to configure Exchange Online to accept emails from the new relay server. Recent changes in EOP email security require that you verify the relay server by specifying the SSL certificate or public IP address used by the relay server.

• Sign-in to the Exchange Online Admin Center with your admin account
• Navigate to Mail Flow > Connectors and add a new connector
• On Select your mail flow scenario, select:
• From: Your organization's email server
• To: Office 365
• Name the new connector "Relay from On-Premises" and click Next
• On How should Office 365 identify email from your email server? you have two choices:
1. If you have installed a valid third-party SSL certificate from a trusted CA on your SMTP relay server you can enter the Subject Name for the cert (i.e., mail.contoso.com, or *.contoso.com for a wildcard cert) on the new connector.
2. If your SMTP relay server has a static public IP address you can enter that IP address. You can use a service like https://www.whatismyip.com/ to get your current IP address. Note that if this IP address ever changes, you'll need to update the connector in Exchange Online.
• Confirm the new connector's setting and click Save.
• Note that it can take up to 15 minutes for this connector and settings to propagate throughout EOP.

Notes/Troubleshooting:

• The SMTP Server feature can be added to any Windows 2003 or better server. I usually use the DirSync server if there is one.
• Unlike Exchange, TLS for IIS 6 SMTP servers is not opportunistic. If the virtual server or a remote domain is configured to use TLS, email will not be sent if the remote domain does not support TLS. Office 365 offers TLS, so we can use it.
• The configuration above allows the IIS 6 SMTP server to send emails to the Internet for the remote domains configured, so you should add the public NAT IP address for this server to your existing SPF record to prevent non-delivery. Use http://whatismyip.com from the SMTP server to determine the NAT IP address.
• Monitor the %systemdrive%\Inetpub\mailroot\Queue folder to ensure that emails are being delivered.
• If emails are not being delivered to Office 365 users, test sending email via Telnet. The IP address may be blocked by an Exchange Online Protection (EOP) blocklist and you will see that response from EOP. If so, send a delist request from your Office 365 admin account to delist@messaging.microsoft.com letting them know the IP address that should be delisted. In my experience it takes up to 36 hours for Microsoft to delist it.
• If emails are not being delivered to external domains, ensure that you have a remote domain type (*.com, *.eu, etc.) configured for those email addresses.
• You can enable logging in the properties of the SMTP virtual server for further troubleshooting. Use the NCSA Common Log File Format. IIS does not automatically groom or delete logs like Exchange does, so turn logging off when you're done troubleshooting.
• The best practice is to create an A record in internal DNS for smtp.yourdomain.com using the SMTP relay server's IP, and configure all application servers and appliances to use that FQDN for email forwarding. That makes it easier to update in the future.

Read more ...