Where do those "# Questionable URLs detected in message" emails come from?

Exchange Online admins may receive emails from time-to-time with the line, "# Questionable URLs detected in message:". The email includes the SMTP headers, a text-only version of the original message, and the original message included as an attachment.

Where are these emails coming from and why are you getting them?

The answer lies within the User Submissions configuration in the Microsoft Defender portal (https://security.microsoft.com). Go to User submissions - Microsoft 365 security and check the Send the reported messages to: configuration.

The default and recommended setting is to send reported messages only to Microsoft, but you can reconfigure it to send to Microsoft and another email address in your organization.

This will send diagnostic info to both Microsoft and the internal email address you specified.

What isn't obvious is that Safe Links in Microsoft Defender for Office 365 (formerly ATP) uses the same configuration. When Safe Links detects questionable URLs in an email, that diagnostic information is sent the same way as User Submissions. So if you've configured User Submissions to send reported messages to an internal email address, you will get Safe Link reports to that address, too.