How to troubleshoot Exchange Online Transport Rules using the Test-Message cmdlet

To help tenant administrators investigate issues related to the processing of their Mail Flow Rules (Exchange Transport Rules aka ETRs), we recently updated the Test-Message cmdlet in Exchange Online PowerShell. This cmdlet enables administrators to perform independent investigations and might eliminate the need to engage Microsoft support for assistance.

Previously, administrators had limited tools to verify if an ETR was correctly processing emails. They could only rely on Message Trace, which only showed matched rules (but not all evaluated ones). Another option was Extended Message Trace, which could display the rules evaluated for a specific message, but this was a time-consuming process. With Message Trace and Extended message trace, administrators cannot see why the rule did not match.

By sending test email messages into the actual Exchange Online ETR pipeline, Test-Message simulates real behavior and generates an immediate evaluation report for administrators. This report contains information about every rule evaluation as well as every condition/exception evaluation result per rule.

Exchange Online Test-Message cmdlet improvements

Recent updates to this Exchange Online cmdlet include:

• Logs are more comprehensive, and it is now possible to distinguish conditions from exceptions.
• The report has been updated to include the message property values evaluated by transport rule conditions and exceptions.
• Once a rule has been evaluated, a final verdict is provided to indicate whether it was a match or not.

These changes can give tenant administrators insights into what properties are passed to the ETR conditions/exceptions and what evaluation results are, per condition/exception.

How to run Test-Message

Please note there is currently no option to utilize test-message from within the Exchange Admin Center.

After connecting to Exchange Online PowerShell, Test-Message cmdlet can be executed in two ways:

1. Utilizing only Sender and Recipient parameters
2. With message sample data provided in the MessageFileData parameter

When Test-Message cmdlet is executed with only the -Sender and -Recipients parameters, the message body and subject will contain default text.  The subject will contain the text “Exchange diagnostic message” and the body contains the text “This message was generated by an Exchange administrator. You can ignore this message unless your administrator has requested otherwise.”

For example:

Test-Message -Sender megan@contoso.com -Recipients adele@contoso.com -SendReportTo admin@contoso.com -TransportRules -UnifiedDlpRules

When the Test-Message cmdlet is executed with message sample data, admins need to:

1. Save a message they want to use as a file (Save a message as a file).
2. Following the Test-Message guide, the  next step is to read this file:

$data = [System.IO.File]::ReadAllBytes(‘C:Datatest.eml’)

3. Use the $data variable in the –MessageFileData parameter  when running Test-Message.
   Test-Message -MessageFileData $data -Recipients adele@contoso.com -SendReportTo admin@contoso.com -TransportRules -UnifiedDlpRules

Note that if the admin executes Test-Message with message sample data, recipient of the message must be specified as well using –Recipients parameter. If the recipient in the –Recipients parameter is different from the one in the message sample, the recipient from the –Recipients parameter will override the recipient in the message sample.  When the admin executes Test-Message with message data and specify sender using –Sender parameter, the value from the message sample will override the value from the parameter –Sender.

The Test-Message cmdlet can work with both the .eml and .msg file formats.

No matter which method is used, reports from Test-Message will be delivered to the email address specified in the -SendReportTo parameter.

Additionally, parameter -TransportRules is required when utilizing Test-Message to define if you want to test Mail Flow Rules.

Here are some examples showcasing how you can utilize Test-Message:

Example 1

In our first example, the ETR utilizes FromScope/SendToScope conditions with values InOrganization/NotInOrganization. The description of the rule is:

If the message:
Is sent to ‘Inside the organization’
and Is received from ‘Outside the organization’
Take the following actions:
Prepend the subject with ‘[EXTERNAL] ‘

Executed Test-Message cmdlet looks like this:

Test-Message -Recipients "briana@contoso.com " -Sender "ari@adatum.com " -SendReportTo "<email-address-of-report-recipient>" -TransportRules

The Test-Message report for this rule looks like this:

Evaluating rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb'
Enter rule 'TestRule1 '
Entering transport rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb'
Checking whether bifurcation is needed on this message
Checking bifurcation criteria for recipient 'briana@contoso.com'.
'SentToScope:InOrganization' evaluation result: True.
Recipient briana@contoso.com matched rule bifurcation criteria.
All recipients on the message matched bifurcation criteria. Fork is not needed.
Evaluating transport rule conditions
Evaluating condition 'FromScope:NotInOrganization'.
Message.Auth property evaluated as: 'FromInternal'
'Predicate' evaluation result: False.
Condition 'Not' evaluation result: True.
Message.From property evaluated as a collection of values: 'ari@adatum.com'
'Predicate' evaluation result: False.
Condition 'FromScope:NotInOrganization' evaluation result: True.
Transport rule evaluation result: True
Execute Actions for rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb'
Rule 'TestRule1 ' matched
Rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb' matched. Executing actions...
Execute Action 'PrependSubject'
Executing action PrependSubject
Finished execution of Actions for rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb'
Exit rule 'TestRule1 '
RuleExit: Exiting rule with id '2b2d88f3-43df-4ce1-b16d-86be84a695eb'

From the report we can see:

Recipient check:

Checking bifurcation criteria for recipient 'briana@contoso.com '.
'SentToScope:InOrganization' evaluation result: True.
Sender check:
Message.From property evaluated as a collection of values: 'ari@adatum.com '
'Predicate' evaluation result: False.
Condition 'FromScope:NotInOrganization' evaluation result: True.

Rule evaluation result:

Transport rule evaluation result: True

As we can see, the report contains properties the rule uses for the evaluation, for both sender and recipient.

The next step would be to check the sender and recipient and confirm whether they are inside the organization or outside the organization. We can see what qualifies the sender/recipient as inside/outside the organization from our documentation.

For this example, it was expected for the rule not to match since the sender’s domain should be in accepted domains. Upon investigation, it was discovered that the sender’s domain was not in the list, which is why the rule ended up matching.

Example 2

In this example, the rule utilizes FromScope and AttachmentExtensionMatchesWords conditions. The description of the rule is:

If the message:
Is received from ‘Outside the organization’
and has an attachment with a file extension that matches one of these values: ‘html’
Take the following actions:
Prepend the subject with ‘[Suspicious!]  ‘

The sender of a message is external sender: ari@adatum.com

Execute Test-Message cmdlet like this:

Test-Message -Recipients "briana@contoso.com" -Sender "ari@adatum.com" -SendReportTo "<email-address-of-report-recipient>" -MessageFileData $data –TransportRules

Where $data is previously defined as:

$data = [System.IO.File]::ReadAllBytes('C:<PathToMessageFile>MessageFile.eml')

The Test-Message report for this rule looks like this:

Enter rule 'TestRule2'
Entering transport rule with id '5ceaa93f-edd1-40bd-a520-02d82d8de73a'
Evaluating transport rule conditions
Evaluating condition 'FromScope:NotInOrganization'.
Message.Auth property evaluated as: 'FromInternal'
'Predicate' evaluation result: False.
Condition 'Not' evaluation result: True.
Message.From property evaluated as a collection of values: 'ari@adatum.com'
'Predicate' evaluation result: False.
Condition 'FromScope:NotInOrganization' evaluation result: True.
Evaluating condition 'AttachmentExtensionMatchesWords'.
Message.AttachmentExtensions property evaluated as a collection of values: 'zip;html'
'AttachmentExtensionMatchesWords' evaluation result: True.
Transport rule evaluation result: True
Execute Actions for rule with id '5ceaa93f-edd1-40bd-a520-02d82d8de73a'
Rule 'TestRule2' matched
Rule with id '5ceaa93f-edd1-40bd-a520-02d82d8de73a' matched. Executing actions...
Execute Action 'PrependSubject'
Executing action PrependSubject
Finished execution of Actions for rule with id '5ceaa93f-edd1-40bd-a520-02d82d8de73a'
Exit rule 'TestRule2'
RuleExit: Exiting rule with id '5ceaa93f-edd1-40bd-a520-02d82d8de73a'

From the report we can see:

Sender check:
Message.From property evaluated as a collection of values: 'ari@adatum.com'
'Predicate' evaluation result: False.
Condition 'FromScope:NotInOrganization' evaluation result: True.

Attachment extensions check:

Message.AttachmentExtensions property evaluated as a collection of values: 'html'
'AttachmentExtensionMatchesWords' evaluation result: True.

Rule evaluation result:

Transport rule evaluation result: True

Looking at the report, it contains the sender address as well as a list of attachment extensions (in this case ‘zip;html‘) that were used for rule evaluation.

In this case, the rule was unexpectedly matched – because the attachment with .html extension was not present in the email client.

Upon investigation using Test-TextExtraction it was confirmed that another attachment contains this attachment embedded – and nested or sub-attachments are inspected by Transport Rules.

Example 3

In this example, the rule utilizes IsMemberOf based condition and exception. The description of the rule is:

If the message:
Is sent to a member of group ‘ Group1@contoso.com ‘
Take the following actions:
Delete the message without notifying the recipient or sender
Except if the message:
Is received from a member of group ‘ Group2@contoso.com ‘

Executed Test-Message message cmdlet looks like this:

Test-Message -Recipients " briana@contoso.com " -Sender " lelia@contoso.com " -SendReportTo "<email-address-of-report-recipient>" -TransportRules

The groups we are inspecting are ‘Group1@contoso.com’ and ‘Group2@contoso.com’

The Test-Message report for this rule looks like this:

Evaluating rule with id '1e545fc0-bdc5-41b7-ae13-e45faf908045'
Enter rule 'TestRule3'
Entering transport rule with id '1e545fc0-bdc5-41b7-ae13-e45faf908045'
Checking whether bifurcation is needed on this message
Checking bifurcation criteria for recipient 'briana@contoso.com'.
Recipient is a member of the group 'Group1@contoso.com'.
SentToMemberOf' evaluation result: True.
Recipient briana@contoso.com matched rule bifurcation criteria.
All recipients on the message matched bifurcation criteria. Fork is not needed.
Evaluating transport rule conditions
Evaluating rule exceptions.
Evaluating 'ExceptIfFromMemberOf'.
Message.From property evaluated as a collection of values: 'lelia@contoso.com'
User 'lelia@contoso.com' is a member of the group 'Group2@contoso.com'.
ExceptIfFromMemberOf' evaluation result: True.
Exception applied.
Condition 'Not' evaluation result: False.
Transport rule evaluation result: False
Exit rule 'TestRule3 '
RuleExit: Exiting rule with id '1e545fc0-bdc5-41b7-ae13-e45faf908045'

From the report we can see:

Recipient IsMemberOf check:

Checking bifurcation criteria for recipient 'briana@contoso.com'.
Recipient is a member of the group 'Group1@contoso.com'.
SentToMemberOf' evaluation result: True.

Sender IsMemberOf check:

Message.From property evaluated as a collection of values: 'lelia@contoso.com'
User 'lelia@contoso.com' is a member of the group 'Group2@contoso.com'.
ExceptIfFromMemberOf' evaluation result: True.

Rule evaluation result:

Transport rule evaluation result: False

The report provides a means to differentiate between condition and exception evaluations.

Additionally, administrators can see the evaluation result of the IsMemberOf condition/exception per group for every sender and every recipient.

Upon investigation, it was determined that sender “lelia@contoso.com” was recently removed from the group and the group cache was not updated. Since the exception match, the rule did not match.

ETR frequent issues

Before running Test-Message cmdlet, there are several things that should be confirmed.

1. Check the rule priority. Check whether there is a triggered rule with a higher priority than the rule you are troubleshooting, whose action(s) can cause a problematic rule to not work as expected.
2. Ensure ~30 minutes have passed since the last rule change.  It can take up to ~30 minutes for rule changes to take effect.
3. Confirm the rule state. Rules must be in an Enabled state to be evaluated.  When rules are created, they are by default in a Disabled state.

Start using Test-Message

If you’re looking for a powerful and convenient way to troubleshoot mail flow rules for your organization, look no further than the Test-Message cmdlet. With full insights into the mail flow rule evaluation process, this tool is an essential part of any tenant administrator’s toolkit. Whether you’re trying to troubleshoot an issue with an existing rule or test a new one you’re creating (in Audit Mode), Test-Message makes it easy to get the job done quickly and efficiently. So, if you want to ensure that your mail flow rules are working as intended, start using Test-Message today and experience the benefits for yourself.

Additional reading:

• Mail flow rules (transport rules) in Exchange Online | Microsoft Learn
• Mail flow rule conditions and exceptions (predicates) in Exchange Online | Microsoft Learn
• Mail flow rule actions in Exchange Online | Microsoft Learn

Milos Nestorovic