Email verification using DMARC - Domain-based Message Authentication, Reporting and Confor

This article follows up on the previous ones

• SMTP protocol and e-mail addresses
• Email verification using SPF - Sender Policy Framework
• Email verification using DKIM - DomainKeys Identified Mail

Domain-based Message Authentication, Reporting, and Conformance (DMARC)

The Domain-based Message Authentication, Reporting, and Conformance (DMARC) method is described in the informational RFC 7489. It is a mechanism by which an email-sending organization (email domain, Domain Owner) can specify policies and preferences for message authentication verification and reporting at the domain level, which an email-receiving organization (Mail Receiver) can use to improve mail processing. The recipient can provide feedback (reports) to the domain owner about the use of their domain (showing legitimate use and abuse). DMARC is a mechanism for distributing policies that enable stricter processing of messages where authentication verification has failed.

DMARC Principle

DMARC extends and complements the existing SPF and DKIM techniques, which perform email domain verification, but do not contain the ability to tell the recipient how to handle messages where the verification has failed. They also lack the ability to inform the domain owner how the received messages were handled. These capabilities are offered by DMARC. The sender's address is found in the SMTP command MAIL FROM (envelope) according to RFC 5321 and in the From header field according to RFC 5322. DMARC verifies the domain from the From address.

Again, a text record in DNS is used to define the DMARC policy. For your sending domain, it is sufficient to publish the appropriate record (it is also recommended to create a mailbox for receiving reports) and thus activate DMARC. To perform checks and reactions on receipt, we need a mail gateway that supports DMARC and activate this function.

DMARC separates the authentication technology from the policy mechanism. Thanks to the fact that it can use multiple authentication technologies (SPF and DKIM), it is more resilient to errors. If we use both technologies (which is recommended), it is sufficient for one check to pass (Pass) and the other to fail (Fail) for the DMARC check to pass. The primary identifier is the domain from the From header field of the message. And in addition to the SPF and DKIM checks, it is also compared that this identifier corresponds to the domain verified using SPF and DKIM. If it matches, it is marked as Identifier Alignment (or SPF Alignment and DKIM Alignment).

DMARC allows the receiving server to generate two types of reports. One for each message where verification has failed (Failure Reports). The second is a regular aggregated report to the domain owner (Aggregate Reports).

Identifier Alignment

SPF verifies the domain used in the SMTP command MAIL FROM, or EHLO/HELO or both. DKIM verifies the domain that signed the message. In both cases, these are domains that the average user does not see and may be different from the displayed sender domain.

DMARC requires that the verified identifier (Authenticated Identifier) from SPF or DKIM match (be aligned with) the From address from the header, which is the one normally visible to the user. This is referred to as identifier alignment (Identifier Alignment). According to RFC 5322, the From header field is mandatory and is typically used to represent the source of the message. This field is also the most common target for abuse. DMARC assumes that the incoming message is valid according to RFC 5322. If, for example, the From field is missing, damaged, or repeated, DMARC does not address such a situation.

Alignment (Alignment) of the domain can be defined as Strict, then it must match exactly. Or Relaxed, where only the organization's main domain (Organizational Domain) is checked, so subdomains are then considered valid as well.

Aggregate Reports

The DMARC aggregate feedback report is sent periodically (typically daily) by the recipient for the sending domain that has a DMARC policy defined with a RUA address. It contains aggregated data on all messages received from the given domain. The domain owner thus gets a statistical overview of what legitimate messages they are sending, what the result of verifying these messages was, and what spoofed emails the recipient is receiving. The report contains information about the organization, the policy found and applied, records for messages containing the results of SPF and DKIM checks, the identifier, and whether it is aligned.

The XML format of the reports is defined in Appendix C of RFC 7489. It is recommended to compress the file using GZIP. The extension is then either xml or xml.gz. The filename and email subject are specified in the RFC.

Example email subject:

Report Domain: sender.com Submitter: recipient.com Report-ID: <ID-1-1-0>

Example report filename (the numbers are the start and end timestamps of the report in UNIX format):

recipient.com!sender.com!1580252403!1580338803.xml.gz

The content of the aggregated reports consists only of statistical data at the domain level, we certainly won't find out that a message from a certain email address to another has passed/failed verification.

Roughly, the report content consists of

• version - the version, which must be 1.0
• report_metadata - metadata about the report creator (organization) and the period for which the report is issued
• policy_published - the applied DMARC policy, contains the domain where the policy was found and policy data - required alignment, policy for domain and subdomains, percentages and failure reporting
• record - one or more records containing the authentication result for a group of messages
  • row - contains the source IP address, number of matching messages, policy evaluation (SPF and DKIM result with DMARC domain alignment, policy-specified action, or reason)
  • identifiers - identifiers header_from, envelope_from, envelope_to (in practice, only the domain from the From header is usually present here)
  • auth_results - SPF and DKIM authentication results without DMARC influence, DKIM may be present 0 or more times, SPF at least once, each part contains the verified domain, the result of the check, the selector for DKIM and for SPF, whether it is mfrom or helo

Example Aggregate Report

<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
    <version>1.0</version>
    <report_metadata>
        <org_name>seznam.cz a.s.</org_name>
        <email>abuse@seznam.cz</email>
        <report_id>szn_firma.cz-2020-01-30</report_id>
        <date_range>
            <begin>1580342400</begin>
            <end>1580428800</end>
        </date_range>
    </report_metadata>
    <policy_published>
        <domain>firma.cz</domain>
        <adkim>r</adkim>
        <aspf>r</aspf>
        <p>none</p>
        <pct>100</pct>
        <fo>0</fo>
    </policy_published>
    <record>
        <row>
            <source_ip>1.2.3.4</source_ip>
            <count>61</count>
            <policy_evaluated>
                <disposition>none</disposition>
                <dkim>pass</dkim>
                <spf>pass</spf>
            </policy_evaluated>
        </row>
        <identifiers>
            <envelope_from>firma.cz</envelope_from>
            <header_from>firma.cz</header_from>
        </identifiers>
        <auth_results>
             <dkim>
                <domain>firma.cz</domain>
                <result>pass</result>
                <selector>dkim2020</selector>
            </dkim>
             <spf>
                <domain>firma.cz</domain>
                <scope>mfrom</scope>
                <result>pass</result>
            </spf>
        </auth_results>
    </record>
</feedback>

Failure Reports

If DMARC authentication fails (resulting in a Fail) and the policy defines a RUF address, then a Failure Report is immediately sent for the given message. This provides the domain owner with a quick notification that an authentication failure has occurred. The report contains more information than the aggregated report, so it provides support for investigating the cause of the failure.

If the domain owner wants to receive these reports, they define the RUF and optionally FO tags in the policy. If the recipient is willing to provide the reports, they generate and send them in the AFRF (Authentication Failure Reporting Using the Abuse Reporting Format) format.

DMARC Verification by the Recipient (Server)

When a message is received, the following steps are performed (steps 2 to 4 occur in parallel):

1. Obtain the domain from the From header field
2. Search for the DMARC policy record in DNS, continue if a valid record is found
3. Check the DKIM signatures (there may be multiple), pass the verification result and domain to DMARC
4. Check the SPF verification, pass the verification result and domain to DMARC
5. Perform the Identifier Alignment check, if one or more verified identifiers are aligned with the From domain, the message is considered compliant (Pass) with the DMARC check, otherwise the check failed (Fail)
6. For messages where the check failed, the DMARC policy is applied

DMARC Policy Record in DNS

The DMARC Policy Record is published in DNS as an TXT (type 16) Resource Record (RR) on the subdomain _dmarc. For example, _dmarc.firma.cz. The record format is similar to DKIM, consisting of tags (tag=value) separated by semicolons.

To specify the recipient addresses for the two types of reports (in the DMARC record), the DMARC URI format is used. It is a mailto URI scheme where commas and exclamation marks are encoded, and optionally may contain an exclamation mark at the end and a maximum report size. Example: mailto:reports@firma.cz!50m. Multiple addresses can be separated by commas.

By default, the address for receiving reports must be on the same domain. If we want to use an external domain, there must be a DNS TXT record for DMARC in the target domain, indicating consent to receive reports. Otherwise, someone could try an attack by directing a large number of reports to an external domain (by publishing a DNS record and then sending many forged messages to various domains).

The name (FQDN) where the TXT record with the value v=DMARC1 is searched for is created based on both domains. For example, we have the domain firma.cz (where the DMARC policy is), but we want to send the reports to the domain prijemce.cz (where the rua/ruf addresses are directed). Then the record is searched for:

firma.cz._report._dmarc.prijemce.cz

Tags Used in the DMARC Record

(P) means that the given tag must be present in the record (it's mandatory).

• v - version (P) - identifies the DMARC record, must be first with the value DMARC1
• p - policy (P) - the policy defined by the domain owner, to be used by the recipient, none no action, quarantine if DMARC fails, the message should be treated as suspicious and quarantined or marked, reject if DMARC fails, the message should be rejected (this can be during the SMTP transaction)
• rua - address aggregate report - addresses to which the aggregated report should be sent, in DMARC URI format
• ruf - address message-specific report - addresses to which the report should be sent in case of message failure, in DMARC URI format
• fo - failure reporting - what failure reports should be generated (must also be defined in ruf), 0 generate DMARC report if all authentication methods fail (default), 1 generate DMARC report if any authentication method fails, d generate DKIM report if the signature is invalid (regardless of alignment), s generate SPF report if SPF fails (regardless of alignment)
• adkim - DKIM Alignment mode - r relaxed mode (default), s strict mode
• aspf - SPF Alignment mode - r relaxed mode (default), s strict mode
• pct - percentage - the DMARC policy should be applied to only a certain percentage of messages, 0 to 100 (default)
• rf - message-specific reports format - which format should be used for the report, only afrf is supported
• ri - report interval - how often the aggregated reports are sent, default 86400
• sp - subdomains policy - the policy (same as p) that applies to subdomains (not the main domain)

Example DNS Record

"v=DMARC1; p=none; rua=mailto:dmarcreports@firma.cz; ruf=mailto:dmarcreports@firma.cz"

To check the DMARC record in DNS, we can use online services, such as the DMARC Check Tool - Check DMARC Records for Errors, or the DMARC record check.

To create the record, we can also find an online generator like the DMARC Record Generator.

Practical DMARC Deployment

DMARC Verification

We need an email gateway (server) that supports DMARC. Enabling the verification is then usually straightforward. An example of a gateway with DMARC support is Cisco Email Security. I briefly described the activation of DMARC checking in the article Cisco Email Security - AntiSpam Solution Configuration. We can change the behavior if the check fails and the DMARC policy defines certain behavior. We also set whether aggregated reports should be sent.

In very simplified terms, DMARC performs

• check SPF and compare the domain from MAIL FROM (SPF) and From, if it passes, it returns Pass, if SPF is not used, it returns Fail
• check DKIM and compare the domain from DKIM-Signature (tag d) and From, if it passes, it returns Pass, if DKIM is not used, it returns Fail
• if at least one result is *Pass*, the entire DMARC check is Pass, otherwise it is Fail and the policy is applied
• sending reports

Enabling DMARC

When we want to deploy DMARC, the recommended general steps are:

• Deploy DKIM and SPF
• Ensure the identifiers are properly aligned (domains) when sending
• Publish the DMARC record with a none policy and report addresses to gather information about message processing and not impact message delivery
• Analyze the reports
• Adjust the DMARC policy to quarantine or reject

The actual activation of DMARC for a specific domain involves publishing a text DNS record. If we send emails from the domain firma.cz, for example bouska@firma.cz, we create a TXT record _dmarc on the firma.cz domain. (FQDN _dmarc.firma.cz). The record value may be (or without the RUF tag if too many messages are received)

v=DMARC1; p=none; rua=mailto:dmarcreports@firma.cz; ruf=mailto:dmarcreports@firma.cz

Which we later change to

v=DMARC1; p=quarantine; rua=mailto:dmarcreports@firma.cz; ruf=mailto:dmarcreports@firma.cz; adkim=s; aspf=s

We can inspect the record easily, for example using a command line on Windows.

nslookup

> set type=txt
> _dmarc.firma.cz
Server:  ns.firma.cz
Address:  192.168.0.10

Non-authoritative answer:
_dmarc.firma.cz   text =
	"v=DMARC1; p=none; rua=mailto:dmarcreports@firma.cz; ruf=mailto:dmarcreports@firma.cz"

Practical Problems

The DMARC technology is interesting and quite widespread. I would say that many organizations have published a DMARC record, but many of them have the policy set to NONE. And it's clear that this is a good thing, because they have fundamental flaws in their email sending and the DMARC check fails. This affects even large organizations like the Ministry of Finance of the Czech Republic, who send official emails from a subdomain where they don't use SPF or DKIM (or DMARC), but on the main domain they have a defined DMARC record (which is also used for all subdomains). Some companies have a DMARC record, but don't use SPF or DKIM.

Another common problem is with sending reports. Many organizations have defined an address for aggregated reports, but it either doesn't exist or the mailbox is full. Again, this affects large companies like Fortinet, Aliexpress, Teamviewer, Čedok. It's even worse when the service that provides DMARC report processing can't deliver the report, like easydmarc.com, dmarcanalyzer.com (perhaps this happens when a company stops paying for the service, but the DMARC record is still directed to this service).

Another amusing situation is on the domain isaca.org (which everyone should be aware of when deploying DMARC). Delivery of the aggregated report fails because the address dmarc_rua@isaca.org doesn't exist. The server responds with an NDR about non-delivery (Bounce Message), but the message is not signed and the sender address (Envelope Sender) is missing. The domain has a DMARC policy set to quarantine, the DMARC check fails, so the message is not delivered.

When DMARC Fails

For SPF, the problem is forwarding emails (Redirect) to another address, if the message doesn't change (sender information). In this case, the sender's domain is the original one, but the message is sent by another mail server. This problem should not affect DKIM, because the message remains the original, so the signature should also be valid. DMARC was designed to check SPF and DKIM, but one successful check was enough. It should therefore be functional when forwarding messages, because DKIM is verified, but the sender's domain is also checked.

When forwarding, some form of sender address rewriting in the envelope (MAIL FROM) is often used, such as SRS. The goal is to pass the SPF check. However, the original address remains in the header (From), so the domains are not aligned and the DMARC SPF check fails.

In practice, however, there are situations (which are considered legitimate) where the DMARC check fails. For example, membership in a Mailing List. Such a service accepts emails that it then distributes to all group members. However, it slightly modifies the message content, for example by adding a footer, but does not change the sender information. So neither SPF nor DKIM signature passes. The same applies to some automatic forwarding methods.

I analyzed some situations where forwarded (Redirect) emails did not pass the DMARC check. The SPF alignment did not match, which is logical, but the DKIM signature was also not in order. A colleague found the reason. Some companies (such as Facebook) sign messages with simple canonicalization. And certain mail servers, such as those from Microsoft, perform certain modifications of non-printable characters when forwarding messages. Then the checksum does not match. For this reason, it is recommended to use the more lenient relaxed canonicalization.

Authenticated Received Chain (ARC)

In aggregate reports from google.com, I occasionally came across information that a local policy was applied for the decision with the note arc=pass. For this, I found that it is the Authenticated Received Chain (ARC) RFC 8617, which should solve the previous problem. It is a protocol that passes the authentication result to the next step when forwarding. If a forwarding service receives a message where DMARC is valid and supports ARC, then when forwarding, it adds additional header items (ARC-Authentication-Results, ARC-Seal, ARC-Message-Signature). Thanks to this, the message retains the original verification result. The receiving server verifies the ARC-Seal header chain and the last ARC-Message-Signature. If it's in order, it should use the original authentication result.

Processing Reports

In addition to DMARC protecting the domain so that no one can impersonate it, it also offers the possibility of reports that provide an overview of email sending. Sending reports is, however, an optional feature according to the RFC, and the Mail Receiver may not perform it. From my experience, it seems that Failure Reports are sent minimally (so far I've only received them from the list), perhaps Cisco Email Security doesn't support it either. Aggregate Reports are sent in some way, but probably in limited numbers (I only receive them from a few large organizations like Google, Seznam, Amazon, Yahoo, IOL).

In any case, if we activate DMARC on a domain, the reports can be interesting, but they need to be processed. This can't be done manually when dozens to hundreds of XML files arrive every day. At a minimum, it requires creating a script (e.g. in PowerShell) that automatically downloads XML files, parses them, and stores them either in a database or in Excel. A better solution is a tool that also performs some analysis. There are several paid online services that process DMARC reports.

The reports should show us if messages are being sent for our domain from public services, for example for marketing. Or if there is message forwarding where the DMARC verification fails. However, it is not easy to analyze the situation from the data in the reports.

Online Tools

List of tools and services I found on the internet.

Domain Check

• Mail Tester
• Domain Health Checker

Create DMARC Record

• DMARC Record Generator
• DMARC Record Generator

Check DMARC Record

• DMARC Check Tool - Check DMARC Records for Errors
• DMARC record check
• DMARC Record Checker
• DMARC Check

View DMARC Aggregate Report XML

• Dmarc Report Analyzer
• DMARC XML to Human Converter

Scripts (Applications)

• DMARC - Harvest DMARC Report files from mailbox and generate HTML Reports v2.3
• Open source DMARC report analyzer and visualizer
• DMARC viewer

Services for DMARC Report Processing (Analysis)

Additional links are provided on the dmarc.org website Analytics and Implementation Support.

Testing DMARC Services

I briefly tried some of the solutions that process DMARC reports. It's always necessary to add (or replace) the RUA address in the DMARC record, and possibly also the RUF. The address is usually generated for the client.

• Postmark DMARC report - sends a summary report once a week (on Monday), contains basic statistics divided into own sources and forwarded sources
• EasyDMARC - the interface doesn't look bad, the basic things are free, but even in the initial Trial period, their service doesn't work well, they seem to have overloaded servers, so 2/3 of the reports are rejected (message exceeds mailbox size), so the result is not good
• URIports - again, it doesn't look bad at first glance, in addition to DMARC it supports TLS-RPT monitoring and then some functions for websites, but I don't understand anything in the DMARC reports, compared to EasyDMARC it seems completely unclear to me, often when I click on some report detail, it returns a blank page - they contacted me themselves and found that this was a bug they fixed, when I went through it now, it looked better (one has to figure out that the details are accessed by clicking Inspect)
• OnDMARC - another service with a nice interface, more services, but for me the reports are unclear, to get to the details of which domain is in SPF and DKIM, you can probably only find it somewhere and again in an unclear way for me