Extracting Sender Domains From Raw Email Logs

To learn how to write a regular expression for extracting sender domains from raw email logs, first decide which field represents the sender you trust. A visible From address, an envelope Return-Path, and domains listed in Received hops answer different questions, so extraction should happen only after that choice is documented.

Australian teams encounter this in mixed environments: Sydney and Melbourne offices may run several cloud mail platforms. Logs can combine UTC with AEST and AEDT, and local domains often end in .com.au or .org.au rather than .com.

The regex should return a defensible hostname, preserve the original record for review, and avoid treating pattern matching as proof of authenticity. DKIM, DMARC, alignment and reputation remain separate controls, especially when a compromised legitimate mailbox sends a deceptive message.

Start with representative, de-identified samples and test successes, misses and false matches before deployment. Keep extracted domains in a dedicated field, apply access controls aligned with the Australian Privacy Act, and avoid retaining full addresses unless the security purpose genuinely requires them.

Choosing the Correct Header

The From header names the mailbox shown to the recipient, but it can be written by an unauthorised sender. Return-Path usually reflects the envelope sender handled by the submitting server. If the business rule concerns spoofed display names, use From; if it concerns bounce routing or the envelope path, evaluate Return-Path instead.

Received headers form a hop chain, and their domains describe servers rather than one sender. Restrict the expression to the chosen field name so a line such as Received: ... or text inside a message body cannot become an accidental result. Record the header type beside every extracted value.

Building a Focused Domain Pattern

For a conventional From header containing either a bare address or a display name, this PCRE or Python-style expression captures only the domain in group 1:

(?im)^[ \t]*From:[ \t]*(?:[^<>\r\n]*<)?[^@<>\r\n]+@([A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?(?:\.[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?)+)

Group 1 is the value to store. The label pattern permits letters, digits and internal hyphens, limits each label to 63 characters, and accepts multi-part domains such as .com.au and punycode names. It deliberately excludes domain literals such as [192.0.2.1] and single-label hosts; add explicit branches if those occur. The i flag ignores case and m lets ^ match each log line, while matching all occurrences still depends on the language’s global option.

Testing Against Realistic Log Samples

Create fixtures that mirror the formats seen in Australian mail gateways, including display names, quoted local parts, plus addressing and uppercase domains:

From: Billing Team <billing@example.com.au>
From: alice@example.org.au
From: "Service Desk" <notice@mail.example.com>

The first two lines should yield example.com.au and example.org.au; the third should yield example.com. Also test malformed addresses, body text containing From:, and a header with no dot in the domain. A missing match is often safer than an invented hostname, and unexpected results should be traced to the surrounding syntax rather than fixed by allowing any character.

Handling Wrapped and Structured Logs

Production logs often arrive as JSON strings, syslog records or escaped multiline payloads. Decode the outer format before applying the header expression; otherwise \r\n may remain literal and line anchors will fail. If a timestamp precedes the message, isolate the mail payload rather than weakening ^From: to .*From:, which can match arbitrary body content.

When the extracted domain will help decide whether to process incoming attachments, the guide on automating attachment trust checks shows how trust verification can sit inside that workflow. Keep the domain extraction stage deterministic, then pass the result to reputation, authentication and attachment controls as separate decisions with their own evidence.

Adding Domain Trust Verification

Extraction establishes only what appeared in a selected header. For broader sender analysis, free sender reputation checks from Trusted Sender Score, operated by Zulu Labs Inc., can be combined with DKIM and DMARC inspection to reveal authentication failures, suspicious domains and inconsistent identities. Bulk domain checking supports migration audits, while an API can place the same check inside ticketing, mail gateway or security workflows.

For Australian security teams, this layered triage supports the Australian Signals Directorate’s Essential Eight approach without pretending that one lookup settles an incident. Record the original header, matched domain, verification result, timestamp in UTC and analyst decision. Store only the domain where possible, restrict raw samples to authorised personnel, and treat Privacy Act obligations as part of the logging design.

Deploying and Monitoring the Extraction

Once the pattern is stable, compile it once and retain the surrounding match context needed by your SIEM. In Python, with the expression above compiled as pattern, the basic extraction is:

domains = {
    match.group(1).lower()
    for match in pattern.finditer(raw_log)
}

Create separate compiled expressions when From and Return-Path require different treatment, and label each result with its source header. Monitor match rates and sampled false positives as providers change formats or new mail systems arrive. A domain that fails extraction should remain visibly unmatched rather than being copied from unrelated text, while a domain that extracts cleanly should still undergo normal authentication and reputation checks. That separation keeps the automated pipeline auditable while leaving authentication failures visible.