Integration testing strategies for sending emails via SMTP

help
1

I'm writing a Rust microservice to render a contact form, accept submissions, handle some custom business logic that's not relevant to this post, and ultimately send out emails. I'm using lettre to render and send the emails via SMTP, but I'm not sure how best to write integration tests covering my lettre code. I would love any advice on this. Thank you in advance!

I'll include my thoughts on how to solve it below, but in XY problem terms, the above is the Y and everything below is the possibly unhelpful X. The options below are so divergent that I thought it would be wise to ask for help before picking a path. I'm leaning toward idea 3, since it's pure Rust, but I'll certainly admit that idea 1 might be a lot less work.

Idea 1: follow lettre's lead

I notice in the Testing section of the lettre README that lettre's own tests use Python's smtpd, specifically the DebuggingServer class. However, the smtpd module has been deprecated and removed. Python recommends switching to aiosmtpd, which appears to lack equivalent functionality. (I have only spent a few minutes looking; it's very possible that I might have missed something.)

I'm fairly uncomfortable with the idea of depending on Python code for my Rust tests, though I'm willing to do so if it's the best way. I'm even more uncomfortable with the idea of deliberately running an outdated version of Python for this purpose. Having said that, I'm already running Postgres via a container, so I can always build a container to isolate the now-legacy Python code, if that's the best solution.

Idea 2: containerized email server

I know email infrastructure terminology just well enough to know that "server" might not quite be the right term but not well enough to be sure of the right term. (MTA?) In any case, I could conceivably run something like Exim, Postfix, etc. in a container; configure it with testing credentials; send emails there via lettre; and check it using one of the methods below.

Of course, this whole idea seems like it's probably about ten times more complex than necessary, which seems to point me back toward containerized Python 3.11 with smtpd.

Option 2(a): IMAP client

I could use a crate like async-imap or email-lib to check the emails.

Option 2(b): POP3 client

I am not aware of any POP3 client crates, but POP3 is so simple that I'm guessing I could implement the necessary subset just fine. I'm assuming the biggest headaches will revolve around security.

Idea 3: Rust crates mailin, mailin-embedded, samotop, or mailtutan

Of these, mailin-embedded seems like it's probably the most obvious choice:

  1. Implement a mailin_embedded::Handler that accepts emails to the satisfaction of lettre, possibly parses them, and sends values back to the test via a channel (probably a tokio::mpsc variant)
  2. In integration tests, start the server in its own thread (presumably via tokio::task::spawn_blocking) and poll for server start
  3. Send emails, await data from channels (perhaps with timeout), and assert received values
2

I've never used it in anger, but aiosmtpd has a built-in debugging handler class, which sounds similar to what you'd use in the smtpd module. Perhaps you can use it for your tests.

Note that SMTP delivery tests are (IMO) not sufficient to fully verify mailing functionality; you also need bounce message handling, and that's difficult to provide in a self-contained SMTP server mock.

3

Thank you for linking to that aiosmtpd class! In hindsight, I think I know how I missed it.

Regarding email bounce messages, I now realize my question was a bit ambiguous. My microservice acts as a client to an email inbox from a commercial provider, i.e. I provide credentials just as if I were sending via Outlook or similar.

When sending emails, lettre only returns Ok(...) if the server's responses to all communications are positive. (To get into way too much detail, send calls command and message, both of which pass on the return value from read_response, which only returns Ok(...) if the server's response is positive.)

My understanding is that my email provider is responsible for handling rejections from remote servers and should do something like place an email in my inbox to notify me of such. (Also, the contact form only ever emails my own inboxes.) I hadn't considered my bounce strategy in this much detail, so thank you for prompting me to do so! Between lettre and my email provider, I think I have bounces handled, unless I'm mistaken?

4

Right, but an email in your inbox is not automatically a signal to your mailing service -- you need to integrate the two through some sort of action triggered by receipt of a bounce. (If you plan to automate bounce handling, that is, and you should.) You also need a reliable identification of the bounced message recipient, using something like VERP. Don't boot the recipient on first failed delivery, since transient faults are common. Etc. All this is a bit finicky to set up and difficult to test in a CI testing environment.

5

Ah. I see what you mean. Yep, I completely agree: under normal business conditions, you're totally right!

This happens to be a tiny, very low volume contact form for a soup kitchen where I will personally receive the bounce notification email (if one ever occurs), where it will be trivial to identify a bounce in several ways, and where the email recipient is always going to be an inbox under my own control. I'm fully confident that this will be completely sufficient for bounce handling in this particular case. Thank you very much for pressing me on the subject, though. If this were a nontrivial or higher-volume application, I would definitely use VERP or similar and automate bounce handling.