SMTP server now - Samotop

Hi, I’ve been toying with this study project for a while and I must say I’m kinda excited! When I started, I’ve just about heard of Rust. Now I’m getting the knack of it. Today, I’ve got a somewhat working SMTP server utilizing PEG grammar and async processing built on top of tokio and futures. It only stores emails to the stdout and has the minimal set of commands to receive a mail, but I think it’s a big thing for me at least!

I’d be super happy if someone feels like trying it out or reviewing the code and giving me feedback.

Special thanks to @shepmaster !!! He would jump on every question I asked on SO. I wonder, does he sleep? Thanks a lot.

I’ve got the project in gitlab https://gitlab.com/BrightOpen/BackYard/Samotop with CI running.

Kind regards and many thanks

8 Likes

Hi friends,

Samotop has been collecting dust while waiting for async/await to stabilize. No more excuses, it is moving to async-std 1.6.2 now. WIP - running SMTP server with broken tests, outdated docs, but async at last in https://gitlab.com/BrightOpen/BackYard/Samotop/-/tree/feature/async-std-upgrade

Meanwhile, @azul upgraded the grammar to PEG 0.6 and I've switched to rustls so all libs are at latest versions!

Thank you and feel free to implement your own mail service - example in ConsoleMail

1 Like

Samotop major preview release 0.11.1 brings:

  • Unix domain socket server and client.
  • LMTP server and client.
  • Simple maildir delivery.
  • Rustls as well as native-tls encryption built in for both client and server.
  • Other encryption tools can be hooked in implementing simple traits.
  • Examples! including TCP, Unix socket, command stdio servers, dirmail and LMTP delivery.
  • Improved modularity and cleaner API for extensions - bring it on.
  • Plenty of bug fixes, refactors, simplifications

I'm delighted with how it's taking shape. May it grow stronger... https://crates.io/crates/samotop

3 Likes

Time flies and 0.12.0 preview release is out:

  • New way of handling commands - command pattern
  • NOM based parser through rustyknife - an alternative to default PEG based parser
  • S/MIME encryption of e-mails at rest
  • Abuse protection - command timeout
  • Merged samotop-model into samotop-core
  • Finished crate cleanup and modularization
  • viaspf upgrade to 0.3

Well, you may wonder, what can you do with this?

Server binary / docker image

The published server itself is pretty basic. I want it to be a stable show of the server, so it doesn't expose all the latest improvements. It simply stores all incoming mail in a single flat maildir. It will check SPF, talks STARTTLS and will drop connections for lazy, buggy clients on timeout... I went on a journey of discovery here, trying to figure out if single binary docker images from scratch were possible with Rust, and voila - they are!

Samotop library

Dipping into the code a bit, you can achieve a lot more. Consider the example of storing all incoming messages in maildir encrypted with S/MIME so no-one but you can read it once stored... S/MIME is another major addition in this release. Since Samotop can also listen to LMTP, I imagine someone could hook this up with postfix and encrypt e-mails at rest.


async fn main_fut() -> Result<()> {
    let setup = Setup::from_args();

    let ports = setup.get_service_ports();

    let mail_service = Builder::default()
        .using(Name::new(setup.get_my_name()))
        .using(Accounts::new(setup.absolute_path("accounts")))
        .using(SMimeMail::new(
            setup.get_id_file_path(),
            setup.get_cert_file_path(),
        ))
        .using(Dir::new(setup.get_mail_dir())?)
        .using(samotop::mail::spf::provide_viaspf())
        .using(SmtpParser::default())
        .using(RustlsProvider::from(TlsAcceptor::from(
            setup.get_tls_config().await?,
        )));

    let smtp_service = SmtpService::new(Arc::new(mail_service));

    info!("I am {}", setup.get_my_name());
    TcpServer::on_all(ports).serve(smtp_service).await
}

Process your mail with a simple IO write implementation

You can implement your own mail dispatch and deal with the message yourself. The traits for that:

pub trait MailDispatch: fmt::Debug {
    fn send_mail<'a, 's, 'f>(
        &'a self,
        session: &'s SessionInfo,
        transaction: Transaction,
    ) -> S2Fut<'f, DispatchResult>
    where
        'a: 'f,
        's: 'f;
}

The task is to put an AsyncWrite sink into the Transaction struct. See the debug mail service for bare basics example.

Forward the mail to some other service

It is also fairly straightforward to forward the mail immediately (no queueing implemented yet) to another service. TCP and Unix socket transports can be used with LMTP. See the examples. Here I pass the mails to dovecot over unix socket mapping recipients to an account per domain:

async fn main_fut() -> Result<()> {
    let rcpt_map = Mapper::new(vec![
        (Regex::new(".*@(.*)")?, "$1@localhost".to_owned()), // use domain as a user name (all domain basket) anyone@example.org => example.org@localhost
        (Regex::new("[^@a-zA-Z0-9]+")?, "-".to_owned()), // sanitize the user name example.org@localhost => example-org@localhost
    ]);
    use samotop::io::client::UnixConnector;
    let lmtp_connector: UnixConnector<NoTls> = UnixConnector::default();
    let mail_service = Builder::default()
        .using(LmtpDispatch::new("/var/run/dovecot/lmtp".to_owned(), lmtp_connector)?.reuse(0))
        .using(rcpt_map)
        .using(SmtpParser::default());
    let smtp_service = SmtpService::new(Arc::new(mail_service));

    TcpServer::on("localhost:2525").serve(smtp_service).await
}

And while the Connector is not implemented yet, you could likewise start and use a child process that speaks LMTP on stdio.

What's next?

My vision is that we the people are in charge of our data and communication. Not just as implied by GDPR and similar big talk, but physically, factually. Powerful business players worked hard to centralize mail services, chats and social networks and gave that to governments, leaked it to unauthorized parties... It's not all lost. E-mail is still ubiquitous and can be leveraged for a seamless opt out.

The bigger mission is to

  1. deliver an SMTP experience, that is easy to provision, subscribe and use by non-technical people with utter focus on privacy and abuse prevention.
  2. enable instant interaction, like or better than chat experience with pending contacts, networking features, integration to open social networks and more.
  3. explore new ways of working with mail - step away from the tried and tested inbox concept and explore AI, visualization, information discovery.
  4. explore the options of improving e-mail infrastructure - specifically address the stuffy security and privacy concerns

I'd like to do to e-mail servers what delta.chat does to e-mail clients :slight_smile:

The Immediate mission is to make a useful contribution, something people can use to solve their problems, something that enchants the Rust community to build a samotop user base to inspire, give feedback and contribute. If you are using samotop, I do want to hear from you :slight_smile: and about your use cases.

3 Likes

Meanwhile, SaMoToP got a new logo :slight_smile: and has been moved from the BackYard to the front of the shop:

samotop

I'm currently working on special storage format which aims at being efficient as well as suitable for privacy. The idea is to write down opaque blocks (encrypted parts of email bodies and metadata) into a journal. Then for each recipient (and the associated public key) write down a reference to the blocks that form the mail, and the symmetric encryption key, all encrypted for the recipient's public key. This way, an email can be written down as it is received from the wire without popping it all into memory, immediately encrypting it so no plaintext is stored on disk and allowing multiple recipients to read it simply proving the ownership of the relevant private key. This is much different from S/MIME though the objective is similar. The new approach completely hides all sensitive information from uninvited eyes. The only thing known to the server after the fact of storing an incoming message is that it can be read by an owner of a certain private key that matches a known public key. Other metadata, such as sender, recipient address, date, subject... are obscured.

I have a draft of this working in plaintext for now merged to develop. A new protocol was born to support it - lozizol. Next step is encryption. Wish me good luck :slight_smile:

1 Like