## Intro

For quite some time, e-mail seemed like an anomaly in my workflow. I am a long time Gmail user, and my decade-old account has a somewhat formidable quantity of labels and filters. My messages are often assigned multiple labels, and I also like to keep only a bunch of messages in the inbox.

Although, in my opinion, Gmail web UI was and still is leagues ahead of many of its competitors and even allows keyboard-centric workflow, it’s awkward to use with a keyboard-driven browser, and for no money on Earth I would enable browser notifications.

Any classical IMAP/SMTP client is hard to use in my case, because a message with multiple labels is copied to IMAP folders for each of the label plus the inbox folder, and the copies look like different messages from the client-side. For example, a message can be read in one label and unread in another.

For a few years, my solution was Mailspring, which provides first-class support for labels. However, it has a feature to deploy spy pixels on emails (and offers no protection from them, obviously), the client is Electron-based with a mouse-driven interface, and the sync engine was closed-source at the time.

So, I found an alternative in Emacs+notmuch+lieer and ditched one more proprietary app (the last big one I can’t let go of is DataGrip).

Notmuch’s tags are just as advanced as Gmail’s labels, so I have basically the same mail structure accessible from Emacs, Gmail Android client and even the web UI when I don’t have access to the first two.

Also, I think the setup I describe here is pretty straightforward and less complex than many I encountered, but my impression is not the most reliable source of such knowledge.

In any case, what follows is a description of my current workflow with instructions of varying levels of precision of how to get there.

## Setting up

### Gmail

Before we start, some setup is required for the Gmail account.

First, as there is no way to enable SMTP without IMAP on Gmail, you have to set “Enable IMAP” in the “Forwarding and POP/IMAP” tab in the settings. If you use two-factor auth, generate an app password.

Also, make sure your labels do not contain whitespaces because if they do, you will have to type them in quotes all the time.

### lieer

lieer (formerly gmailieer) is a program that uses Gmail API to download email and synchronize Gmail labels with notmuch tags. Because of its usage of Gmail API instead of IMAP, there are no problems with duplicating emails in different labels, etc.

As I need to use multiple versions of Python & Node.js for other reasons, I manage my installations of them with Anaconda (Miniconda, to be precise). You may instead use venv or even the system-wide installation of Python and omit the `conda` clauses, but in my experience Anaconda makes life easier in that regard.

    # Create an environment with the name "mail"
    conda create --name mail
    # Activate the environment
    conda activate mail
    # Install Python
    conda install python
    # Download and install lieer
    git clone https://github.com/gauteh/lieer.git
    cd lieer
    pip install .

After which we may check if the `gmi` executable is available:

    which gmi

### Notmuch

Notmuch is present in most of the package repositories, so you can install it with your package manager, which is `pacman` in my case.

    sudo pacman -S notmuch

After the installation, run `notmuch setup`. That will inquire the parameters and create the `.notmuch-config` file with the answers.

    Your full name [Pavel]: Pavel Korytov
    Your primary email address [pavel@pdsk.(none)]: thexcloud@gmail.com
    Additional email address [Press 'Enter' if none]:
    Top-level directory of your email archive [/home/pavel/mail]: /home/pavel/Mail
    Tags to apply to all new messages (separated by spaces) [unread inbox]: new
    Tags to exclude when searching messages (separated by spaces) [deleted spam]:

It is important to set the `new` tag for the new messages instead of the default `unread` and `inbox`.

Next, add the rule to ignore JSON files to the `[new]` section of the `.notmuch-config` file, so it would look like this:

    [new]
    tags=new
    ignore=/.*[.](json|lock|bak)$/

That is needed to ignore the lieer config files. Although, as I have noticed, notmuch is generally pretty good at detecting wrong files in its directories, an explicit ignore rule won’t hurt.

Now, create the mail directory and run the notmuch new command. As notmuch has probably already noticed you, it uses the maildir format, which basically means that one message is stored in one file.

    # The same directory mentioned in the 4th question
    mkdir ~/Mail
    # Initialize notmuch
    notmuch new

### Add an account

After that, we can create a directory for a mail account and initialize lieer.

    cd ~/Mail
    # Use whatever name you want
    mkdir thexcloud
    cd thexcloud
    # Intialize lieer
    gmi init thexcloud@gmail.com

Running `gmi init` will run an OAuth authentication to your Gmail account. The credentials will be stored in `.credentials.gmailieer.json` file, so make sure not to expose it somewhere.

We also can add a few settings for lieer, which will make life easier. First, dots seem to be less awkward to type than slashes for the nested tags:

    gmi set --replace-slash-with-dot

Then, we don’t want the `new` tag to be pushed back to Gmail

    gmi set --ignore-tags-local new

Now we can finally download the mail directory. To initiate the download, run

    gmi sync

The first download can easily take several hours, depending on the size of your email and the speed of your internet connection, but subsequent runs will be much faster.

The last thing to do here is to add the `gmi sync` command to notmuch’s pre-new hook, so that the email will be synchronized on the `notmuch new` command.

    # Create the hooks folder
    mkdir -p ~/Mail/.notmuch/hooks
    # Create the file
    cd ~/Mail/.notmuch/hooks
    cat > pre-new <<EOF
    #!/bin/bash
    eval "$(conda shell.bash hook)"
    conda activate mail
    (cd /home/pavel/Mail/thexcloud/ && gmi sync)
    EOF
    chmod +x pre-new

Side note: as a hook for `conda` tends to be rather slow, I run the `gmi` command with system-wide Python as follows:

    #!/bin/bash
    GMI="/home/pavel/Programs/miniconda3/envs/mail/bin/gmi"
    (cd /home/pavel/Mail/thexcloud/ && $GMI sync)

Which doesn’t seem to cause any particular trouble in that case.

### Emacs

There are plenty of different frontends for notmuch (even GUI apps), but the one I’m sticking with the Emacs.

Configuration for Emacs is pretty straightforward, but you probably want to use the notmuch package which came with the system package, because otherwise, you may end up with different versions of frontend and backend.

That’s how it can be done with `use-package`:

    (use-package notmuch
      :ensure nil
      :commands (notmuch)
      :config
      (add-hook 'notmuch-hello-mode-hook
                (lambda () (display-line-numbers-mode 0))))

The only notable observation here is that `display-line-numbers-mode` seems to break formatting of the `notmuch-hello` page.

If you use evil-mode, you also should enable the evil-collection mode for notmuch.

Now run `M-x notmuch` and the `notmuch-hello` page should appear. Running `notmuch-poll-and-refresh-this-buffer` (`gR` with evil bindings) will run the `notmuch new` command and refresh the buffer. All the syncronized messages should be present.

I should note that notmuch frontend for Emacs is the most user-friendly Emacs app I have seen so far. UI, commands and keybindings are self-descriptive, all the options can be configured with the build-in `customize` interface. It may be useful to look through emacs tips at the official site and notmuch man pages, in particular syntax for notmuch queries.

### Reading mail

`notmuch-search-show-thread` (`RET`) opens the thread under the cursor.

`notmuch-show-view-part` (`. v` with evil) opens an attachment with associations defined in .mailcap file. Mine looks like this:

    audio/*; mpc add %s

    image/*; feh %s

    application/msword; /usr/bin/xdg-open %s
    application/pdf; zathura %s
    application/postscript ; zathura %s

    text/html; /usr/bin/xdg-open %s

Here watch out for the last line, default version of which may be set as follows:

    text/html; /usr/bin/xdg-open %s ; copiousoutput

Which causes a temporary file to be deleted before it could be opened because recent versions of `xdg-open` do not block the input.

As expected, Emacs mail reader does not trigger any spy pixels or other tracking contents of email (not any I know of, at least). However, opening an HTML email in a browser will even run embedded JavaScript. Therefore, in no case open emails you do not trust with `xdg-open`. Even if you use NoScript, the browser will still load all the CSS, videos and even iframes, which can be used to track you.

Even Gmail web UI is preferable to view the message in a browser, because the former blocks most of the malicious stuff and does not seem to leak your IP to the sender, for what it’s worth.

### Sending mail

To start composing a message, run `notmuch-mua-new-mail` (`C` with evil bindings).

After doing so, `C-c C-c` will run `notmuch-mua-send-and-exit`, which will invoke the function stated in the `message-send-mail-function` variable. The default value of the variable is `sendmail-query-once`, which will inquire the parameters and save them as custom variables.

If SMTP is used, `send-mail-function` will be set to the one from the built-it smtpmail package. SMTP parameters for Gmail are listed here.

Authorization parameters will be saved to your authinfo file. If you didn’t have one, the plaintext `.authinfo` will be created, so it’s reasonable to encrypt it:

    cd ~
    gpg -o .authinfo.gpg -c --cipher-algo AES256 .authinfo

However, if you plan to use multiple accounts with different SMTP servers, it makes more sense to use something like MSMTP to manage multiple accounts. Here are a couple of examples (1, 2) how to do that.

Another alternative for Gmail is to use lieer as sendmail program. That may make sense if you don’t want to enable IMAP and SMTP on your account.

There are also a bunch of ways to set up address completion if the built-in completion based on notmuch database does not suffice.

I also use LanguageTool for Emacs to do a spell checking of important emails (integrations like that really make Emacs shine). For some reason, developers don’t give a link to download the server on the frontpage, so here it is. And here is the relevant part of my Emacs config:

    (use-package langtool
      :straight t
      :commands (langtool-check)
      :config
      (setq langtool-language-tool-server-jar "/home/pavel/Programs/LanguageTool-5.1/languagetool-server.jar")
      (setq langtool-mother-tongue "ru"))

As a last note here, to set up a signature create the `.signature` file in the `$HOME` directory. If you need more complex logic here, for instance, different signatures for different accounts, you can put an arbitrary expression to the `mail-signature` variable or apply this gnus-alias tip.

## Another account

### Adding an account

Now we can send and receive mail from one account. Adding another account is also pretty easy.

If another account is Gmail, the process starts the same as before:

    # Create a directory
    mkdir -p ~/Mail/progin6304
    cd ~/Mail/progin6304
    # OAuth
    gmi init progin6304@gmail.com
    # Settings
    gmi set --replace-slash-with-dot

However, before running `gmi sync` for the second account, we want to make sure that we can distinguish the message from different accounts. To do that, I add the `main` for the main account and `progin` for the second account. We also don’t want these labels to be pushed:

    cd ~/Mail/thexcloud
    gmi set --ignore-tags-local new,mail,progin
    cd ~/Mail/progin6304
    gmi set --ignore-tags-local new,mail,progin

Now we can use notmuch’s `post-new` hook to tag the messages based on their folder as follows:

    cd ~/Mail/.notmuch/hooks
    cat > post-new <<EOF
    #!/bin/bash
    notmuch tag +main "path:thexcloud/** AND tag:new"
    notmuch tag +progin "path:progin6304/** AND tag:new"
    notmuch tag -new "tag:new"
    EOF
    chmod +x post-new

Now it finally makes sense why we wanted to use the `new` tag in the first place. In principle, any kind of tagging logic can be applied here, but for the reasons I stated earlier, I prefer to set up filters in the Gmail web interface.

The last thing to do is to modify the `pre-new` hook:

    #!/bin/bash
    GMI="/home/pavel/Programs/miniconda3/envs/mail/bin/gmi"
    (cd /home/pavel/Mail/thexcloud/ && $GMI sync)
    (cd /home/pavel/Mail/progin6304/ && $GMI sync)

After which we can finally tag the existing messages and download ones from the new account

    notmuch tag +main "path:thexcloud/**"
    notmuch new

The obvious problem, however, is that the messages are fetched sequentially, which is rather slow. A solution is to use something like GNU Parallel:

    #!/bin/bash
    GMI="/home/pavel/Programs/miniconda3/envs/mail/bin/gmi"
    parallel -j0 "(cd /home/pavel/Mail/{}/ && $GMI sync)" ::: thexcloud progin6304

I haven’t encountered any trouble with that solution so far (and I don’t see anything thread-unsafe in the lieer code), but I’ll keep an eye on that.

In principle, it shouldn’t be too hard to add a normal IMAP account as well with mbsync, but I expect it would require something like iterating through the directory structure and assigning notmuch labels based on that. I’ll probably try that some time in the future.

### Emacs

With that done, I also want separate entries on the start page for each of the accounts. Doing that is easy enough, just modify the `notmuch-saved-searches` variable with `customize-group` or like this:

    (setq notmuch-saved-searches
       '((:name "inbox (main)" :query "tag:inbox AND tag:main")
         (:name "unread (main)" :query "tag:unread AND tag:main")
         (:name "sent (main)" :query "tag:sent AND tag:main")
         (:name "all mail (main)" :query "tag:main")
         (:name "inbox (progin)" :query "tag:inbox AND tag:progin")
         (:name "unread (progin)" :query "tag:unread AND tag:progin")
         (:name "sent (progin)" :query "tag:sent AND tag:progin")
         (:name "all main (progin)" :query "tag:progin")
         (:name "drafts" :query "tag:draft")))

## Notification for new messages

Now, we can send and receive mail, but we also probably want notifications for new emails. To do that, I wrote a simple script:

    #!/bin/bash
    # To run notify-send from cron
    export DISPLAY=:0
    # A file with last time of sync
    CHECK_FILE="/home/pavel/Mail/.last_check"
    QUERY="tag:unread"
    ALL_QUERY="tag:unread"
    # If the file exists, check also the new messages from the last sync
    if [ -f "$CHECK_FILE" ]; then
        DATE=$(cat "$CHECK_FILE")
        QUERY="$QUERY and date:@$DATE.."
    fi

    notmuch new
    NEW_UNREAD=$(notmuch count "$QUERY")
    ALL_UNREAD=$(notmuch count "$ALL_QUERY")

    # I don't really care if there are unread messages for which I've already seen a notification
    if [ $NEW_UNREAD -gt 0 ]; then
        MAIN_UNREAD=$(notmuch count "tag:unread AND tag:main")
        PROGIN_UNREAD=$(notmuch count "tag:unread AND tag:progin")
        read -r -d '' NOTIFICATION <<EOM
    $NEW_UNREAD new messages
    $MAIN_UNREAD thexcloud@gmail.com
    $PROGIN_UNREAD progin6304@gmail.com
    $ALL_UNREAD total
    EOM
        notify-send "New Mail" "$NOTIFICATION"
    fi

    # Save sync timestamp
    echo "$(date +%s)" > $CHECK_FILE

The script is launched with cron every 5 minutes:

    */5 * * * * bash /home/pavel/bin/scripts/check-email

Here’s how the notification looks like:

## Caveats

- lieer has an extensive list of caveats concerning Gmail API
- Make sure that you understand the implications of lieer’s `--ignore-tags-locally` and `--ignore-tags-remote`
- If two of your accounts receive the same email, it will be stored as one email in notmuch, so tags from these accounts will be merged and pushed back on the next sync. To solve that, you can set tags from one account to be ignored on the rest of the accounts
- A sent email is being downloaded again on the next sync. Not a great deal, but it is somewhat annoying to download recently sent attachments.