HACK / MAKE

Mac Brain: Automatically Processing Email with Mail.app

Similar to the way that Hazel can automate moving around files, Mail.app can process emails in your inbox based on rules you set. Rules are often used used to flag messages and move them to folders based on certain conditions, but that’s not very exciting.

Processing email is a problem. There’s a lot that comes into your inbox and you continually have less attention to pay to it. I get a lot of recurring notification-type messages for things I need to do, like an email from my bank letting me it’s time to pay my monthly credit card bill. A thing that repeats with the same input and output is a thing that should be automated.

Routing Emails

Part of automation is setting something and never having to see or think about it. To make that part happen, I had to route all of the emails I wanted automatically processed in a separate email account1 so I never saw them in my main inbox. I set up a second Fastmail (referral link) account that would be used only on the mini server. Anything I routed to that email address would be processed. In Fastmail, I can also easily set up aliases so different to: addresses would trigger different things. I could set up omnifocus@myserveraddress.com to process OmniFocus tasks or have emails that go to alerts@myserveraddress.com ping my phone2 when important messages come in.

Once I had this new address set up, I had to configure the services (my bank for example) to send these notifications to that correct address. About half of my banking services allowed me to set up these notifications to go to a secondary address and the ones that didn’t had a unique from or subject that I was able to use server-side filtering to redirect any of those emails coming into my personal account to my Brain account.

This got me to a place where all of the emails I wanted to automatically process were in an inbox on my mini.

Setting up the Rules

Again like Hazel, setting up Mail.app rules is pretty easy once you know what to look for. For emails from my bank, I looked at the history of their messages and checked for a unique address for bill notifications or a consistent convention for the subjects of the emails. I wanted to be precise. Doing wildcards to redirect and forward all emails from @bank.com or whatever could mean that important emails I did need to see would get batched into an inbox I didn’t look at often.

All of these emails had unique conditions which could be passed on to a specified action, like the from: containing billnotifications@mybank.com or the subject is equal to Your bill has arrived.

We’ve captured the messages, now the fun part: doing cool stuff with boring emails.

Turning Emails to Actions

Once we’ve gotten to this point, we know the specific intention of the email. It has passed all of the tests and we just need to do something with it.

Most of the emails that come into my work or personal inbox I process and capture into OmniFocus. I wanted to be able to automate this step for the emails I could. David Sparks put together this great Applescript to run within Hazel to add actions to OmniFocus when he scans documents that match specified criteria. Sound familiar? We’re basically doing the same thing but the input is an email not a OCR’ed document and the rules are processed by Mail.app and not Hazel.

I have Mail.app “perform the following actions” when it receives a matching email:

  • Move it to Archive
  • Mark it as read
  • And run an Applescript on it

My Applescript is exactly the same as David’s but I’ve added in the time it was created as the Start Date of the OmniFocus action. To do this, you just need to add start date:theDate to the list of task properties. This helps me recognize new automatically added actions in one of my OmniFocus perspectives.

Now—all automatically—when my bank sends out an email to let me know my statement is ready, it filters into my Brain email account, gets checked through Rules, and when it matches, it adds an OmniFocus task to “Review and Archive my bank statement” or create two separate tasks like “Review credit card statement” and “Pay off credit card balance”.

This rule is set up to run a “Create OmniFocus Task” Applescript, but you can set it up to trigger any Applescript you can think of.

The Point of Automating

Sure, it’s cool that a computer can check your email for you but the value comes from reducing the amount of time and attention you need to be spending processing repetitive items and reducing the friction of getting something into your outboard brain. Not only is the action being put into your outboard brain, but an outboard brain is doing the processing now too. In a lot of cases it works to set repeating tasks for something like bills but I find it more direct to have the inputs of the notifications and the action align rather than scheduling a task and have it start a couple days earlier than the bill is available because of a long weekend or something. It’s all small things, but the reduction of the attention required to process along with being able to rely more on your outboard brain because of its ability to capture and process things without your input is what makes the automation worth the setup investment over time.


  1. I played around with server-side archiving, but Mail seems to only run rules on things that are in your inbox. 

  2. I’ll walk you through this soon. Having a server set up to send you an SMS is pretty cool. 

Mac Brain: Managing Files with Hazel

This post is part of a series on setting up a Mac server and automation system to simplify computing and take greater advantage of iOS devices. You can read all of the posts from my Mac Brain project here.

One of the main goals of setting up an automated Mac was to relieve constraints in iOS and the iPad. One of those that has always been a challenge is managing files. A file system doesn’t really exist on the iOS (though you could argue that Dropbox has mostly filled that gap) and it’s relatively small drive isn’t ideal for long-term file storage. iCloud and Dropbox have stepped up to solve many of these issues for normal iPad users but for someone like myself, who wants to make the iPad my main device, there are concerns about putting my trust in the cloud.

There were two main categories of files I wanted to handle using my iPad: things I wanted constant and current access to and stuff I have to collect and wanted to archive. The stuff I wanted access to are things like my plain-text notes and photos and the things I needed to collect then archive are things like bank statements that would come in and be processed and working files that I no longer need immediate access to. I used to keep all of my documents in Dropbox but I recently realized that I didn’t need anytime-access to most of them on any device. What I lost by moving them out of Dropbox I gained by not having a lot of personal files in a more likely to be compromised system. Now things that don’t need to be in the cloud aren’t. They’re safely and obsessively organized on my Mac. This isn’t anything new, most of you have files archived on your machines, but the nice thing for me is that this isn’t a day-to-day used computer so I don’t notice files piling up in Finder. The beauty of this system is that I can set it up to organize my files the way that I want and then I never have to worry about it or even see the files.

Archive and Document Folder Structure

When I was first setting up this server, I spent some time in Finder organizing what documents and random files I had around into two main places: an Archive folder, and the standard ~/Documents folder. Into Archive, I moved a bunch of files from my Dropbox into a /YYYY/MM_Month/ folder structure based on the created date of the file and made a couple Smart Folders to more easily find old archived PDFs and images. Into ~/Documents go some more current and specific files that don’t quite fit into a ‘junk drawer’ archive, but should be more cleanly organized. Right now, this is where I put bank statements.

Both of these endpoints have the same in-point. It’s a folder on my Dropbox named [DROP]. In this folder, I can drop a whole range of files and based on Hazel rules I set up, my filing will happen automatically. Archival into my /YYYY/MM_Month/ directories will happen whenever I drop files into a [To Archive] folder in that drop folder. With this system, I can pass pretty much any file type on the iPad to “Open in Dropbox” where I’ll be prompted to choose a folder to put it in. Choosing [DROP] will sync the file over to my Mac brain and Hazel will do what I ask of it.

Let’s go through a couple of the Hazel rules as examples and so your imaginations can be piqued as to what kind of things you’ll be able to do to automate your system. This will be more of the methods and thought process of working with Hazel since setting up the tasks are pretty straightforward.

PDFs: From iPad to Archive

Bank statements were never really something that were too important to me because I figured that I’d just always be able to get them from my bank’s website. A few months ago, my Mom asked me for record or some payment stuff from years ago for something legal or tax related. I went to my bank and realized that they only kept my statements from within 18 months. At this point I figured it would be worth holding onto them on my side but until I set up this machine, I didn’t really keep up with it because of the nuisance of pulling down and organizing the files.1 With Hazel, I just need to get the file into my [DROP] folder and it’ll do the rest. To set this up, I pulled down the PDFs from my banks site and looked to make sure I could find a consistent pattern in the file name that I could tell Hazel to look for. All four of my banking related service providers named the PDF statements in a way that Hazel could uniquely recognize. So with Hazel matching the filename containing ‘eStatement’ and extension is ‘.pdf’ in my [DROP] folder, it’ll rename the PDF to something like BANKNAME_CHECKING_2012-09.pdf and move it into the folder ~/Documents/Bank Name/Checking/YYYY/, again all based on the rules I set up. I also have it importing those documents into Evernote. Doing both isn’t necessary but I wasn’t totally sure which system I wanted for this type of document archival, so I went with both for now. In the last step of the Hazel task, I use this Applescript to get the document into Evernote:

set m to do shell script "date +'%m'"
set y to do shell script "date +'%Y'"

tell application "Evernote" activate
    create note title "Name of Bank Checking Statement for " & y & "-" & m notebook "Bank Statements" from file theFile
end tell

In the end, your task will look something like this.

Organizing Your Plain-text Notes

Another way I use Hazel is to keep my plain-text notes folder clean. Some people just leave everything in this folder whether they’ll need it again or not, but I want to keep it clean. I have both scratchpads and reports I create from my analytics tracker, Gaug.es, go into my Notes directory but I only really need immediate access to these within the week they are created. Once they get over a week old, Hazel moves them into a sub-folder in /Notes so I still have access to them, but they aren’t clogging up my main list. The Hazel rule for this is basic: if it matches my Scratchpad_YYYY-MM-DD file naming convention and the Date Created is not This Week, then it gets dumped into a /Scratchpad/YYYY/MM_Month/ folder. Now these files are still easily accessible but they also aren’t in the way when I’m searching through my notes on Notesy for iPad.

I want to be getting deeper into using Hazel to help organize my life. If you have some interesting Hazel jobs, I’d love to hear about them (I’m @nickwynja on Twitter). What I’ve got so far is pretty basic but has helped simplify and keep me organized while using iPad.


  1. Watch for an upcoming post about Mail.app rules to see how I set up the automatic creation of OmniFocus tasks to tell me to review and archive my bank statements whenever my bank sends me an email to let me know the new document is up. 

Mac Brain: Server.app

This post is part of a series on setting up a Mac server and automation system to simplify computing and take greater advantage of iOS devices. You can read all of the posts from my Mac Brain project here.

There’s a few basic things that any server needs to do: serve files, control access, and get on a network. OS X Server.app is the most simple way to do this on a Mac.

Apple has moved away from creating Server OSes and now ships a Server application that runs on top of the standard consumer Mac OS. Along side Mountain Lion, Server.app is available in the App Store for $20. The things that this app does can all be done for free with standard unix utilities, but believe me, the $20 is worth the price so you don’t have to learn how to set up apache and DNS stuff on your own. You can easily open up the app and toggle the features you want to serve. The main ones I use are Websites, DNS, File Sharing, and the option that lets you set up some port forwarding and firewalls with a connected AirPort Extreme.

There’s a great guide that helped me with get familiar and set up 10.8 Server.app here at Krypted.com.

The first thing I needed to do since I was running this machine headless at home was (right after the box was delivered to my office) to plug it in to a display, get it updated, install Server.app, and config it to allow remote SSH and VNC access. This way, when I brought it home, I’d be able to power it up and from my iPad using Screens, log into the server and control it to do the rest of the configuration and set up.

To do this, go into Server.app and in the Hardware section at the top, click on your computer name and turn on these:

  • Allow remote login using SSH
  • Enable screen sharing and remote management
  • Allow remote administration using Server

You may also want to set up your AirPort to allow these connections so you can manage your server from outside of your local network. Go into your AirPort in the Hardware section and hit the + and choose which service you’d like to enable. This will open the port in your firewall and forward it to your server.1

Once these are set up, you should be able to manage your server via Screen Sharing on Mac, a VNC app like Screens on your iPad, SSH at any command line, and using the same Server.app you downloaded from the Mac App Store on a different computer to log in and admin your server. There are plenty of tutorials around for VNC and SSH, so search for those if you need a hand getting it set up. Part of the fun of this project is to learn things, so challenge yourself and try SSH and learning some of the command line tools.

Next, I’ll go through some of the basic tasks I’ve got running that manage my files. If you’ve got questions about setting things up, or have something specific you’re interesting in me going over, chat with me on Twitter.


  1. Abstracting this type of configuration is so nice. You’ll understand what I’m talking about if you’ve ever had to log into crappy routers and manage this kind of stuff. 

Mac Brain: The Hardware and Software

The goal of my Mac brain project is to simplify and automate parts of computing so that the computer can take care of some of the work that my minor obsessive habits create. I’ve been using and really loving my iPad recently and want to keep using it more. One of the goals of this setup is to ease the pain that restrictions iOS has in automating work in my life. The software will do things like reliable backups, maintain a clean file structure, and download and archive documents like bank statements.

None of this is heavy work, but the set up is important for things to work in a way that I can stop thinking about it. First, let’s talk about the machine.

Hardware

I’m using a baseline refurbished Mac mini. It’s 2.3GHz i5 with 2GB of memory and a 500GB spinning hard drive. At some point, I’ll upgrade the RAM (this Mac can handle a wapping 16GB), though it’s not running memory intensive tasks at the moment. 500GB is enough space for now, since I don’t have a lot of media. Again, it being a spinning drive and not an SSD isn’t a big deal since I’m not doing my main computing on it and don’t need the I/O for fast app launching.

The machine is running headless (it’s not connected to a monitor, keyboard, or mouse) so it sits pretty and quietly on a shelf.

One of the requirements I had was that the machine would be hardwired directly into my AirPort Extreme. Wifi is getting better, but nothing compares to a hard 1Gbps connection.

The other important piece of the puzzle is the AirPort Extreme. OS X Server.app and the Extreme talk back and forth really well making configuration easy for things like port forwarding, which is necessary to get this setup running smoothly.

You don’t need this exact hardware to get your system automated, but for me, simplicity is important and the right hardware make things a lot easier.

Software

What makes this machine an outboard brain is the great software that it runs. I talked a bit about what I wanted to get done with this mini in my introduction post. Ultimately, the software I needed (or you need) depends on what exactly I wanted to get done. There were a few main jobs: move around and organize files, backup the application data that I use on my other Mac and iOS devices which can sync back to this machine, automate downloads, and backup all of my data securely.

The basic tools that handle all sort of jobs are Hazel, Lingon, Server.app, OmniFocus, Mail.app, Transmission, Evernote, iPhoto, iTunes, Skype, Backblaze, Dropbox, and Airfoil.

I’m just going to do an overview of the apps at this point and will go into more detail in the future about setting them up and explain what kind of tasks I have them do for me.

Hazel: This is one of the great OS X automation apps. It installs as a preference pane, so you know it’s always going to be running, watching folders, and waiting for whatever tasks you delegate it. The basics of Hazel is this: You tell it a folder to keep an eye on, certain parameters to watch for, like file extension, created date, or content, and then give it something to do when it finds a file that matches those parameters. Maybe you move it into another folder, delete the matching file, or run an Applescript or shell script on that file.

A lot of nerds use Hazel extensively so there are plenty of examples of Hazel actions and setups out there.

Lingon: Lingon is a GUI for the launchd and launchctl system utilities in OS X. I’ve worked directly with those services on the command line, and believe me, you want to use Lingon. It will schedule scripts or jobs that you give it to run at a specific time of day, time intervals, when something happens to a file or folder, or whenever you plug a drive in. Again, this app is incredibly flexible, and will take any job you need to schedule or run often. A couple things I have it do right now is to [backup my tweets] to a text file and generate reports from Gaug.es analytics tracking.

Server.app: This is the packaged application that sets up, configures and manages server utilities like apache, handles DNS, file serving, permissions, and all that neck beard stuff. Luckily, the GUI makes it much easier to get set up for people who aren’t familiar with managing all of these things from a CLI, though Server.app does have a command line interface. Server.app is $20 in the Mac App Store and will install as an app on Mountain Lion, rather than as an OS like previous versions of OS X Server. If you want to be running a file server or local websites for controlling your server, Server.app is the best way to admin those things.

Dropbox: Just the standard install of Dropbox. It syncs to get my files to the Mac mini, where I will back them up or have Hazel watch folders and do actions on them. I have a Drop folder within Dropbox that when I put a file in there from my iOS device, it’ll move it to an Archive folder or maybe add it into Evernote, depending on what type of file it is. Again, I’ll go more into this when I expand on Hazel.

OmniFocus and Evernote: These apps are installed with a standard configuration using my accounts to accept incoming actions or notes that are trigger from Applescripts. Once the action is automatically created based on scripts, it’ll will sync back to my other devices with it’s built in sync service.

Mail.app: For email, obviously. Mail runs with my personal email account running as IMAP, which is constantly pulling down email. The way Mail.app works, it saves downloaded emails into a system folder so when I backup my drive it also backs up all of my email. If something catastrophic were to happen to Fastmail (referral link), I would still have copies of all of my emails backed up. Mail is also running a second account with a different address which I can channel emails into for automated rules and not have to see those emails come into my other inbox. I have rules to create tasks in OmniFocus that I email to myself or to create a task like “Review Visa statement” when my bank emails me letting me know that my monthly statement is ready. Having these separate accounts, and filters on the mail server-side allows the Mac Brain to process parts of my inbox rather than me having to.

iTunes and iPhoto: Pretty self explanatory but a necessary part of the flow. I’ve got iTunes set up to automatically sync and backup my iPhone and iPad wirelessly. Right now, I have my devices backing up to iCloud, but I’m looking for a way to also have them automatically backup locally. You can do this by right-clicking on the device in the sidebar and then “Back Up”, but I want to automate this. iPhoto is open and running so it properly pulls down my Photo Stream and saves it locally since iCloud only keeps 30 days or something.

Transmission: For torrents. There’s lots of other resources if you’re looking to set this up but briefly, Transmission has a web interface you can access from the web to start downloading whatever legal stuff you’re downloading, I guess, and Hazel can move downloads to iTunes so you can access them via Home Sharing on iPad.

Skype: I have Rode Podcaster mic set up with the mini. For recording podcasts, Skype for iPad doesn’t support this mic well and requires a bunch of extra cables to make it happen. I’ve got the mic set up with Skype so I can record high-quality podcasts easily. Skype is a good reason to have your Mac hardwired. I’ve been working on a cool remote control interface for my iPad to answer incoming calls and hangup, hold, and mute active calls on my mini so that I can manage Skype running on the headless server.

Airfoil Speakers: This is a temporary thing until I get a better HiFi system set up, but it’s a cool app to turns your Mac into a set of Airplay-enabled speakers.

Backblaze: In the past couple of months, I’ve heard a couple horror stories of lost data. You do not want this to happen to you. Backblaze runs in the background to backup all of your stuff (you can exclude directories) to their cloud servers so that you have a reliable off-site backup. Their service is dead-cheap at $5 a month for unlimited storage and provide seeding options for you to mail them a drive with your stuff in case you have a slow connection and have trouble uploading all of your files. If you take away only one thing from this lengthy series, go sign up for Backblaze and worry less about losing precious data.

That’s the main tool set that runs this brain. I’ll go into more depth on each one specifically and show you the kinds of things I have them doing. For now, it would be worth downloading the apps if you don’t already have them and play around with what they can do. Get some of your own jobs set up and start automating your machine.

Mac Brain: Automating and Simplifying My Computing

The ways of computing have changed in the last five years. We’re in a revolution and a post-PC era where the tools available to us are smarter and many of the problems we have to solve have been simplified. Apps encourage doing one thing well which has shifted our focus while computing. The Cloud has abstracted a core problem for ubiquitous data.

But for geeks, the transition can hurt a little.

The power in iOS software can be diminished by the constraints it’s under. The main market for iPhones don’t need fully-supported background processes and the nerds who want that may not appreciate the tradeoffs that comes with. Geeks want our devices to do more and we want the cloud to do more. Incredible software innovation has happened from this want and will continue to happen over the next decades of this new technical revolution.

But for now, we need something to ease the pain.

I bought an iPad as soon as I could. From the start, I didn’t know exactly how I would use it but I knew it was a blank canvas device that could be turned into nearly any tool just with an app; I wanted that. The new iPad made this device totally irresistible to me. The screen—oh, that screen—was gorgeous. The things that were on it seemed more alive because they weren’t constrained to living in these little RGB boxes. Well, they were but I couldn’t see the pixels anymore so in my mind, what was on screen was more real. LTE meant the things I wanted to do could be done faster. Way faster. I quickly found that my MacBook Air stayed at the office and that my iPad became my main device.

But I found myself sometimes caught in the middle of the freedom and simplicity of reading, writing, playing, and making on my iPad and the constraints simplicity brought. The questions of where my stuff actually lived when it was “in the cloud” and whether I should trust said cloud to be the only place I back up my memories worried me. I needed to patch the gap between the future of computing and where we are now. I needed something that could be an outboard brain for my iOS devices and a smarter cloud for my data. I didn’t need a file server or a home theatre PC, I needed a brain that could worry about computing so that I didn’t have to.

I picked up a Mac mini a few weeks ago and have been working to set it up so that it can be an outboard brain—for my devices and for me. It’s running headless1 because I don’t need a desktop computer. It’s not plugged into a TV, partly because I don’t own one.

This humble silver box will live most of its life doing thankless tasks. It’ll help manage files—a job which has become “simpler” yet more abstract on iOS. It’ll back up my digital life without even being asked. My humble mini will handle some things that my iPad can’t or I don’t want it to and it will be a system for me to experiment with automation in my home and in my life.

The core of this system is powered by Dropbox, Hazel, Lingon, iTunes, and Server.app. I’ll be writing more about how I’ve set this up, how I continue to manage it, and the impact it has on my computing. In a series of posts that will cover things like backups, handling and organizing text files, archiving, automating apps and tasks, and using your iPad as a launch center to control your Mac brain, I’ll hopefully be able to lead you through setting up a similar machine. You don’t need to have a standalone machine dedicated to these tasks to take advantage of what it can do. If you have a Mac at home, put it to work.


This machine will become an important part in my system. One of the last innovations Steve Jobs left us was iCloud. It would shift the model of how computers were designed to interact with each other.

Jobs at WWDC 2011, addressing the problem of sync:

So we’ve got a great solution for this problem. And we think this solution is our next big insight. Which is we’re going to demote the PC and the Mac to just be a device. Just like an iPhone, an iPad or an iPod Touch. And we’re going to move the digital hub, the center of your digital life, into the cloud.

This approach has worked in the past year and adding a Mac to handle some tasks I give it will fit well into this architecture. My Mac brain will continue to be just another device, but one reserved for handling the computing jobs that I’m not ready to give up control of yet and jobs that will allow me to think less about computing and more about creating.

Subscribe to the feed or join the mailing list to get updates on this project.


  1. I do see the irony of a “headless” computer (one running without a monitor) being a “brain.”