Parsing incoming emails is a common task for many web application which deals with emails, and then take decision on the parsed contents. However, piping the incoming mail stream to an external script is not everyone’s food and it needs good knowledge on shell scripting and also you need to be good with your mail server to install/configure the pipe properly. So there are third party services out there which makes it easy for you by accepting your emails to predefined mailboxes. They also parse the email and post the content over HTTP to an external script (i.e webhook) which makes the whole process a lot easier. You just outsource the email parsing to these third party providers and spend time rather on analyzing the content of the email leveraging their email->http webhook. Cloudmailin, MailNuggets, and MailGun are one the few providers which I’ve tested and they work really good with very low latency. In this article I am going to show you how to configure your free Mailgun account to accept incoming mails, parse and then accept the mail content to your php script.
Step1: Register a free account with MailGun
Don’t worry – they dont charge. And Mailgun’s free plan is quite good with 200 mails/day which is pretty good during development phase. Once you are ready, you may opt in to purchase their plans which suit you best.
Step 2: Add a domain
Free accounts are restricted to use *.mailgun.org domain for now. So just log into you Mailgun control panel and create your own domain.
Step 2: Create a mailbox to accept incoming mails
You need to configure a mailbox to start accepting inbound emails first, so you can process them later using different rules. Just go to your mailboxes tab in the Mailgun control panel and create a new mailbox.
Step 4: Link your incoming emails to your external script.
This step was a little confusing at first and I really didnt understand how to achieve the result. Mailgun will automatically parse these incoming mails, which you can filter by recipients or senders or by wildcard and then forward to your external script. To do that, just go to your Routes tab in the Mailgun control panel and create a new route, like the one shown in the screenshot below.
See the forward() function does the job where my confusion began – I was almost clueless 😛
Ok we are done. Mailgun will now accept all mails to this mailbox [email protected] and parse the mail content including attachments, and then post the parsed contents as an array to the external PHP script we have configured in step 4. In your php script just capture those data from $_POST and do whatever you want to.
Shameless Plug
We develop beautiful dashboard and admin panel templates for the web application developers, which you can plug into your application easily. These templates comes with well documented code, even better documentation of components and a pack of essential plugins. Check out one of our admin template “Utopia” and grab your copy if you like it
It started in a funny way rather than a typical project plan, and finally turned into an important project for us in Leevio. Everyone in our team was so much excited to release our first flagship product, a web based service (mashup you can say) we called MiproApps. It is a visual designing tool for Facebook fanpages.
MiproApps
Facebook fanpages are popular media to publish your company information, portfolio or upcoming events on Facebook. To make your product outreach a massive amount of users, it is a popular tool that works good. While creating our fanpage for Leevio, we found that we are limited by available applications to decorate and present a page “really” well. What we could do is design our page in an ordinary way, like writing FBML tags and and embed those in a tab. Or we could develop an application and embed it’s tab to another tab on our page. Which one did we go with? NONE!
We thought that it would be really interesting if we can develop a service (some sort of a visual designer for such pages) by which everyone (anyone, seriously) can design and decorate their fanpage with some data they really want to deliver to their audience. So we did a quick meeting to find out if it was really possible by us, and the challenges we found are the followings
1. Limited canvas size,
2. Works with different data sources
3. Drag and drop friendly, with proper layering.
4. Compatible with major browsers (Firefox, Safari, IE, Opera and Chrome)
5. Proper caching and storing widget data
We decided to develop this project in a “release early, release often” fashion. We didnt want to deliver a HUGE giant bloated with too many widgets and services after working for tons of hours. Instead we wanted to deliver a working barebone skeleton for our users which performs it’s job pretty well, and then to add feature s gradually in a regular interval. It would also help us to decide quickly how users are reacting with our project.
We four developers worked really hard for next couple of weeks to make it work, in short sprints and milestones. And now it is available for your use – you can check it out at http://miproapps.com. There are more than 25 pretty useful widgets to help you designing your fanpage impressively. We’ve also added tutorials and screencasts for you. Lets see how did we work with all our challenges.
MiproApps Design Canvas
1. Limited Canvas Size
To make sure users still be able to design his page easily, we keep three pre defined size for each widgets. Width of these three sizes are designed in such a way so that anyone can design multi-column page, easily.
2. Works with different data sources.
The core of MiproApps are external data sources and a very well designed parser to parse those data source in an unified way. Guess what, most of these data source are ready to be pulled out. Almost all of them offers valid RSS/Atom feeds. And that saved us HUGE (GIGANTIC cud be more appropriate) amount of work to do by ourselves. We just collect username for each of these data sources (like twitter, youtube) and then we pulled out feeds for that specific user. Everything is about public data, and that also saved us from some pain about treating private and sensitive information. Except one widget (I will come to that later) we just asked for plain username from users. And for twitter, we had to deal with oath based authentication system so that we can pull out user’s timeline using their token. These tokens are for read-only access to be more trusted by our users. We set a caching time of 10 minutes so that we have to call only 6 times max for an user in an hour. Do you kno what is the current rate limit per user per hour in Twitter? it’s only 75!
Configuration Editor
3. Drag and drop friendliness
We choose jQuery UI as we are very good at jQuery. And I must confess, that was a very good decision indeed. The excellent support of managing draggable and droppable components in jQuery UI made our life really easier. The only thing we found tricky was to persist the draggable state of each component on “Design Canvas”. But anyway, it was done very effectively and is working really great. You should check this out at http://miproapps.com
And oh, by the way, the z-index layering was quite a UI challenge for us. On a Facebook page canvas area, the internal Facebook components has maximum z-ndex of “101”. So we had to deal with that to keep our components z-index lower than that, to avoid an overlay over any internal Facebook item (a dialog box, for example, or the notification dropdown 🙂 )
4. Compatible with major browsers
Oh boy, we were doomed in hell with a monster called IE and that really made our day terrible every minute!. However, we tamed the beast quite good and now it works well with MiproApps. Another friendly monster betrayed us at last minute, has a name “Chrome”. we are working on that subtle UI issues and hope to deliver some updates pretty soon.
5. Proper caching and storing user data
Now that was a major challenge we dealt with. Each user can use hundreds of widgets, which colt actually pull data from hundreds of different data sources out there. So caching is a very important step we’d taken care of from the first day. We used Memcached. And we designed the architecture to cache data from every widget separately, not as a page in a whole. And it was challenging to deal with failsafe condition (you remember twitter’s fail whale, right?). Some widgets needed to be dealt with extra care for the caching technology of Facebook itself. But in overall it was a good architecture and worked impressively well. We are proud of it!
Useful set of widgets
Lets have a look at the tools, libraries and services we use for MiproApps
1. Subversion, definitely a life saver version controlling system.
2. Springloops as a subversion repository and deploy manager. It’s deploy manager is really awesome!
3. Lighthouse for Issue Tracking
4. Netbeans as our IDE, Mac OSX and Ubuntu as development OS. PHP and ZendFramework. For oAuth, we used PECL oAuth library. Memcached was used for caching.
5. jQuery and jQuery UI. The image slider you see on the front page was done using the brilliant Coin Slider.
6. Screencast was recorded by Screenium. Tutorial screenshots were taken using Jing.
7. Icons came from MediaLoot and legendary Silk by Famfamfam.
So that’s It – have a look at MiproApps at http://miproapps.com. Currently the application is in beta mode (all plans are free during beta – beside that, there will always be a free plan for everyone) and we are adding exciting widgets everyday. Hope you will like it.
I got a bad news this morning that zembly is shutting down their service from 30th next. that’s a very bad news because zembly was a nice service for developers. They’d started their business from june 2007, provided exciting APIs to developers to develop mashups on top of popular services like Facebook, MySpace, Flickr and so forth. I’ve seen another popular service like this , appJet, went down this year. Now it’s extremely frustrating that such types of services are shutting down. Why? most probably they’ve failed to secure funding, or may be they’ve some fight in their board of directors, or whatever, who knows and who cares.
But the point is, what happen to developers who has developed their app using services like zembly or appJet? all their efforts to learn these new technology, new tools and to develop some really cool applications using these services just became useless all at a sudden. That’s pathetic, and thats an utter lack-of-respect to your own developers community. These developers are your everything. They were your everything. If these developers didn’t put their efforts for you zembly, you will never survive a quarter, may be six months.
I really dont understand, If you dont have the guts to run your product thats fine. I have no problem with that. But before saying “All our service will be unavailable from blah blah blah” and “We are sorry and blah blah blah” , come on – show a minimal respect to your developer community. After 2.5 years of run, its not a small community after all. And grow up kiddo, before shutting down your company, at least make your project open source – as a sign of minimal respect and loyalty to the developers community who kept you running for so long.
I am sorry for Zembly But I am more sorry for the developer community.
