In MiproApps, our Visual Facebook Fanpage Desiger from Leevio, everything is built on top of a scalable plugin based architecture. Every plugin manages it’s data using a central plugin manager. Most of these data are isolated from each other, stored and served by the plugin manager without any special coding required from plugin developers. That makes everything simple. As storage and serving is fully managed by Plugin Manager, it helps us to cache, validate and sanitize user data properly from a single place.
But when we decided to add support for box.net, there comes a challenge. We have asked for username and password for box.net account from our users to pull out the data from their shared folders and files. “PASSWORD” – and that is the thing everyone thinks twice before providing to a third party. Everyone cares about their personal data security.
In MiproApps every data collected from user are submitted to storage service via Ajax request. And we simply cant send plain password collected by users in an Ajax request. What we did in this case is we had signed user’s sensitive information using a 128 bit public key (RSA) in client side. The private key is stored securely in our server and that encrypted information is decrypted only in server side. So client application has just the public key.
Box.net widget in Facebook Fanpages powered by MiproApps
There were other challenges as well, while we went to implement this encryption in client side by Javascript and Decryption by PHP. Unfortunately PHPs Mcrypt doesnt support RSA, and Zend Framework doesnt provide any component for that. And there was a trick when you encrypt your data in JS. You must add a null byte at the end of your data, otherwise PHP cant decrypt it.
You can see some example code at here and you can use the RSA public/private key pair either by openssl shell command or the RSA key generator from ohdave.
The service layer is working smooth. Plugin developers doesnt need to bother about encryption and decryption. Everything was managed transparently under the hood. And so far we are only developing our plugins, data is secured and safe. Users can add their publicly shared box.net files directly in their facebook fanpage. For a sample output, you can check out my page. You will find the box.net component at the bottom right corner.
It started in a funny way rather than a typical project plan, and finally turned into an important project for us in Leevio. Everyone in our team was so much excited to release our first flagship product, a web based service (mashup you can say) we called MiproApps. It is a visual designing tool for Facebook fanpages.
MiproApps
Facebook fanpages are popular media to publish your company information, portfolio or upcoming events on Facebook. To make your product outreach a massive amount of users, it is a popular tool that works good. While creating our fanpage for Leevio, we found that we are limited by available applications to decorate and present a page “really” well. What we could do is design our page in an ordinary way, like writing FBML tags and and embed those in a tab. Or we could develop an application and embed it’s tab to another tab on our page. Which one did we go with? NONE!
We thought that it would be really interesting if we can develop a service (some sort of a visual designer for such pages) by which everyone (anyone, seriously) can design and decorate their fanpage with some data they really want to deliver to their audience. So we did a quick meeting to find out if it was really possible by us, and the challenges we found are the followings
1. Limited canvas size,
2. Works with different data sources
3. Drag and drop friendly, with proper layering.
4. Compatible with major browsers (Firefox, Safari, IE, Opera and Chrome)
5. Proper caching and storing widget data
We decided to develop this project in a “release early, release often” fashion. We didnt want to deliver a HUGE giant bloated with too many widgets and services after working for tons of hours. Instead we wanted to deliver a working barebone skeleton for our users which performs it’s job pretty well, and then to add feature s gradually in a regular interval. It would also help us to decide quickly how users are reacting with our project.
We four developers worked really hard for next couple of weeks to make it work, in short sprints and milestones. And now it is available for your use – you can check it out at http://miproapps.com. There are more than 25 pretty useful widgets to help you designing your fanpage impressively. We’ve also added tutorials and screencasts for you. Lets see how did we work with all our challenges.
MiproApps Design Canvas
1. Limited Canvas Size
To make sure users still be able to design his page easily, we keep three pre defined size for each widgets. Width of these three sizes are designed in such a way so that anyone can design multi-column page, easily.
2. Works with different data sources.
The core of MiproApps are external data sources and a very well designed parser to parse those data source in an unified way. Guess what, most of these data source are ready to be pulled out. Almost all of them offers valid RSS/Atom feeds. And that saved us HUGE (GIGANTIC cud be more appropriate) amount of work to do by ourselves. We just collect username for each of these data sources (like twitter, youtube) and then we pulled out feeds for that specific user. Everything is about public data, and that also saved us from some pain about treating private and sensitive information. Except one widget (I will come to that later) we just asked for plain username from users. And for twitter, we had to deal with oath based authentication system so that we can pull out user’s timeline using their token. These tokens are for read-only access to be more trusted by our users. We set a caching time of 10 minutes so that we have to call only 6 times max for an user in an hour. Do you kno what is the current rate limit per user per hour in Twitter? it’s only 75!
Configuration Editor
3. Drag and drop friendliness
We choose jQuery UI as we are very good at jQuery. And I must confess, that was a very good decision indeed. The excellent support of managing draggable and droppable components in jQuery UI made our life really easier. The only thing we found tricky was to persist the draggable state of each component on “Design Canvas”. But anyway, it was done very effectively and is working really great. You should check this out at http://miproapps.com
And oh, by the way, the z-index layering was quite a UI challenge for us. On a Facebook page canvas area, the internal Facebook components has maximum z-ndex of “101”. So we had to deal with that to keep our components z-index lower than that, to avoid an overlay over any internal Facebook item (a dialog box, for example, or the notification dropdown 🙂 )
4. Compatible with major browsers
Oh boy, we were doomed in hell with a monster called IE and that really made our day terrible every minute!. However, we tamed the beast quite good and now it works well with MiproApps. Another friendly monster betrayed us at last minute, has a name “Chrome”. we are working on that subtle UI issues and hope to deliver some updates pretty soon.
5. Proper caching and storing user data
Now that was a major challenge we dealt with. Each user can use hundreds of widgets, which colt actually pull data from hundreds of different data sources out there. So caching is a very important step we’d taken care of from the first day. We used Memcached. And we designed the architecture to cache data from every widget separately, not as a page in a whole. And it was challenging to deal with failsafe condition (you remember twitter’s fail whale, right?). Some widgets needed to be dealt with extra care for the caching technology of Facebook itself. But in overall it was a good architecture and worked impressively well. We are proud of it!
Useful set of widgets
Lets have a look at the tools, libraries and services we use for MiproApps
1. Subversion, definitely a life saver version controlling system.
2. Springloops as a subversion repository and deploy manager. It’s deploy manager is really awesome!
3. Lighthouse for Issue Tracking
4. Netbeans as our IDE, Mac OSX and Ubuntu as development OS. PHP and ZendFramework. For oAuth, we used PECL oAuth library. Memcached was used for caching.
5. jQuery and jQuery UI. The image slider you see on the front page was done using the brilliant Coin Slider.
6. Screencast was recorded by Screenium. Tutorial screenshots were taken using Jing.
7. Icons came from MediaLoot and legendary Silk by Famfamfam.
So that’s It – have a look at MiproApps at http://miproapps.com. Currently the application is in beta mode (all plans are free during beta – beside that, there will always be a free plan for everyone) and we are adding exciting widgets everyday. Hope you will like it.
After searching for help to connect with LinkedIn via their oAuth protocol using PECL oAuth extension, I’ve found that lots of people are posting in their forum for the code samples. And only very few obscure code examples are available. I’ve found phplinkedin script but that is just too bulky for a simple oAuth dance 🙂
So here are two files to perform 3 step oAuth Dance for both twitter and linkedin. Just set your consumer key and consumer secret key in these scripts (config.php) and in your LinkedIn and Twitter application, set the url of these scripts as authentication callback 🙂 thats it 🙂
When lifestreaming services come under the spotlight, Storytlr is definitely a promising one. They started their journey not more than a year ago, was loved by many other people out there, and decided to shutdown! Ok, that makes sense when the economy was so scary lately, but the good thing is that – they didn’t just announced “We are shutdown, and we don’t care what the fuck had you built with us”. Those brilliant people behind Storytlr did something very much appreciable, they made their brain child open source. I really loved that. The source code is available to download from their google code project page.
Other than Storytlr, there is another open source life streaming service available at this moment which is “pubwich” and is available to download from http://pubwich.org
My post is about how you can grab the source code of Storytlr and set it up in your own webserver. I’ve seen many people were asking on the mailing list an their news blog about how to install this one successfully. So I decided to give it a shot and install it. Follow these steps and you are good to go 🙂
#1 Add a wildcard A record pointing to your server’s IP #2 If you are hosted in VPS, and you have multiple domains hosted, you may want to create a Virtual Host with the following entries. I just assumed that your document root for Storytlr will be at “/var/www/storytlr”
[sourcecode lang=”php”]
<VirtualHost *:80>
ServerName domain.tld
ServerAlias domain.tld
ServerAdmin [email protected]
DocumentRoot /var/www/storytlr
<Directory /var/www/storytlr>
Options FollowSymLinks
AllowOverride All
</Directory>
</VirtualHost>
[/sourcecode]
Dont forget to restart/reload your web server after this.
#3 change your current workign directory to /var/www/storytlr and download the latest available storytlr from google code, and then extract it 🙂
[sourcecode lang=”bash”]
wget http://storytlr.googlecode.com/files/storytlr-0.9.2.tgz
tar -zxvf storytlr-0.9.2.tgz
[/sourcecode]
Now you will see some directories in the same path.
#4 Create a mysql database named “storytlr” (You can name it whatever, just dont forget to change the DB settings accordingly in the config. I will be discussing this later) – also create a user for this db and grant it all permission for this db.
[sourcecode lang=”sql”]
mysql> create database storytlr;
mysql> grant all on storytlr.* to ‘storytlr’@’localhost’ identified by ‘password’;
mysql> flush privileges;
[/sourcecode]
#5 Now you need to import storytlr’s schema into it. change your working directory to /var/www/storytlr/protected/install and import the database.sql into the recently created “storytlr” db
[sourcecode lang=”bash”]
mysql -u storytlr -p storytlr < database.sql
[/sourcecode]
#6 You need to install a PECL extension “tidy” to make Storytlr work properly. So if you dont have that installed on your server, you can do this using the following command
[sourcecode lang=”bash”]
apt-get install php5-tidy
or
pecl install tidy
[/sourcecode]
if you install it via “pecl” then dont forget to add this line “extension=tidy.so” in your php.ini. Also in this case, make sure that put tidy.so inside the directory which is mentioned in the php.ini as “extension_dir”
restart your webserver
[sourcecode lang=”bash”]
/etc/init.d/apache2 restart
[/sourcecode]
#7 Now you need to add write permission to some directories that storytlr needs. Change your workign directory to /var/www/storytlr/protected and apply the following commands
[sourcecode lang=”bash”]
chmod 0755 logs
chmod 0755 feeds
chmod 0755 temp
chmod 0755 upload
[/sourcecode]
#8 We are almost done. Now change your working directory to /var/www/storytlr/protected/config and copy the config.ini.sample as config.ini. Edit this config.ini – you need to make various changes here.
#line number 1: if you have PDO and PDO_MYSQL installed in your machine, leave it as is – or else change it to “MYSQL” instead of “PDO_MYSQL”
Change DB settings. Add appropritae uername, db name and password there
Change line number 14, 15, 16 and remove the comments (the semicolon 😉 from in front of them. Its better to add caching.
Change line number 19 and add your host name. You may also change the timezone to your working one.
Change line # 25, set app.closed = 0. You want to enable registration for your visitors, right?
Comment out line number 28 (app.user = admin) and add a semicolon in front of it 🙂
Change line #39 (flickr.api_key=) – add your Flickr API key there. This is optional 🙂
You may also change line number 57 and add a google map api key for your domain. This is also optional 😉
#9 Last step, create /tmp/cache directory and add write permission to it 🙂
My first PHP book was “Beginning PHP4” which I read in 2002. I still remember that dark red cover with smiling faces on top of it :). It was a good book. I should rather say it was a very good one for kick-starting PHP that time.
So far I’ve read a lot of books written on PHP. But when someone asks me which one is my favorite, I get three or may be four which were very good. First one was Professional PHP Programming. Then comes PHP5 Power Programming. Next comes Advanced PHP Programming and Finally I must say about Zend Enterprise PHP Patterns. Of course there are other books which are very good in fact. But these are my favorite (Well, you see that Beginning PHP4 and Professional PHP4 are obsolete by now, but they helped me a lot that time)
I got a bad news this morning that zembly is shutting down their service from 30th next. that’s a very bad news because zembly was a nice service for developers. They’d started their business from june 2007, provided exciting APIs to developers to develop mashups on top of popular services like Facebook, MySpace, Flickr and so forth. I’ve seen another popular service like this , appJet, went down this year. Now it’s extremely frustrating that such types of services are shutting down. Why? most probably they’ve failed to secure funding, or may be they’ve some fight in their board of directors, or whatever, who knows and who cares.
But the point is, what happen to developers who has developed their app using services like zembly or appJet? all their efforts to learn these new technology, new tools and to develop some really cool applications using these services just became useless all at a sudden. That’s pathetic, and thats an utter lack-of-respect to your own developers community. These developers are your everything. They were your everything. If these developers didn’t put their efforts for you zembly, you will never survive a quarter, may be six months.
I really dont understand, If you dont have the guts to run your product thats fine. I have no problem with that. But before saying “All our service will be unavailable from blah blah blah” and “We are sorry and blah blah blah” , come on – show a minimal respect to your developer community. After 2.5 years of run, its not a small community after all. And grow up kiddo, before shutting down your company, at least make your project open source – as a sign of minimal respect and loyalty to the developers community who kept you running for so long.
I am sorry for Zembly But I am more sorry for the developer community.
Many of you already know that managing session is a critical task for web applications, specially when you want to avoid I/O hop and also a significant load over your database by writing a custom session handler. Beside that, if your application makes use of multiple web servers behind a proxy, then its more than a critical job to share and manage session data among these servers effectively. This is why a central session manager is very important for your application to scale. In this article I am going to show you how you can use the latest Memcached extension (developed by Andrei Zmievski and his team) to isolate the session storage from web servers. I will show you how to compile the extension and use it.
Step1: Install Memcached Server
If you are using Debian its just plain simple
[sourcecode lang=”bash”]
apt-get install memcached
[/sourcecode]
Step 2: Run memcached instances
Lets run two instances of memcached in same machine (well, this article is just for demonstrating you how you can get things done. In the production environment, you can deploy as many memcached instances as you want in different servers in same network)
[sourcecode lang=”bash”]
memcached -d -l 127.0.0.1 -p 11211 -u <username> -m 16
memcached -d -l 127.0.0.1 -p 11212 -u <username> -m 16
[/sourcecode]
Above commands will run two instances of memcached listening on port number 11211 and 11212, same IP 127.0.0.1. Each of them get an allocation of 16 MB of memory (on RAM).
Step 3: Install the PECL Memcached extension.
Lets install the new PECL memcached extension in your web server. This new extension depends on libmemcached. You can grab the latest distribution of libmemcached from https://launchpad.net/libmemcached and compile it in your own machine. Make sure you have the dependencies met.
[sourcecode lang=”bash”]
wget http://launchpad.net/libmemcached/1.0/0.34/+download/libmemcached-0.34.tar.gz
tar -zxvf libmemcached-0.34.tar.gz
cd libmemcached-0.34
./configure
make && make install
[/sourcecode]
Considering everything went fine, lets install the PECL memcached extension
[sourcecode lang=”bash”]
pecl install memcached
[/sourcecode]
If everything goes fine, you should see the output similar like this
[sourcecode]
Build process completed successfully
Installing ‘/usr/lib/php5/20060613/memcached.so’
install ok: channel://pecl.php.net/memcached-1.0.0
configuration option "php_ini" is not set to php.ini location
You should add "extension=memcached.so" to php.ini
[/sourcecode]
Make sure that memcached.so is placed in your PHP extension_dir folder (here /usr/lib/php5/20060613). Add the line “extension=memcached.so” in your php.ini and restart your web server.
To make sure, everything’s done and working – run a phpinfo() and check the output. There should be a “memcached” sesction which will look like the following one.
Memcached PECL Extension
Now we need to make change in our php.ini to register Memcached as a session handler and set the necessary properties there. Open your php.ini and add the following two lines. If you find any similar line un-commented, comment them out first.
Restart your web server. And …… you are done! 🙂 – Now all your session data will be saved and served from these memcached servers. No matter whenever you need to extend your setup by adding extra web servers, all user data and session data will remain valid and served from a central location. No I/O issue, no huge write load on DB servers.
We, the developers at Leevio are currently developing an open source group collaboration application. We are using RackSpace Cloud servers since the very beginning of the development and we will also deploy in on rack space cloud.
As a part of our study we’d found that RackSpace provides excellent API to manage your cloud services. Unfortunately there are no language binding library released yet (for Cloud Server). But they’ve mentioned that they are developing one for python.
So we have spend our time for last two days and developed a PHP client library to use RackSpace cloud server API from inside a PHP application.
This client library is released under New BSD license. And is available to download/svn-checkout from it’s project page at http://code.google.com/p/phprackcloud/
There is plenty of documentation and example included. The code is mostly self explanatory. Please dont forget to check the Wiki page to get some idea on how to use this library and how far you can go. We have covered all available methods by RackSpace cloud server in our client library.
Embedding documents in web applications is a challenging work. Because you need to convert these documents into flash movie and only then you can embed them in your web applications. There are many converters available out their for use. Some of them are proprietary, Some of them comes for free. There are command line tools as well as web services also to perform these conversions and make your life easier. In this post I am going to introduce you to IPaper, an excellent service from Scribd. Using IPaper API you can upload and convert your documents to flash movie on the fly, and then you can embed them using Scribd’s excellent document viewer. Scribd offers a collection of client libraries and fortunately, we have a PHP lib in that collection.
Scribd API can convert numbers of document types including the followings
* Adobe PDF (.pdf)
* Adobe PostScript (.ps)
* Microsoft Word (.doc, .docx)
* Microsoft PowerPoint (.ppt, .pps, .pptx)
* Microsoft Excel (.xls, xlsx)
* OpenOffice Text Document (.odt, .sxw)
* OpenOffice Presentation Document (.odp, .sxi)
* OpenOffice Spreadsheet (.ods, .sxc)
* All OpenDocument formats
* Plain text (.txt)
* Rich text format (.rtf)
To use IPaper API, you need to create a developer account in ScribD and get your API key and secret key. Also, download the PHP client library from http://www.scribd.com/developers/libraries
You are ready to go once you get your Scribd api key and secret key. You can upload different types of files to scribd server and of course you can make them private or publicly accessible. After successful upload and conversion you will receive a doc_id and access_key from Scribd web service. Using those data you can embed your document in your web app. Lets have a look. In the following code examples, you may find the getEmbedUrl() and getEmbedCode() functions handy. You can also use SWFObject for quick embedding 🙂
By the way, IPaper PHP client library has some cool methods like uploadFromUrl() – Check them out )
Stay in touch, In next post I will show you how to use some other exciting tools which you can use and host in your own server :). that will be fun to host your own service like this 🙂
wordpress is one of the very popular blogging platforms which is not only free but also released as a open source project. officially wordpress runs only on mysql. so if you want to run it with SQLite, you will not find any official support for that. but fortunately justin addie has written an excellent plugin (pdo for wordpress) which will work as an adapter just between the wordpress DAL and mysql. this plugin hooks all the SQL queries which are sent to execute on mysql by wordpress, convert those into PDO compatible statements and then execute them. currently PDO for WordPress is written to work smoothly with mysql and sqlite.
on the other hand, WPMU (wordpress mu or multi-user) is another version of wordpress which uses the core wp with some modifications and convert any single user wordpress blog into a multi user blogging platform. in this blog post i will show you how you can convert a general installation of wordpress into multi user blogging platform (like WPMU, but fully featured) and take advantage of SQLite 🙂 – so let the fun begin.
step 1: check if your domain support wildcard A record. if so, then create a “*” A record and point it to your server. if it works, now you can point http://.. to your server 🙂 – that is very much required. alo please check that in your webserver it has “pdo” and “pdo_sqlite” php extensions enabled. you can check that using phpinfo(); – needless to say, you need PHP5
step 2: install wordpress
step 3: install pdo for wordpress
download the pdo_for_wordpress plugin. inside it you will find one file named “db.php” and a folder named “pdo”.
put the db.php inside “wp-content” directory
now put the “pdo” folder inside “wp-content” directory
open wp-config.php
find the following line
[sourcecode lang=”php”]
define(‘DB_COLLATE’, ”);
[/sourcecode]
add the following line just below that line
[sourcecode lang=”php”]
define(‘DB_TYPE’, ‘sqlite’);
[/sourcecode]
step 4: create a directory named “userdb” anywhere in your server, make sure to give it write access. in this example we’ve created one as /opt/userdb – DONT FORGET TO ASSIGN WRITE PERMISSION to this “userdb” directory.
step 5: now we will be modifying that “db.php” which we had copied into “wp-content” folder.
find the line in db.php where it says
[sourcecode lang=”php”]
define (‘FQDBDIR’, ABSPATH .’/wp-content/database/’);
define (‘FQDB’, FQDBDIR .’MyBlog.sqlite’);
[/sourcecode]
now change those lines as below
[sourcecode lang=”php”]
define (‘FQDBDIR’, ‘/opt/userdb/’);
define (‘FQDB’, FQDBDIR ."{$_SERVER[‘SERVER_NAME’]}.sqlite");
[/sourcecode]
and tada – you are done. now point your browser to any subdomain under your domain and it will comeup with registration page. register some blog and check it out 🙂 (at the bottom of this post you can find the link of a live demo)
now you can extract some themes in the wp-content/themes folder or some plug-ins into wp-content/plugins folder. all users will be able to choose only from those pre installed themes and plugins, and everyones data and configuration will remain separate from others, as everyone is running from their own database. set only “read” access to “wp-content/themes” and “wp-content/plugins” folders to avoid any security loophole.
happy blogging.
[note – i’ve checked it against latest wordpress 2.8.4 and it runs okay]
To check out a live example of a running wordpress 2.8.4 on SQLite – click on http://anything.twistats.com/wp/ – and to create your own – just go to this url http://<any_subdomain>.twistats.com/wp and register your one :). You can select from six pre installed themes too 🙂
