Category: OpenSource

Installing Storytlr in your own domain

When lifestreaming services come under the spotlight, Storytlr is definitely a promising one. They started their journey not more than a year ago, was loved by many other people out there, and decided to shutdown! Ok, that makes sense when the economy was so scary lately, but the good thing is that – they didn’t just announced “We are shutdown, and we don’t care what the fuck had you built with us”. Those brilliant people behind Storytlr did something very much appreciable, they made their brain child open source. I really loved that. The source code is available to download from their google code project page.

Other than Storytlr, there is another open source life streaming service available at this moment which is “pubwich” and is available to download from http://pubwich.org

My post is about how you can grab the source code of Storytlr and set it up in your own webserver. I’ve seen many people were asking on the mailing list an their news blog about how to install this one successfully. So I decided to give it a shot and install it. Follow these steps and you are good to go 🙂

#1 Add a wildcard A record pointing to your server’s IP
#2 If you are hosted in VPS, and you have multiple domains hosted, you may want to create a Virtual Host with the following entries. I just assumed that your document root for Storytlr will be at “/var/www/storytlr”
[sourcecode lang=”php”]
<VirtualHost *:80>
ServerName domain.tld
ServerAlias domain.tld
ServerAdmin [email protected]
DocumentRoot /var/www/storytlr
<Directory /var/www/storytlr>
Options FollowSymLinks
AllowOverride All
</Directory>
</VirtualHost>
[/sourcecode]

Dont forget to restart/reload your web server after this.

#3 change your current workign directory to /var/www/storytlr and download the latest available storytlr from google code, and then extract it 🙂
[sourcecode lang=”bash”]
wget http://storytlr.googlecode.com/files/storytlr-0.9.2.tgz
tar -zxvf storytlr-0.9.2.tgz
[/sourcecode]

Now you will see some directories in the same path.

#4 Create a mysql database named “storytlr” (You can name it whatever, just dont forget to change the DB settings accordingly in the config. I will be discussing this later) – also create a user for this db and grant it all permission for this db.
[sourcecode lang=”sql”]
mysql> create database storytlr;
mysql> grant all on storytlr.* to ‘storytlr’@’localhost’ identified by ‘password’;
mysql> flush privileges;
[/sourcecode]

#5 Now you need to import storytlr’s schema into it. change your working directory to /var/www/storytlr/protected/install and import the database.sql into the recently created “storytlr” db
[sourcecode lang=”bash”]
mysql -u storytlr -p storytlr < database.sql
[/sourcecode]

#6 You need to install a PECL extension “tidy” to make Storytlr work properly. So if you dont have that installed on your server, you can do this using the following command
[sourcecode lang=”bash”]
apt-get install php5-tidy
or
pecl install tidy
[/sourcecode]

if you install it via “pecl” then dont forget to add this line “extension=tidy.so” in your php.ini. Also in this case, make sure that put tidy.so inside the directory which is mentioned in the php.ini as “extension_dir”

restart your webserver
[sourcecode lang=”bash”]
/etc/init.d/apache2 restart
[/sourcecode]

#7 Now you need to add write permission to some directories that storytlr needs. Change your workign directory to /var/www/storytlr/protected and apply the following commands
[sourcecode lang=”bash”]
chmod 0755 logs
chmod 0755 feeds
chmod 0755 temp
chmod 0755 upload
[/sourcecode]

#8 We are almost done. Now change your working directory to /var/www/storytlr/protected/config and copy the config.ini.sample as config.ini. Edit this config.ini – you need to make various changes here.

#line number 1: if you have PDO and PDO_MYSQL installed in your machine, leave it as is – or else change it to “MYSQL” instead of “PDO_MYSQL”

Change DB settings. Add appropritae uername, db name and password there

Change line number 14, 15, 16 and remove the comments (the semicolon 😉 from in front of them. Its better to add caching.

Change line number 19 and add your host name. You may also change the timezone to your working one.

Change line # 25, set app.closed = 0. You want to enable registration for your visitors, right?

Comment out line number 28 (app.user = admin) and add a semicolon in front of it 🙂

Change line #39 (flickr.api_key=) – add your Flickr API key there. This is optional 🙂

You may also change line number 57 and add a google map api key for your domain. This is also optional 😉

#9 Last step, create /tmp/cache directory and add write permission to it 🙂

Done and Enjoy!

Using new PECL Memcached extension for storing session data

Many of you already know that managing session is a critical task for web applications, specially when you want to avoid I/O hop and also a significant load over your database by writing a custom session handler. Beside that, if your application makes use of multiple web servers behind a proxy, then its more than a critical job to share and manage session data among these servers effectively. This is why a central session manager is very important for your application to scale. In this article I am going to show you how you can use the latest Memcached extension (developed by Andrei Zmievski and his team) to isolate the session storage from web servers. I will show you how to compile the extension and use it.

Step1: Install Memcached Server
If you are using Debian its just plain simple
[sourcecode lang=”bash”]
apt-get install memcached
[/sourcecode]

Step 2: Run memcached instances
Lets run two instances of memcached in same machine (well, this article is just for demonstrating you how you can get things done. In the production environment, you can deploy as many memcached instances as you want in different servers in same network)
[sourcecode lang=”bash”]
memcached -d -l 127.0.0.1 -p 11211 -u <username> -m 16
memcached -d -l 127.0.0.1 -p 11212 -u <username> -m 16
[/sourcecode]

Above commands will run two instances of memcached listening on port number 11211 and 11212, same IP 127.0.0.1. Each of them get an allocation of 16 MB of memory (on RAM).

Step 3: Install the PECL Memcached extension.
Lets install the new PECL memcached extension in your web server. This new extension depends on libmemcached. You can grab the latest distribution of libmemcached from https://launchpad.net/libmemcached and compile it in your own machine. Make sure you have the dependencies met.

[sourcecode lang=”bash”]
wget http://launchpad.net/libmemcached/1.0/0.34/+download/libmemcached-0.34.tar.gz
tar -zxvf libmemcached-0.34.tar.gz
cd libmemcached-0.34
./configure
make && make install
[/sourcecode]

Considering everything went fine, lets install the PECL memcached extension
[sourcecode lang=”bash”]
pecl install memcached
[/sourcecode]

If everything goes fine, you should see the output similar like this
[sourcecode]
Build process completed successfully
Installing ‘/usr/lib/php5/20060613/memcached.so’
install ok: channel://pecl.php.net/memcached-1.0.0
configuration option "php_ini" is not set to php.ini location
You should add "extension=memcached.so" to php.ini
[/sourcecode]

Make sure that memcached.so is placed in your PHP extension_dir folder (here /usr/lib/php5/20060613). Add the line “extension=memcached.so” in your php.ini and restart your web server.

To make sure, everything’s done and working – run a phpinfo() and check the output. There should be a “memcached” sesction which will look like the following one.

Memcached PECL Extension
Memcached PECL Extension

Now we need to make change in our php.ini to register Memcached as a session handler and set the necessary properties there. Open your php.ini and add the following two lines. If you find any similar line un-commented, comment them out first.

[sourcecode lang=”php”]
session.save_handler=memcached
session.save_path="127.0.0.1:11211, 127.0.0.1:11212"
[/sourcecode]

Restart your web server. And …… you are done! 🙂 – Now all your session data will be saved and served from these memcached servers. No matter whenever you need to extend your setup by adding extra web servers, all user data and session data will remain valid and served from a central location. No I/O issue, no huge write load on DB servers.

some very useful apps for mac osx, free as well :)

i am a big time fan of mac osx. if you think there are no free+useful app for mac, you are quite wrong. here is my personal favorite list of some free and very useful apps which i use everyday.

CyberDuck CyberDuck: it is my most favorite FTP and SFTP client, and its really very cool. Well, you can use it as a WebDAV and S3 client too. Its open source and you can check it out from here

ALunch: if you use your dock very frequently and are really tired to reconfigure it again and again, alunch is a very nice sticky app launcher for you. it’s organiser is simply awesome and you can arrange all your applications in a a well categorized manner easily. its very nice and one of my very favorites. Check it out from here

ALunch

iStat Pro and Menus: though it has a name “pro” but it comes for free. it is an awesome device monitoring tool and supplies you very essential information about your mac device, like bandwidth consumption (both eth and bt) temperature, fan RPM, memory usage, cpu usage and it supplies each of these with details. download it from here – btw, forgot to tell you that you can install it in your iphone and ipod touch too!

iStat Pro

Chicken of VNC: it is a nice VNC client for mac. if you have multiple macs in your home or you want to use it as a generic VNC client to your other machines, from you mac – it is the perfect tool. open source and free 🙂 – check it out from here

Caffeine: it is another very useful and handy when you really dont want your screen to black out (heh heh). it sits on your top bar with a icon of coffee cup, and once you click on it the machine will stay active for 30 minutes (configurable). no screensaver, no dimming – its show time :D. check it out from here

Caffeine

বাংলা স্ক্রিপ্ট গুলোর রিফ্যাক্টরিং এর কাজ শুরু করলাম – সাহায্য দরকার আপনাদের থেকে

প্রথম স্ক্রিপ্ট টি লিখেছিলাম সেই ২০০৪ সালে, আসকি মোডে বাংলা এক্সপ্রেসে বিজয় ইন্টারফেসে লেখার সুবিধা দিতে। এরপর মেজর রিফ্যাক্টরিং হয়েছিল ২০০৬ সালে সামহোয়্যারইন ব্লগ ইউনিকোড করার সময় (ইউনিজয়) । তারপর এসেছে ফোনেটিক, ২০০৬ সালের জুলাইয়ে।

এরপর থেকে স্ক্রিপ্টগুলো অলমোস্ট আগেরমতই আছে, মাঞ্চু মাহারা (সবুজ কুন্ডু) আপডেট করেছে এগুলো, তারপর আমি অ্যাড করেছি ইন্টেলিজেন্ট ফোনেটিক পার্সার টা। মাঝে এসেছে প্রভাত এবং ইনস্ক্রিপ্ট (মাঞ্চু)। সবশেষে প্রথম আলো ব্লগ তৈরী করার সময় রিলিজ দিয়েছি বিজয় (ইউনিজয় না) এবং ভার্চুয়াল কীবোর্ড

আমার জানামতে রাজু (প্রজন্ম ফোরামের), ইফতেখার এবং রাশেদ (আমার ব্লগের) এবং রায়হান (নির্মান ব্লগের) ছাড়া ছাড়া ভাবে তাদের অ্যাপ্লিকেশনের জন্য কিছু কুইক ফিক্স/হ্যাক ছেড়েছেন, ধন্যবাদ তাঁদেরকেও।

আমার এবং মাঞ্চুর প্রতিটা রিলিজের পিছনে কাজ করেছে অমি আজাদ। প্রতিটা স্ক্রিপ্ট রিলিজ দেয়া হয়েছে এলজিপিএল লাইসেন্সের অধীনে (তবে খুব শিঘ্রই আমি এটা চেঞ্জ করে নিউ-বিএসডি করে ফেলব )

এখন আমি চাইছি ফিক্স হগুলো একসাথে করতে এবং আপনাদের পরামর্শ অনুযায়ী স্প্রিপ্ট গুলো আপডেট করতে। সেজন্য আপনার যদি আমাকে আপনাদের পরামর্শগুলো জানান আমি চেষ্টা করব সে অনুযায়ী একটা রিলিজ প্ল্যান করতে। তাতে আশাকরি সবার জন্যই স্ক্রিপ্ট গুলো আরো বেশী করে কাজে আসবে (কারন এখন অলমোস্ট ৯০% বাংলা ওয়েব অ্যাপগুলোতে এই স্ক্রিপ্ট গুলো ব্যবহার করা হয়)

আমাকে পরামর্শগুলো জানান [email protected] এই ইমেইলে। কারো যদি সচলায়তন বা আমার ব্লগ বা প্রথম আলোতে অ্যাকা্উন্ট থেকে থাকে অনুগ্রহ করে সেখানে একটা ডুপ্লি পোস্ট করবেন কি?

বাংলা কম্পিউটিং সহজ হোক সবার জন্য – আরো সহজ হোক দিন দিন – এই কামনায়। আমি আশা করব আপনারা যারা প্রোগ্রামিং জানেন (বা জানেন না) এগিয়ে আসবেন ওপেন সোর্স বাংলা কম্পিউটিং কে সমৃদ্ধ করতে, সামর্থ্য অনুযায়ী।

ধন্যবাদ সবাইকে।

using oauth pecl extension to talk to twitter

if you are interested in developing twitter applications, you must have read about twitter API and it’s authentication protocol. your application can fetch user’s private data but it has to authenticate itself as the user for that. so there are two ways to do it

1. asking user to provide his twitter username and password to your application (well, i am not interested to give away my PASSWORD to anyone!!!)
2. let twitter handle the authentication on behalf of you and ask user to grant permission to your application (hmm!! interesting)

now you see that #2 is more safe for your user. and i think most security concerned users will choose this way. so your application have to initiate this type of authentication system using twittter’s supported authentication protocol oAuth (it’s a commonly used authentication protocol used among number of popular service providers like yahoo, google and others)

to implement oauth in php, the best way is to use an existing library. there are now numbers of libraries available for this purpose. following are some of them
1. oauth lib by andy smith
2. oauth library by marc worrell
3. oauth pecl extension by rasmus lerdorf and john jawed and felipe pena

now you see, pecl extensions are written in c and runs pretty faster. so i choose it without thinking much abt it. i have assumed that you know how to install a pecl extension in your php hosting and i am not going to blog detail about that right now. all that can help you right now is shell command “pecl install -f oauth” – you know, nothing talks better than command or code 🙂

after installing oauth extension in my hosting account, i start developing my twitter application. first i have to register my application with twitter. you can create your one by pointing your browser to http://twitter.com/oauth_clients/new. please remember that you have to provide a callback url which twitter use to redirect user of your application after a success/unsuccessful authentication. i will refer to that url as “callback_url” through out this blog post. my applications callback_url is “http://mydomain.tld/auth.php”

after you have done registering your application with twitter, it will give you the following important data.
1. consumer key
2. consumer secret
3. request token url
4. access token url
5. authorize url

you will be going to use all of these in your application. now lets see how oauth works in brief. it initiate the talk using your consumer key and secret key. and then it request the “request token” from the service provider. if u r successful, you have to forward user of your application to the “authorize url” with the “request token”. now the service provider will ask to grant permission to your application from the user. if user grants (or disagree) the permission, the service provider (here, twitter) will forward your user again to the “callback url” of your application with a “new token”. now with the help of this new token and the token grabbed from previous “request token” your application will ask for “access token”. once you have the access token, you can authorize you application as the user itself with same privilege.

lets see how to do it in php with the help of oauth pecl extension. here we are going to initiate the talk, get the token and forward user to the service provider’s authorizing url.

token.php
[sourcecode lang=”php”]
< ?php //token.php $oauth = new OAuth("consumer key","consumer secret",OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); //initiate $request_token_info = $oauth->getRequestToken(“http://twitter.com/oauth/request_token”); //get request token
file_put_contents(“token.txt”,$request_token_info[‘oauth_token_secret’]);//store the oauth token secret of request token
header(‘Location: http://twitter.com/oauth/authorize?oauth_token=’.$request_token_info[‘oauth_token’]);//forward user to authorize url
?>
[/sourcecode]

you see that we are storing the oauth_token_secret of the “request_token” because we need it in our next step to fetch access token. in the example above i am storing it in flat file, but you will have to store it in db/file with proper index to the userid so that you can retrieve it later in our next step.

if user visit this page, he will be redirected to twitter authorize url and that may look like the following one with different app name.
picture-26

now lets see how we handle if the user click “allow” or “deny” in the above page.

this is the callback file you specified in settings of your app [auth.php]
[sourcecode lang=”php”]
< ?php //auth.php $oauth = new OAuth("consumer key","consumer secret",OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); //initiate $request_token_secret = file_get_contents("token.txt"); //get the oauth_token_secret of request token we stored previously if(!empty($_GET['oauth_token'])){ $oauth->setToken($_GET[‘oauth_token’],$request_token_secret);//user allowed the app, so u
$access_token_info = $oauth->getAccessToken(‘http://twitter.com/oauth/access_token’);
}
?>
[/sourcecode]

access token is the most important token for your application. there are two object in this token – one is “oauth_token” and “oauth_token_secret”. if you print_r the access token it will look like the following one (actual value is not shown here)

Array (
    [oauth_token] => abcdefg
    [oauth_token_secret] => uvwxyz
)

you have to store this access token for authorizing later as this user (the user that was visiting). using this token you can anytime authorize yourself as that user and fetch user’s data from twitter. so lets see how we can fetch user’s profile data in rss (or json) format. the REST API url to fetch this data is “http://twitter.com/account/verify_credentials.json”. you can find other important REST urls to fetch user’s timeline, public timeline and friends timeline (also update status) in twitter’s documentation of it’s REST API

fetch user’s profile data
[sourcecode lang=”php”]
< ?php //profile.php $oauth = new OAuth("consumer key","consumer secret",OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); //initiate $oauth->setToken($accesstoken[‘oauth_token’],$accesstoken[‘oauth_token_secret’]);
$data = $oauth->fetch(‘http://twitter.com/account/verify_credentials.json’);
if($data){
$response_info = $oauth->getLastResponse();
echo “

";
    print_r(json_decode($response_info));
    echo "

“;
}
[/sourcecode]

the output of this code is the following one (my twitter username is hasin)

[sourcecode lang=”php”]
stdClass Object
(
[time_zone] => Dhaka
[friends_count] => 97
[profile_text_color] => 666666
[description] => Smoking too much PHP
[following] =>
[utc_offset] => 21600
[favourites_count] => 2
[profile_image_url] => http://s3.amazonaws.com/twitter_production/profile_images/84574185/afif_b_normal.jpg
[profile_background_image_url] => http://s3.amazonaws.com/twitter_production/profile_background_images/5565492/777481225666153.jpg
[profile_link_color] => 2FC2EF
[screen_name] => hasin
[profile_sidebar_fill_color] => 252429
[url] => http://hasin.wordpress.com
[name] => hasin
[protected] =>
[status] => stdClass Object
(
[text] => ok, understood how twitter auth works via oauth pecl ext. of #php. thanks to @rasmus for his excellent example
[in_reply_to_user_id] =>
[favorited] =>
[in_reply_to_screen_name] =>
[truncated] =>
[created_at] => Sat May 02 16:08:28 +0000 2009
[id] => 1679349376
[in_reply_to_status_id] =>
[source] => web
)

[profile_sidebar_border_color] => 181A1E
[profile_background_tile] => 1
[notifications] =>
[statuses_count] => 1147
[created_at] => Fri Nov 09 10:40:14 +0000 2007
[profile_background_color] => 1A1B1F
[followers_count] => 265
[location] => Dhaka, Bangladesh
[id] => 10094392
)
[/sourcecode]

develop your own gtalk/jabber/aim/yahoo bot using imified API

i’ve found http://bot.im (imified) few days ago and it’s a nice free service for creating your own messenger bots. it provides excellent API to develop bots for various platform like AIM/Yahoo/Gtalk/Jabber and interact with your bot users. To ease developing bots, I have written a wrapper class of imified API for php developers and made it open source under BSD license. You can download the wrapper from below

Download the wrapper with example

To see the example bots developed using this wrapper, please add “[email protected]” in your gtalk/jabber client or “storytellerbot” in your YM client. then type help and TADA!

please note that processing “help” is different. you need to type the response of “help” command in your bot settings page. also bots allows only one HTML tag which is <br>

example [the callback of this bot is set to http://bot.ofhas.in]

include_once("class.imified.php");
if(!isset($_REQUEST['msg']))
{
die ("This is Hasin's personal bot [email protected] to demonstrate the imified API - please add this bot from your gtalk/jabber client Or add [email protected] in your YM client. Source code is available from hasin's personal blog at http://hasin.wordpress.com");
}
//initiate
$im = new ImifiedHelper("7DCDECB6-C895-9643-909CDC85CBF09954 ","[email protected]","Oye!MyPassword!");
//callback
$im->setCallback("callback");
function callback($message, $step, $network, $userKey)
{
global $im;
$message = strtolower($message);
if("whois"==$message)
{
echo "about me";
}
else if("work"==$message)
{
echo "about my work";
}
elseif("status"==$message)
{
echo "my status";
}
elseif("quit"==$message)
{
echo "ok, bye";
}
else
{
echo "Sorry, I dont understand hierogliphics @^#%@*&#(&!*&@^!*&@#!!!";
}
$im->resetStep();
}

This class also supports fetching user info and sending message to a group of bot users. Check it out

Happy bot development

wrapper for google static map api

google maps is one of the very best mapping services available these days. using their api you can embed interactive maps of different types in your web pages. and there are many plugins to make your life easier to embed such maps in your web pages and applications.

to embed interactive google map, you need to include some external javascript files first. now when you are using some hosted solutions like wordpress, they may not allow you to include external javascript files in your blog/pages for different security pre-cautions. this is where google static map api comes handy.

google static maps api takes latitude and longitude of some places as parameter, along with other optional formatting params and delivers a static image in your expected image format (default is gif, but you can choose among gif, png and jpeg). so you dont need to bother about permission of including any other javascript files in your pages and you can embed such static maps by specifying the url as a “src” attribute of <img> tag.

there is still one problem that might knock you down. for example if you want to display “dhaka”, the capital or bangladesh in your static map, you need to know it’s latitude and longitude. so not everyone of us speaks in a language of ‘latitude’ and ‘longitude’ but onle the name of the places. like we all know about “university of berkeley” but only very few can figure out it’s lat-lon or they have to find it out from existing geocoding services.

yesterday night i was studying about this static map api and decided to build a geocoding wrapper so that you can pass only the name of a place and my wrapper class will convert it to latitude and longitude using geociding services and then generate a static map image for you. the whole wrapper will work around two objects – geocoding service and google static map api. for geocoding i prefer to use google because of it’s amazing precision but i must say that yahoo’s geocoding service is also pretty good.

here is my wrapper class and i am sure you can find it useful. now you can use it to embed google map to anywhere you would like to :). this class is released under LGPL.

there is a sample usage file (gmap.php which you will make use of) and a source object “staticgmap.php”. this class became a part of orchid framework and prothom alo blog. if you enjoy/use this class, dont forget to drop a line here in this post. the class and usage file is moderately commented, so you will not face any problem to understand. however the usage file is also pretty fine working.

please download the class from this page
download: http://svnpg.net/gmap/

supported features
1. geocoding service, both yahoo and google (default is yahoo)
2. support of zooming (0-19, default is 12)
3. support of image format (gif, jpeg and png. default is GIF)
4. maptype support (roadmap, satellite, mobile, hybrid and terrain where default is roadmap)
5. support of markers (just as plain location name)
6. variable dimension support (default size is 400×400)

sample maps

locate dhaka and moti jheel
http://svnpg.net/gmap/gmap.php?location=dhaka&markers=dhaka,moti jheel
dhaka and motijheel marked on a static google map

locate berkeley and emeryvlle
http://svnpg.net/gmap/gmap.php?location=berkeley&markers=berkeley,emeryville&zoom=11
berkeley and emeryville marked on google static map


place where i live – uttara 14 with custom zoom level and custom size

http://svnpg.net/gmap/gmap.php?location=uttara 14&zoom=16&size=400×300

uttara 14

Look Ma, everyone's computing out there for me!

SETI@Home is probably the greatest example of low cost distributed computing which become a big hit. After their tremendous success, many others over there started following the same strategy and used the power of distributed computing for other purposes like cancer research. In this article I will show you how we can use the same power at almost zero cost, and specially for your web applications.

As I am currently working on building an open source version of FriendFeed (not targetted to be an alternative, coz those people at FriendFeed done their job really well) and scaling such a huge load effectively at low cost, so I will mainly talk more about FriendFeed through out this blog post and use it as an example for my proposal.

If you consider FriendFeed as a repository of feed URLs, a lot of peoples and how they are related to each other, you can assume how big it is or it could be in near future. And scaling such a service would cost numbers of sleepless nights of many developers out there. So in basic, lets focus where actually the problem is and how we can introduce distributed computing.

Beside optimizing database to serve huge sets of data, one of the main problems of such a service has to parse millions of feeds in a regular interval. If we want to bear all the loads on your server, fine, if you can afford. But what about some low cost solutions. Lets consider a simple scenario, if your application has one million of users and each of them browse your application for 10 minutes a day, you really have 10 millions of computational power just wasting over there, in lanes and by lanes of internet – heh heh. So lets make use of such an incredible CPU power. All you have to do let the visitors machine do some calculations for you and free your server from gigantic load.

When the users of your application and relation among them are stored in your database, you can easily find out the first degree and second degree friends of a specific user. If you don’t know what does that mean, its simple, If A is a friend of B and C is a friend of A, then A is B’s first degree friend and C is B’s second degree friend. For a huge social network, it may look like the following one when you visualize the relationship


image courtesy: http://prblog.typepad.com

Now what we want to do is when B is visiting our application we want to parse most of his/her second degree friends in client using his browser. So while generating the page for B, we will supply him a bunch of feed URLs, a hash of their last known update time or hash of the latest item of each of these corresponding feeds, and a javascript based parser (for example Google’s ajax Feed API would do fine) script. Now, while B is browsing our application we will parse those feeds of his second degree friends using javascript without bothering him for a single second and post back the parsed contents (after checking against the hash for really updated contents) to a server side script which will then blindly (not totally, after some validation or authentication for sure) insert those result to our database. Now when A comes and visit his page (A is C’s first degree friend), he will get all the latest result from C’s feeds as B has already done the parsing job for us and we have those latest result from C’s feeds stored in our Database.

There are definitely more challenge than it is explained here, like what if a person is second degree friend of a multiple user. In such cases as we supplied last update time of these feeds while generating a page, we can calculate from server side which feeds we really want to parse.

And moreover, we can add more check for our javascript parser than just blindly parse those feeds. We can download a chunk of RSS or Atom feeds (using a proxy script developped using curl range options) and read just up to the latest update time and extract the time using simple reg ex or string functions instead of downloading full feed data. Now if we know that a script is not uptodate just by downloading 1-2 killobyte of data instead of downloading full feed, parsing the xml data, it would save us more computing resources for performing other jobs.

But of course, you cannot live completely on your client’s machine for parsing all your data. You must have several cron’d scripts to parse left overs or other feeds at server side. But what I am saying is that with little help of javascript you can make use of such a tremendous distributed computing power in your application, all at almost no cost.

I will come with example code once I am done developing my open source clone of FriendFeed and then I am sure, you will find it was worth writing a blog post about.

Distributed Computing
image courtesy: http://www.naccq.ac.nz/bacit/0203/2004Caukill_OffPeakGrid.htm

Have a nice RnDing time. 🙂

How to make your own springloops in PHP

Springloops is a nice code management service recently came into focus. It helps you to manage the code base of your application, monitor the commit and deploy the final version easily to another server. So if you are wondering how to build such a system and how it actually works, this article is for you.

Please note that I am neither way affiliated with Springloops nor any of it’s contacts. This article expresses completely my own opinion.

The primary obstacles of making such a service are
1. Managing the subversion repositories and users
2. Interaction with subversion repositories (Not same as option 1)
3. Payment gateways
4. Code browser
5. User friendliness
6. Scaling

Among these options, 4 and 6 are out of the scope of this article. There are thousands of article explaining those topics to you. So I am not going to talk about them. There are excellent libraries available to manage many popular payment gateways. Just google it and you are done. And about user friendliness and design, you can hire a consultant who’s specializing on this subject and get it done.

1. Managing the subversion repositories and users
In springloops or any other subversion hosting service, you need to do some basic shell scripting which is required for adding the subversion repositories dynamically and to add users in it. There are some excellent articles how you can do it with apache2. To avoid any traffic related issue, best practice will be hosting these repositories in another server, and using htaccess – just point your user’s svn URL to the exact url. Check out the following urls to find out how to host subversion repositories with apache. Linux should be the first choice, heh heh.

Reference:
1. Setting up subversion with apache in Ubuntu gutsy gibon
2. Setting up subversion with apache2
3. Setting up subversion with apache2 and DAV in debian
4. Host your open source projects in ubuntu in 3 easy steps

All you have to do is writing shell script to automate these steps and restart the apache2 demon once an user register his/her project with your service. You are done

2. Interaction with subversion repositories (Not same as option 1)
Now this is quite a challenging part and many of you are lost how to achieve this kind of functionality for such services. For PHP developers, you know there are an excellent repository of extensions named PECL and also repository of libraries name PEAR. In PECL there is an excellent extension which is cent percent appropriate for this work. Yup, I am talking about “SVN” extension. You have to install this extension to use it with your PHP code. Once you are done with installing it, now you got the complete power to interact with the subversion repositories hosted by your users. This extension is very rich and provides all the functionalities you need to interact. For details please point your browser to the appropriate section in PHP manual.

Reference
1. Installing PECL SVN extension for PHP in Ubuntu Gutsy Gibon
2. PECL svn package
3. SVN extension documentation

4. Code Browser
Another challenging part of the total setup. If you cannot provide excellent code browser which will clearly demonstrate the changes in code from different revisions, figure out the log and the notes made during each commit and finally, make note on it to show to your fellow team members.

Using svn_diff, svn_ls and svn_log you can clearly find out the difference of a file in two different revisions, the file and directory structure of the repository and the log of each commit. So basically once you got the file and directory structure of your repository for a revision, you can just traverse through it and display as a nice report using your PHP code. And when user will ask you to display the difference, you will find the difference using svn_diff which returns the difference as standard diff format. Now using regular expression (or whatever way) you will mark the lines which was changed and just display them highlighting using different colors to your user, as a nice report. And You can also browse the log of any revision using svn_log and show it to your user.

And last but not the least, you can create tags, branch and whatever addition by maintaining a local shadow working copy of your repository. Only the thing that you cannot do (or still I am thinking how-to) is merging, heh heh.

And when it comes about deploy the latest code base to your server, you can do it using svn_update where the path is the working directory in your server.

Reference
1. Standard Diff Format
2. svn_diff
3. svn_log
4. svn_ls
5. svn_add
6. svn_update
7. svn_checkout

Basically this is how a service like Springloops is built. I hope, you’ve enjoyed this article as much as I did to write it, heh heh.

WorldTimeEngine – How about making your own in PHP?

I recently came by this site WorldTimeEngine where users can search the local time of any place using the name, street address or just latitude and longitude. Since that time I was thinking how easily you can make your own. As long there are some good people over there (For Geocoding API) – its a not a big deal, you know? Lets have a look at the following PHP code.


<?
$place = "Bucharest";

$geodata = getGeoCode($place);
print_r($geodata);
echo getTime($geodata['lat'],$geodata['lng']);

function getTime($lat, $lng)
{
$url = "http://ws.geonames.org/timezone?lat={$lat}&lng={$lng}";
$timedata = file_get_contents($url);
$sxml = simplexml_load_string($timedata);
return $sxml->timezone->time;
}
function getGeoCode($address)
{
$_url = 'http://api.local.yahoo.com/MapsService/V1/geocode';
$_url .= sprintf('?appid=%s&location=%s',"orchid_geocode",rawurlencode($address));
$_result = false;
if($_result = file_get_contents($_url)) {
preg_match('!<Latitude>(.*)</Latitude><Longitude>(.*)</Longitude>!U', $_result, $_match);
$lng = $_match[2];
$lat = $_match[1];
return array("lat"=>$lat,"lng"=>$lng,"address"=>$address);
}
else
return false;
}
?>

Changing the variable “$place” to “Bucharest”,”Dhaka”, “Toronto” and “Oslo” gives me the following result


Array
(
[lat] => 44.434200
[lng] => 26.102955
[address] => Bucharest
)
2008-03-01 08:41

Array
(
[lat] => 23.709801
[lng] => 90.407112
[address] => Dhaka
)
2008-03-01 12:42

Array
(
[lat] => 43.648565
[lng] => -79.385329
[address] => Toronto
)
2008-03-01 01:42

Array
(
[lat] => 59.912280
[lng] => 10.749980
[address] => Oslo
)
2008-03-01 07:43

Nice, Huh?