Category: performance

Caching Ajax requests in WordPress

In modern WordPress themes, theme developers are serving the page requests via ajax. Both paginated blog page and single posts (might be in full or partial) are served. Though WordPress is very fast if properly tuned, but it’s meaningless to pull the same data from WordPress database again and again unless there is any change. Which means, you can safely cache such ajax requests and reduce loads on your database by adding a few lines of extra code in your theme’s functions.php. In this article, we will learn how to take advantage of such caching in WordPress

Step 1: Register Ajax callback for serving paginated blog pages
Here is a typical codeblock that can serve paginated blog pages via ajax. There is no caching yet
[sourcecode language=”php”]
add_action("wp_ajax_blog", "get_blog");
add_action("wp_ajax_nopriv_blog", "get_blog");

function get_blog(){
global $post;
$page = $_REQUEST[‘page’];
if(!$page) $page=1;
$pp = get_option("posts_per_page");
$offset = ($page*1-1)*$pp;
$args = array(
"posts_per_page"=>$pp,
"offset"=>$offset
);
$posts = get_posts($args); //get the posts
foreach($posts as $post){
setup_postdata( $post );
get_template_part("templates/post",get_post_format($post->ID));
}
die();
}
[/sourcecode]

Now if you go to the url “http://your_blog_url/wp-admin/admin-ajax.php?page=1 ” you can see the latest posts, change the page value to 2,3,4 and so on to see previous posts. Each posts in the loop are rendered via get_templates(). Please note how we used the get_post_format() function in the loop to serve different templates based on post formats.

So what is bad in this code? Nothing, yet Everything! If 100 users are visiting your blog and they are visiting blog pages, each and every time these posts are pulled from database. Each and everytime they are pulling the very same data until you had edited any post, or created a new post. So why this extra load should go to DB? lets minimize it ๐Ÿ™‚

Step 2: Add caching layer
here is a revised format of the code block above, but now with caching.
[sourcecode language=”php”]
add_action("wp_ajax_blog", "get_blog");
add_action("wp_ajax_nopriv_blog", "get_blog");

function get_blog(){
global $post;
$page = $_REQUEST[‘page’];
if(!$page) $page=1;

/** supply from cache **/
$page_from_cache = get_option( "artpress_page{$page}", 0 );
if($page_from_cache){
die($page_from_cache);
}
/** cache end **/

$pp = get_option("posts_per_page");
$offset = ($page*1-1)*$pp;
$args = array(
"posts_per_page"=>$pp,
"offset"=>$offset

);
$posts = get_posts($args);

ob_start();
foreach($posts as $post){
setup_postdata( $post );
get_template_part("templates/post",get_post_format($post->ID));
}
$new_cache = ob_get_flush();

/** save the cache **/
update_option( "artpress_page{$page}",$new_cache);
/** end cache **/
die($new_cache);
}
[/sourcecode]

In the code above, we first checked the page number. Then, checked in the wordpress options table if there is a cache present with a key named “artpress_page”. Now, for the first time it wont be there. So we just pull the records from the database as usually and then using the output buffering functions we capture the output, instead of pushing instantly to the browser. And then we update the options table with this output. So next time when same page will be requested, database wont be hit. Ok, database will be hit (this options table data is coming from the db) but you can see it wont touch the posts table again, and avoid running expensive queries (which includes many joins) in that table. It also saves CPU a bit.

Step 3: Invalidate cache whenever a post is edited, or created
This is an important step. We need to flush the cache whenever a new post or page is created to make sure our blog shows the up to date data.

[sourcecode language=”php”]
add_action("save_post","invalidate_cache");
function invalidate_cache(){
$counts = wp_count_posts();
$total_published_posts = $counts->publish;
$posts_per_page = get_option( "posts_per_page");
$total_page = ceil($total_published_posts/$posts_per_page);
for($i=1;$i<=$total_page;$i++){
delete_option( "artpress_page{$i}" );
}
}
[/sourcecode]

Now, we are in good shape. Our ajax requests are served from cache and properly purges whenever necessary. You can serve your single post same way.

If you want to serve single posts from the cache, one thing you need to keep in mind is that you are loading up an important table “options” with a lot of data. So you can change the code above and store the cache in filesystem instead of options table. You should have write permission in the uploads directory by default. So that wont be tough at all.

I hope you enjoyed this article ๐Ÿ™‚

Using new PECL Memcached extension for storing session data

Many of you already know that managing session is a critical task for web applications, specially when you want to avoid I/O hop and also a significant load over your database by writing a custom session handler. Beside that, if your application makes use of multiple web servers behind a proxy, then its more than a critical job to share and manage session data among these servers effectively. This is why a central session manager is very important for your application to scale. In this article I am going to show you how you can use the latest Memcached extension (developed by Andrei Zmievski and his team) to isolate the session storage from web servers. I will show you how to compile the extension and use it.

Step1: Install Memcached Server
If you are using Debian its just plain simple
[sourcecode lang=”bash”]
apt-get install memcached
[/sourcecode]

Step 2: Run memcached instances
Lets run two instances of memcached in same machine (well, this article is just for demonstrating you how you can get things done. In the production environment, you can deploy as many memcached instances as you want in different servers in same network)
[sourcecode lang=”bash”]
memcached -d -l 127.0.0.1 -p 11211 -u <username> -m 16
memcached -d -l 127.0.0.1 -p 11212 -u <username> -m 16
[/sourcecode]

Above commands will run two instances of memcached listening on port number 11211 and 11212, same IP 127.0.0.1. Each of them get an allocation of 16 MB of memory (on RAM).

Step 3: Install the PECL Memcached extension.
Lets install the new PECL memcached extension in your web server. This new extension depends on libmemcached. You can grab the latest distribution of libmemcached from https://launchpad.net/libmemcached and compile it in your own machine. Make sure you have the dependencies met.

[sourcecode lang=”bash”]
wget http://launchpad.net/libmemcached/1.0/0.34/+download/libmemcached-0.34.tar.gz
tar -zxvf libmemcached-0.34.tar.gz
cd libmemcached-0.34
./configure
make && make install
[/sourcecode]

Considering everything went fine, lets install the PECL memcached extension
[sourcecode lang=”bash”]
pecl install memcached
[/sourcecode]

If everything goes fine, you should see the output similar like this
[sourcecode]
Build process completed successfully
Installing ‘/usr/lib/php5/20060613/memcached.so’
install ok: channel://pecl.php.net/memcached-1.0.0
configuration option "php_ini" is not set to php.ini location
You should add "extension=memcached.so" to php.ini
[/sourcecode]

Make sure that memcached.so is placed in your PHP extension_dir folder (here /usr/lib/php5/20060613). Add the line “extension=memcached.so” in your php.ini and restart your web server.

To make sure, everything’s done and working – run a phpinfo() and check the output. There should be a “memcached” sesction which will look like the following one.

Memcached PECL Extension
Memcached PECL Extension

Now we need to make change in our php.ini to register Memcached as a session handler and set the necessary properties there. Open your php.ini and add the following two lines. If you find any similar line un-commented, comment them out first.

[sourcecode lang=”php”]
session.save_handler=memcached
session.save_path="127.0.0.1:11211, 127.0.0.1:11212"
[/sourcecode]

Restart your web server. And …… you are done! ๐Ÿ™‚ – Now all your session data will be saved and served from these memcached servers. No matter whenever you need to extend your setup by adding extra web servers, all user data and session data will remain valid and served from a central location. No I/O issue, no huge write load on DB servers.

removing empty elements from an array, the php way :)

removing empty elements from array is most likely a very common task for everyday programming. different people work on it differently. some runs N times loop and some other tries by finding offset and then by unsetting that. let me show you some common approach of doing it and possibly the best way too :), needless to say, in a php way – most of the time such a microsecond does not matter, but well, its still good to find out the best solution ๐Ÿ™‚

first of all, lets write a function which will create a random array with empty elements
[source lang=’php’]
function generateRandomArray($count=10000)
{
$array = range(0,($count-1));
for($i = 0;$i<1000;$i++)
{
$offset = mt_rand(0,$count);
$array[$offset] = “”;
}
return $array;
}
[/source]

now lets see some different approaches to get it done.
probably most common approach
[source lang=’php’]
$array = generateRandomArray();
$len = count($array);
$start = microtime(true);
for ($i=0;$i< $len;$i++) { if(""==$array[$i]) unset($array[$i]); } $end = microtime(true); echo ($end-$start); [/source] you can see the output varies from 0.13-0.14 seconds for an array with 10000 elements in a 2.66 ghz core 2duo machine running mac osx 10.5.8 with 4GB ram. here is another better approach using array_diff() [source lang='php'] $array = generateRandomArray(); $start= microtime(true); $empty_elements = array(""); $array = array_diff($array,$empty_elements); $end = microtime(true); echo ($end-$start); [/source] this one takes 0.052-0.059 seconds and surely is a significant improvement over the last one here is another improved version using array_filter() [source lang='php'] $array = generateRandomArray(); $start= microtime(true); $array = array_filter($array); $end = microtime(true); echo ($end-$start); [/source] it takes like 0.002 seconds to complete ๐Ÿ™‚ - pretty good one, eh? (thanks damdec, for reminding about it) and this is the last one which was my favorite one using array_keys() taking advantage of the optional search_values parameter ๐Ÿ™‚ [source lang='php'] $array = generateRandomArray(); $start= microtime(true); $empty_elements = array_keys($array,""); foreach ($empty_elements as $e) unset($array[$e]); $end = microtime(true); echo ($end-$start); [/source] this is an amazing improvement over the previous solutions, it takes only around 0.0008 - 0.0009 seconds for an array of 10000 elements. i hope you enjoyed this post with micro benchmarks ๐Ÿ˜€ - happy phping

I would love to be a baby sitter

Well, it may sound funny but this is what I want to be after 6 years. I am quite planned about my future and I will definitely retire from my regular web app dev job after 6 years. Beside this I love kids, love talking to them and love to see them growing up ๐Ÿ˜€

so I am going to be a baby sitter after 6 yrs and I am sure, I will do good ๐Ÿ˜€

WorldTimeEngine – How about making your own in PHP?

I recently came by this site WorldTimeEngine where users can search the local time of any place using the name, street address or just latitude and longitude. Since that time I was thinking how easily you can make your own. As long there are some good people over there (For Geocoding API) – its a not a big deal, you know? Lets have a look at the following PHP code.


<?
$place = "Bucharest";

$geodata = getGeoCode($place);
print_r($geodata);
echo getTime($geodata['lat'],$geodata['lng']);

function getTime($lat, $lng)
{
$url = "http://ws.geonames.org/timezone?lat={$lat}&lng={$lng}";
$timedata = file_get_contents($url);
$sxml = simplexml_load_string($timedata);
return $sxml->timezone->time;
}
function getGeoCode($address)
{
$_url = 'http://api.local.yahoo.com/MapsService/V1/geocode';
$_url .= sprintf('?appid=%s&location=%s',"orchid_geocode",rawurlencode($address));
$_result = false;
if($_result = file_get_contents($_url)) {
preg_match('!<Latitude>(.*)</Latitude><Longitude>(.*)</Longitude>!U', $_result, $_match);
$lng = $_match[2];
$lat = $_match[1];
return array("lat"=>$lat,"lng"=>$lng,"address"=>$address);
}
else
return false;
}
?>

Changing the variable “$place” to “Bucharest”,”Dhaka”, “Toronto” and “Oslo” gives me the following result


Array
(
[lat] => 44.434200
[lng] => 26.102955
[address] => Bucharest
)
2008-03-01 08:41

Array
(
[lat] => 23.709801
[lng] => 90.407112
[address] => Dhaka
)
2008-03-01 12:42

Array
(
[lat] => 43.648565
[lng] => -79.385329
[address] => Toronto
)
2008-03-01 01:42

Array
(
[lat] => 59.912280
[lng] => 10.749980
[address] => Oslo
)
2008-03-01 07:43

Nice, Huh?

We are close to release Orchid, the new Framework

orchid framework

Am I talking about another framework? Well yeah. I thought it will be real fun for web application developers to use orchid to build their apps. Orchid is still in preview state and we are planning to release it by the mid of January. But if you are interested you can check the orchid blog and checkout the nightly build version from svn repository. Orchid is very fast and lightweight and really painless to kickstart the development.

URL : http://orchid.phpxperts.net

Some of the core features of Orchid which I think are really helpful to develop real world applications, are listed here

1. Caching Engine with support of Memcache, MySQL and SQLite as Storage
2. MVC
3. RoR like Layout and segregated template components.
4. Partial Caching in Template
5. Very efficient object loading and dispatching
6. DAL for MySQL (Both MySQL and MySQLi), SQLite, PostgreSQL and MSSQL
7. Support for PDO
8. Builtin Unit Testing Library where you can write unit tests inside the controller without extra pain.
9. Internationalization using Language files
10. Session Manager (Regular and DB Based)
11. Bundled Google Map Library
12. Bundled Google Chart Library
13. Bundled JSON Library
14. Bundled Prototype, jQuery, Scriptaculous and SWFObject Scripts with on-demand gzip support
15. On demand gzipping for javascripts
16. Active Record
17. Excellent Benchmarking and Profiling support which you can use to profile your application
18. CSS based Button Manager (What is this???? — checkout)
19. AJAX Library

Beside many other features it also comes with an exceptional feature which is hard to find elsewhere ๐Ÿ™‚ which is the following one

20. Bundled “Colorful Moments”

๐Ÿ™‚

Orchid is planned to release by mid januray, as I told. Currently some of us are working on documentation. But you can still check out the code and sample controllers.

Ifthese sounds interesting – you can visit the official blog of orchid at http://orchid.phpxperts.net

Performance tips for web applications

I have recently came by the article โ€œHigh performance websitesโ€ in yahoo developer network. This article pointed 13 key points to speed up your application. These are

1. Make Fewer HTTP Requests
2. Use a Content Delivery Network
3. Add an Expires Header
4. Gzip Components
5. Put CSS at the Top
6. Move Scripts to the Bottom
7. Avoid CSS Expressions
8. Make JavaScript and CSS External
9. Reduce DNS Lookups
10. Minify JavaScript
11. Avoid Redirects
12. Remove Duplicate Scripts
13. Configure ETags

I was trying to improve performance for a web application recently keeping them in mind. Let me describe how to do some of them technically.

Making fewer HTTP requests:
Each time file loads from your web server, it generates an http request. So you can reduce number of http requests by caching some contents which are almost static or changes very rarely. if these contents are loaded from browser cache, there will be no request for them over HTTP. You can use apache’s mod_expire module to cache specific tyyou (image, swf etc) of contents in your site. The following line in httpd.conf loads mod_expire successfully.

LoadModule expires_module modules/mod_expires.so

After that, write the following lines in .htaccess to cache all the image, swf and javascripts for one month from the current date, with the help of mod_expire

ExpiresActive On
ExpiresByType image/gif A2592000
ExpiresByType image/png A2592000
ExpiresByType image/jpg A2592000
ExpiresByType image/jpeg A2592000
ExpiresByType application/x-javascript A2592000
ExpiresByType application/x-Shockwave-Flash A2592000

If you are confused by the letter “A” with number of seconds associated with it, it actually means “Access time”

For reducing number of HTTP requests, you can also merge all your css files and javascript files into one css and js file.

Use a content delivery network
Yeah, that works really cool. You can serve your static contents more effectively from popular content delivery network (CDN) like akamai or Amazon S3. The benefit of using popular CDNs is that these CDNs are scaled and distributed and really knows how to serve your content faster than ever.

Add an Expiry Header
Expiry tags helps your browser to understand when to invalidate cache for a cached content. So you can ask how to add expiry header to your content. You can do it by injecting HTTP headers while delivering it from your server to end user’s machine i.e browser. Apache’s mod_expire module does the same thing, by MIME type of contents. But it doesn’t help to cache a single file or multiple files. So if you want to cache a specific file, you can do it using following PHP code.

<?php
header("Expires: ".gmdate("D, d M Y H:i:s", time()+600)." GMT");
header("Cache-Control: max-age=600");
?>

This will set the expiry time 600 seconds from the time of content delivered.

Gzip components
While delivering any data from web server to browser, you can apply gzip compression over it. These gzipped data are decompressed when received by brwsers and treated as regular data. Almost all of the modern browsers supports gzip compression. To compress your content, you can do it either automatically using Apacheโ€™s mod_deflate or manually via your code. If you like to do it using mod_deflate, then you have to enable mod_deflate first and then modify your .htaccess file to make use of that. The following line in httpd.conf enables mod_deflate with apache

LoadModule deflate_module modules/mod_deflate.so

Now if you want to make use of mod_deflate and compress your content on the fly while delivering, you can add the following line in your .htaccess file.

AddOutputFilterByType DEFLATE text/html text/plain text/xml application/x-javascript

Or you can write PHP code for delivering gzipped contents. The following piece of code delivers any javascript file as a gzipped one.

<?php
//gzipjs.php
ob_start("ob_gzhandler");
header("Content-type: text/javascript; charset: UTF-8");
header("Cache-Control: must-revalidate");
$offset = 60 * 60 * 24 * 3;
$ExpStr = "Expires: " .
gmdate("D, d M Y H:i:s",
time() + $offset) . " GMT";
header($ExpStr);
include(urldecode($_GET['js']));
?>

To deliver a javascript file (say prototype.js) using gzipjs.php you can load your scripts like this

<script type=โ€text/javascriptโ€ src=โ€gzipjs.php?js=prototype.jsโ€ ></script>

But hey, donโ€™t just include any file passed to it (as i did it here in gzipjs.php). I wrote the code quickly to demonstrate the process. In practice you must (M.U.S.T) sanitize the supplied argument before including. Otherwise it could be a very dangerous security breach.

Minify Javascripts
Minifying means compressing javascripts by removing unnecessary white space, comments and others. You can make use of customized rhino engine which is used by dojoโ€™s shrinksafe. Rhino is a nifty tool for doing these things.

So, how to do it? Download custom_rhino from dojoโ€™s shriksafe page. After that compress your javascripts using following command.

java -jar custom_rhino.jar -c infile.js > outfile.js

That means you must have JRE installed in your machine to execute the command shown above (rhino is developed using java). Now if you have number of script files and you want to compress them all at once, without applying his command for each of them, you can make a nifty batch file to do it for you. Here is the code. It will compress each script files into script.js.packed file.

for /F %%F in ('dir /b *.js') do java -jar custom_rhino.jar -c %%F > %%F.packed 2>&1

Place custom_rhino.jar file and all your script files in same directory and run it. All your scripts will be packed.

I hope these tips will really boost up performance of your web application. This is just a basic article and I didnt go details of these tips. There are also other ways (like javascript on demand) which will also help increasing the performance.

Don’t forget to check other options from the original article at yahoo developer network.

Reference
1. http://betterexplained.com/articles/speed-up-your-javascript-load-time/
2. http://www.fiftyfoureleven.com/weblog/web-development/css/the-definitive-css-gzip-method
3. http://httpd.apache.org/docs/2.0/mod/mod_expires.html
4. http://httpd.apache.org/docs/2.0/mod/mod_deflate.html
5. http://developer.yahoo.com/performance/rules.html