Jangro.com

Website Performance Tools: How I Installed Memcached

We should all be lucky enough to have performance problems due to heavy website traffic.

“Plan for success” is one of my favorite cliches, and in that spirit, I’m trying to get/stay ahead of the curve on a website that’s growing in popularity, BUMPzee.

It not so much BUMPzee’s popularity put it in jeopardy of performance issues, it’s that it has widgets available for members to put on their own websites. Now, all of a sudden, BUMPzee’s server load is dependent hits from these widgets all over the place. And if one of those members gets hit with a Digg homepage mention, BZ’s servers get hit as well. Lately, a few high-traffic sites have put BZ widgets on their pages, and the last thing I want is to be the cause of slowdowns.

There are lots of ways to scale a web application, from throwing hardware at it, to optimizing database queries. I’ve been hearing a lot, however, about a software program called Memcached. It runs on one or more servers, and acts as a shared memory resource for any applications that may want to store information in fast-access RAM, like a long-running database query.

When to use Memcached

Memcached is great for dynamic database driven websites that retrieve data again and again from a database. Reading the same data over and over again from the database is a waste when you can store it in memory and use it from there. Even if the data gets updated fairly often, storing a cached version in memory for only 10 seconds can be highly effective if you get several pageviews during that period of time. And unless someone is saving data (at which point you’d automatically update the cache), nobody is going to notice that the data is 10 seconds delayed.

I’m just happy to have memcached in my toolbox. It will be valuable for when things get really busy and there’s a big query that gets run over and over

Installing Memcached

I began the installation process and quickly ran into some problems. After some searching, trial and error, and persistence I did get it running. The good news is that once it is installed and running, it’s easy. And, if you have more than one server, you can run memcached on many servers acting as one shared resource.

I didn’t find, however, a how-to document with all the steps outlined on getting it running. So I put together those steps here.

There are two components to a memcached implementation. The server software that runs as a daemon on your server(s). That’s a simple application that you must download and install from source code. The other component is your application that uses is. There are libraries available for most web technologies: PHP, Ruby, Perl, etc. I use PHP.

Install Libevent

Memcached requires LibEvent, so I had to install that first. If you’re not sure, you can try installing memcache first and the configure process will tell you what you’re missing.

Go to the Libevent website and download the source files: Libevent

Once the file was downloaded, I used the following commands. Note, your filenames may differ if you have a newer version.


$ tar -xvf libevent-1.3a.tar
$ cd libevent-1.3a
$ ./configure
$ make
$ make install


Install memcache

Download the latest Memcached source code: Memcached


$ gunzip memcached-1.2.1.tar.gz
$ tar -xvf memcached-1.2.1.tar
$ cd memcached-1.2.1
$ ./configure
$ make
$ make install


All should be good now. However, when I run memcached, I get an error:

$ memcached
memcached: error while loading shared libraries: libevent-1.3a.so.1: cannot open shared object file: No such file or directory

I discovered somewhere that if I run a command like this, I get details about where it’s looking for the libraries:

$ LD_DEBUG=libs memcached -v
18990: find library=libevent-1.3a.so.1 [0]; searching
18990: search cache=/etc/ld.so.cache
18990: search path=/lib/tls/i686/sse2:/lib/tls/i686:/lib/tls/sse2 [...long path snipped...]
18990: trying file=/lib/tls/i686/sse2/libevent-1.3a.so.1
18990: trying file=/lib/tls/i686/libevent-1.3a.so.1
18990: trying file=/lib/tls/sse2/libevent-1.3a.so.1
18990: trying file=/lib/tls/libevent-1.3a.so.1
18990: trying file=/lib/i686/sse2/libevent-1.3a.so.1
18990: trying file=/lib/i686/libevent-1.3a.so.1
18990: trying file=/lib/sse2/libevent-1.3a.so.1
18990: trying file=/lib/libevent-1.3a.so.1
18990: trying file=/usr/lib/tls/i686/sse2/libevent-1.3a.so.1
18990: trying file=/usr/lib/tls/i686/libevent-1.3a.so.1
18990: trying file=/usr/lib/tls/sse2/libevent-1.3a.so.1
18990: trying file=/usr/lib/tls/libevent-1.3a.so.1
18990: trying file=/usr/lib/i686/sse2/libevent-1.3a.so.1
18990: trying file=/usr/lib/i686/libevent-1.3a.so.1
18990: trying file=/usr/lib/sse2/libevent-1.3a.so.1
18990: trying file=/usr/lib/libevent-1.3a.so.1
18990:
memcached: error while loading shared libraries: libevent-1.3a.so.1: cannot open shared object file: No such file or directory

There are probably lots of ways to get the library file to be found. I just created the following symlink to put it in a spot that memcached will find it in:

$ ln -s /usr/local/lib/libevent-1.3a.so.1 /lib/libevent-1.3a.so.1

Which does the trick nicely. Once I do that, memcached will run.

That’s the end of the installation

Running Memcached

$memcached -d -u nobody -m 128 127.0.0.1 -p 11211

simple. Now memcache is running, waiting for applications to store and retrieve information.

Install the Memcache PHP Extension

The memcache PHP extension is available as a dynamic library. Typically, these can be installed using PEAR and PECL. But those didn’t work on my server:

$ pear install memcache
No releases available for package "pear.php.net/memcache" - package pecl/memcache can be installed with "pecl install memcache"
Cannot initialize 'memcache', invalid or missing package file
Package "memcache" is not valid
install failed


From what I understand, PHP5 is required for using pecl, and in any case, that didn’t work either:

$ pecl install memcache
WARNING: channel "pear.php.net" has updated its protocols, use "channel-update pear.php.net" to update
downloading memcache-2.1.0.tgz ...
Starting to download memcache-2.1.0.tgz (19,567 bytes)
.......done: 19,567 bytes
4 source files, building
running: phpize
Configuring for:
PHP Api Version: 20020918
Zend Module Api No: 20020429
Zend Extension Api No: 20050606
/usr/local/bin/phpize: /tmp/tmpFqYI1O/memcache-2.1.0/build/shtool: /bin/sh: bad interpreter: Permission denied
Cannot find autoconf. Please check your autoconf installation and the $PHP_AUTOCONF
environment variable is set correctly and then rerun this script.

ERROR: `phpize' failed


I spent a good amount of time trying to get these things running. Ultimately, I think that may be because my server has noexec on the /tmp directory for security purposes (which I highly recommend!). Anyway, instead of trying to figure that out, I installed it manually:

Manual Install of the memcache PHP extension


$ wget http://pecl.php.net/get/memcache-2.1.0.tgz
$ gunzip memcache-2.1.0.tgz
$ tar -xvf memcache-2.1.0.tar
$ cd memcache-2.1.0
$ phpize
$ ./configure
$ make
$ make install


Note the installation location on the make install command:

$ make install
Installing shared extensions: /usr/local/lib/php/extensions/no-debug-non-zts-20020429/


In order to get this extension to load, you need to add the command to the php.ini file

Since the above directory seemed odd to me and I had to configure PHP anyway, I moved the extension up one level. This requires an edit to php.ini. This was ok since I don’t have any extensions loading from anywhere else. You should confirm the same for your setup.

Here’s the change I made to php.ini:

; Directory in which the loadable extensions (modules) reside.
# extension_dir = "./"
extension_dir = "/usr/local/lib/php/extensions";


And I copied the memcache.so into that dir:

$ cp /usr/local/lib/php/extensions/no-debug-non-zts-20020429/memcache.so /usr/local/lib/php/extensions/


Once that’s there, we can add the dynamic extension to the php.ini file. There’s a section for extensions where I put this.

extension=memcache.so


and restart the web server.

Test PHP script

Once we’ve got things running, we need a test. This is a simple script that stores some data in the cache and retrieves it.

< ?php

?>

And here’s the output of the working script:

Server's version: 1.2.1
Store data in the cache (data will expire in 10 seconds)
Data from the cache:
object(stdClass)(2) { ["str_attr"]=> string(4) “test” ["int_attr"]=> int(123) }


Do Something Useful

Now it’s up to you to figure out where the heavy lifting occurs on your web applications and lighten the load by storing stuff in memory.

Do it now before you need it. Good luck!

Additional Reading

Distrubuted Caching with Memcached
Use Server-Side Caching When Possible (memcached)
If You’re Not Using memcached You’re Doing It Wrong

If you enjoyed this post, please subscribe to my RSS feed