PHP Professionals

tech stuff, tips and the occasional rant

Drupal performance tuning (2) and a patch to the memcache module

Posted on | December 9, 2008 |

Two methods are explained below to serve some of your (anonymous) pages entirely from memcache. The second one is nicer and follows the Drupal way of handling things. The first one is more like a hack.

The hack: modify index.php
If you’re allready using the Drupal memcache module, there is a somewhat clunky way to get your Drupal frontpage (and some additional pages) entirely cached by memcached wihout any database call, you could try the following code instead of the usual index.php file (for Drupal 5, something similar is possible with Drupal 6) .

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
//get the frontpage from cache
define('MY_FRONTPAGE_CACHE',false);
define('MY_FRONTPAGE_CACHE_TIME',300);//5 min
$_paths_to_cache=array('/','/anotherurl');

if (MY_FRONTPAGE_CACHE && empty($_POST) && in_array($_SERVER['REQUEST_URI'],$_paths_to_cache)) {
    //memcache connection
    require_once('sites/default/settings.php');

    if (isset($conf['memcache_servers'])) {
        $memcache = new Memcache;
        $_cache_key = 'UNIQUE_PREFIX_'.str_replace('/','_',$_SERVER['REQUEST_URI']);
        foreach ($conf['memcache_servers'] as $mserver => $mserver_cluster) {
            $mserver = str_replace(':11211','',$mserver);
            $memcache->addServer($mserver,11211);
        }
    }  

    //get it
    $cachereturn = $memcache->get($_cache_key);
    if ($cachereturn !== false) {
        die($cachereturn.'<!-- from memcached '.$_cache_key.' -->');
    }
}

/**
* @file
* The PHP page that serves all page requests on a Drupal installation.
*
* The routines here dispatch control to the appropriate handler, which then
* prints the appropriate page.
*/

require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$return = menu_execute_active_handler();

// Menu status constants are integers; page content is a string.
if (is_int($return)) {
    switch ($return) {
        case MENU_NOT_FOUND:
            drupal_not_found();
            break;
        case MENU_ACCESS_DENIED:
            drupal_access_denied();
            break;
        case MENU_SITE_OFFLINE:
            drupal_site_offline();
            break;
    }
}
elseif (isset($return)) {
    // Print any value (including an empty string) except NULL or undefined:
    if (MY_FRONTPAGE_CACHE && in_array($_SERVER['REQUEST_URI'],$_paths_to_cache)) {
        $_resultfront = theme('page', $return);
        print $_resultfront;
        if ($cachereturn ==false) {
            $memcache->set($_cache_key,$_resultfront,1,MY_FRONTPAGE_CACHE_TIME);
        }
    } else {
        print theme('page', $return);
    }
}

drupal_page_footer();

The above code tries to connect to your memcache instance (so you have to have the memcache connection data defined in settings.php or you should change the above code if you have a separate memcache instance) and get the eventually cached data out. When finishing up, it writes your data to the cache if it wasn’t there in the first place.

Compare it with the default index.php:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
/**
* @file
* The PHP page that serves all page requests on a Drupal installation.
*
* The routines here dispatch control to the appropriate handler, which then
* prints the appropriate page.
*
*/
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$return = menu_execute_active_handler();

// Menu status constants are integers; page content is a string.
if (is_int($return)) {
    switch ($return) {
        case MENU_NOT_FOUND:
            drupal_not_found();
            break;
        case MENU_ACCESS_DENIED:
            drupal_access_denied();
            break;
        case MENU_SITE_OFFLINE:
            drupal_site_offline();
            break;
    }
}
elseif (isset($return)) {
    // Print any value (including an empty string) except NULL or undefined:
    print theme('page', $return);
}

drupal_page_footer();

You could add pages to the caching mechanism by adding them to the $_paths_to_cache array and you’ll need to make sure that all content on those specific pages is the same for loggedin and anonymous users (no login form, no user profile information).

The Drupal way: use the early page cache
As sais, this could be done in a much nicer way. You could use the early page cache, which is checked and triggered as one of the first bootstrap phases. I suggested a patch here, this is what it does:

Add this function to the memcache.module file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
if (variable_get('page_cache_fastpath', 0)) {
    function memcache_user($op, &$edit, &$account, $category) {
        switch ($op) {
            case 'login':
                // Cookie used to find out is user is logged in.
                setcookie('drupal_uid', $account->uid, time() + (60 * 60 * 24 * 30), '/');
                break;
            case 'logout':
                // Clear the cookie
                setcookie('drupal_uid', $account->uid, time() - 60, '/');
                break;
        }
    }
}

And this function to dmemcache.inc:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
/**
* Main callback from DRUPAL_BOOTSTRAP_EARLY_PAGE_CACHE phase
* This enables us to get the cached page from memcache, avoiding db calls
*/
function page_cache_fastpath() {
     global $base_root;
     if (empty($_POST) &amp;&amp; !$_COOKIE['drupal_uid']) {//anon user and no submit
         $cache = cache_get($base_root . request_uri(), 'cache_page');
         if (!empty($cache)) {
             // display cached page and exit
             drupal_page_header();
             if (function_exists('gzencode')) {
                 header('Content-Encoding: gzip');
             }
             print $cache->data;
             //you can comment the previous line and uncomment the next one to see if it works.
             //print gzencode("<!-- from early page cache -->");
            return TRUE;
       }
}
else {
        // If $_POST is not empty, the user has submit a form (ie a comment was
        // posted) so we don't serve the page from the cache, instead letting Drupal
        // process the form submission.  If the 'drupal_uid' is set, a logged in
        // user is viewing the page and so again we don't serve the page from the
        // cache.
        return;
    }
}

This will make the early page cache bootstrap phase serve pages from memcached to anonymous users. as described here.

If you use memcache.db.inc (with db fallback), you ‘ll have to add a check if the db connection is there, otherwise you’ll get a fatal error (line 7 and 18 have a change compared to the original memcache.db.inc):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function cache_get($key, $table = 'cache') {
    global $user, $active_db;

    // Garbage collection necessary when enforcing a minimum cache lifetime
    $cache_flush = variable_get('cache_flush', 0);
    // Check first if the db connection exists since in the early page cache bootstrap fase (BOOTSTRAP_EARLY_PAGE_CACHE), we have no db connection
    if ($active_db && $cache_flush &amp;&amp; ($cache_flush + variable_get('cache_lifetime', 0) <= time())) {
        // Time to flush old cache data
        db_query("DELETE FROM {". $table ."} WHERE expire != %d AND expire <= %d", CACHE_PERMANENT, $cache_flush);
        variable_set('cache_flush', 0);
    }

     // If we have a memcache hit for this, return it.
    if ($cache = dmemcache_get($key, $table)) {
        return $cache;
    }

    // Look for a database cache hit.
    // Check first if the db connection exists since in the early page cache bootstrap fase (DRUPAL_BOOTSTRAP_EARLY_PAGE_CACHE), we have no db connection
    if ($active_db && $cache = db_fetch_object(db_query("SELECT data, created, headers, expire, serialized FROM {". $table ."} WHERE cid = '%s'", $key))) {
...

And as a sidenote: you could tweak the $memcache->addServer() calls in dmemcache.inc (see manual) to not use persistent connections. If you’re using multiple frontends the default (persistence) could give you a headache.

have fun…

Category: Drupal, Snippets

Comments

2 Responses to “Drupal performance tuning (2) and a patch to the memcache module”

  1. webastien
    July 8th, 2009 @ 2:06 pm

    In function memcache_user() :
    Use $account->uid instead of $user->uid.

  2. jonas
    July 8th, 2009 @ 2:29 pm

    Thanks webastien, you’re correct. I’ll fix it in the code.

Leave a Reply