PHP Professionals

tech stuff, tips and the occasional rant

Current state of the Horde project

Posted on | December 12, 2008 | 1 Comment

A few days ago I wanted to have a look at the recent releases of IMP and Horde but the main website was down. Since it was a long time ago that I’ve been having a look at it, I was wondering if the project was shut down entirely. I worked with Horde a few years ago and I learned a lot from its code. They implemented some really nice OO stuff years before the PHP 5 release, before everyone was shouting ‘Frameworks are hot’ or ‘OO is a must’.

The wiki however was up and running, so were some of the mirrors.  More reassurance a few days later when I received some mails mentioning new releases for the core Horde projects and I also found this interview with Jan Schneider, the project lead in charge now. He talks about the current state of Horde and development going towards more AJAX support, better CalDAV and WebDAV support and continuing support for open protocols. So, Horde is somewhat under the radar compared to the other buzzword frameworks, but it’s alive. Luckily. :)

As as sidenote: the founder of Horde, Chuck Hagenbuch, also PEAR and Zend Framework contributor now apparently works for Blue State Digital, the techies behind the hugely succesfull my.barackobama.com . More on that here.

Category: PHP Frameworks
Tags:

Drupal performance tuning (2) and a patch to the memcache module

Posted on | December 9, 2008 | 2 Comments

Two methods are explained below to serve some of your (anonymous) pages entirely from memcache. The second one is nicer and follows the Drupal way of handling things. The first one is more like a hack.

The hack: modify index.php
If you’re allready using the Drupal memcache module, there is a somewhat clunky way to get your Drupal frontpage (and some additional pages) entirely cached by memcached wihout any database call, you could try the following code instead of the usual index.php file (for Drupal 5, something similar is possible with Drupal 6) .

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
//get the frontpage from cache
define('MY_FRONTPAGE_CACHE',false);
define('MY_FRONTPAGE_CACHE_TIME',300);//5 min
$_paths_to_cache=array('/','/anotherurl');

if (MY_FRONTPAGE_CACHE && empty($_POST) && in_array($_SERVER['REQUEST_URI'],$_paths_to_cache)) {
    //memcache connection
    require_once('sites/default/settings.php');

    if (isset($conf['memcache_servers'])) {
        $memcache = new Memcache;
        $_cache_key = 'UNIQUE_PREFIX_'.str_replace('/','_',$_SERVER['REQUEST_URI']);
        foreach ($conf['memcache_servers'] as $mserver => $mserver_cluster) {
            $mserver = str_replace(':11211','',$mserver);
            $memcache->addServer($mserver,11211);
        }
    }  

    //get it
    $cachereturn = $memcache->get($_cache_key);
    if ($cachereturn !== false) {
        die($cachereturn.'<!-- from memcached '.$_cache_key.' -->');
    }
}

/**
* @file
* The PHP page that serves all page requests on a Drupal installation.
*
* The routines here dispatch control to the appropriate handler, which then
* prints the appropriate page.
*/

require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$return = menu_execute_active_handler();

// Menu status constants are integers; page content is a string.
if (is_int($return)) {
    switch ($return) {
        case MENU_NOT_FOUND:
            drupal_not_found();
            break;
        case MENU_ACCESS_DENIED:
            drupal_access_denied();
            break;
        case MENU_SITE_OFFLINE:
            drupal_site_offline();
            break;
    }
}
elseif (isset($return)) {
    // Print any value (including an empty string) except NULL or undefined:
    if (MY_FRONTPAGE_CACHE && in_array($_SERVER['REQUEST_URI'],$_paths_to_cache)) {
        $_resultfront = theme('page', $return);
        print $_resultfront;
        if ($cachereturn ==false) {
            $memcache->set($_cache_key,$_resultfront,1,MY_FRONTPAGE_CACHE_TIME);
        }
    } else {
        print theme('page', $return);
    }
}

drupal_page_footer();

The above code tries to connect to your memcache instance (so you have to have the memcache connection data defined in settings.php or you should change the above code if you have a separate memcache instance) and get the eventually cached data out. When finishing up, it writes your data to the cache if it wasn’t there in the first place.

Compare it with the default index.php:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
/**
* @file
* The PHP page that serves all page requests on a Drupal installation.
*
* The routines here dispatch control to the appropriate handler, which then
* prints the appropriate page.
*
*/
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$return = menu_execute_active_handler();

// Menu status constants are integers; page content is a string.
if (is_int($return)) {
    switch ($return) {
        case MENU_NOT_FOUND:
            drupal_not_found();
            break;
        case MENU_ACCESS_DENIED:
            drupal_access_denied();
            break;
        case MENU_SITE_OFFLINE:
            drupal_site_offline();
            break;
    }
}
elseif (isset($return)) {
    // Print any value (including an empty string) except NULL or undefined:
    print theme('page', $return);
}

drupal_page_footer();

You could add pages to the caching mechanism by adding them to the $_paths_to_cache array and you’ll need to make sure that all content on those specific pages is the same for loggedin and anonymous users (no login form, no user profile information).

The Drupal way: use the early page cache
As sais, this could be done in a much nicer way. You could use the early page cache, which is checked and triggered as one of the first bootstrap phases. I suggested a patch here, this is what it does:

Add this function to the memcache.module file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
if (variable_get('page_cache_fastpath', 0)) {
    function memcache_user($op, &$edit, &$account, $category) {
        switch ($op) {
            case 'login':
                // Cookie used to find out is user is logged in.
                setcookie('drupal_uid', $account->uid, time() + (60 * 60 * 24 * 30), '/');
                break;
            case 'logout':
                // Clear the cookie
                setcookie('drupal_uid', $account->uid, time() - 60, '/');
                break;
        }
    }
}

And this function to dmemcache.inc:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
/**
* Main callback from DRUPAL_BOOTSTRAP_EARLY_PAGE_CACHE phase
* This enables us to get the cached page from memcache, avoiding db calls
*/
function page_cache_fastpath() {
     global $base_root;
     if (empty($_POST) &amp;&amp; !$_COOKIE['drupal_uid']) {//anon user and no submit
         $cache = cache_get($base_root . request_uri(), 'cache_page');
         if (!empty($cache)) {
             // display cached page and exit
             drupal_page_header();
             if (function_exists('gzencode')) {
                 header('Content-Encoding: gzip');
             }
             print $cache->data;
             //you can comment the previous line and uncomment the next one to see if it works.
             //print gzencode("<!-- from early page cache -->");
            return TRUE;
       }
}
else {
        // If $_POST is not empty, the user has submit a form (ie a comment was
        // posted) so we don't serve the page from the cache, instead letting Drupal
        // process the form submission.  If the 'drupal_uid' is set, a logged in
        // user is viewing the page and so again we don't serve the page from the
        // cache.
        return;
    }
}

This will make the early page cache bootstrap phase serve pages from memcached to anonymous users. as described here.

If you use memcache.db.inc (with db fallback), you ‘ll have to add a check if the db connection is there, otherwise you’ll get a fatal error (line 7 and 18 have a change compared to the original memcache.db.inc):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function cache_get($key, $table = 'cache') {
    global $user, $active_db;

    // Garbage collection necessary when enforcing a minimum cache lifetime
    $cache_flush = variable_get('cache_flush', 0);
    // Check first if the db connection exists since in the early page cache bootstrap fase (BOOTSTRAP_EARLY_PAGE_CACHE), we have no db connection
    if ($active_db && $cache_flush &amp;&amp; ($cache_flush + variable_get('cache_lifetime', 0) <= time())) {
        // Time to flush old cache data
        db_query("DELETE FROM {". $table ."} WHERE expire != %d AND expire <= %d", CACHE_PERMANENT, $cache_flush);
        variable_set('cache_flush', 0);
    }

     // If we have a memcache hit for this, return it.
    if ($cache = dmemcache_get($key, $table)) {
        return $cache;
    }

    // Look for a database cache hit.
    // Check first if the db connection exists since in the early page cache bootstrap fase (DRUPAL_BOOTSTRAP_EARLY_PAGE_CACHE), we have no db connection
    if ($active_db && $cache = db_fetch_object(db_query("SELECT data, created, headers, expire, serialized FROM {". $table ."} WHERE cid = '%s'", $key))) {
...

And as a sidenote: you could tweak the $memcache->addServer() calls in dmemcache.inc (see manual) to not use persistent connections. If you’re using multiple frontends the default (persistence) could give you a headache.

have fun…

Category: Drupal, Snippets

Shell tricks (1)

Posted on | December 4, 2008 | 2 Comments

Some shell tricks, most of them interesting for deploying and handling your files:

find files recently modified (say in the last 2 days, -mtime 1):

1
find . -type f -daystart -mtime 1

synchronize an entire directory to a remote server (overwrites everything, remote server has to have rsync installed too):

1
rsync -avzpog yourdir you@remotebox:/remotedir/

other way around:

1
rsync -avzpog you@remotebox:/remotedir/* yourdir/

build a listing of md5 checksums usefull for controlling the integrity of your files later:

1
find ./yourdir/ -type f -exec md5sum {} \; > checksumlisting.lst

use the generated list to check your files’ integrity:

1
md5sum -c checksumlisting.lst | grep -i failed

remove SVN directories from your tree of files:

1
find ./ -name '*svn*' -exec rm -rf {} \;

likewise for CVS directories:

1
find ./ -name '*cvs*' -exec rm -rf {} \;

if you messed up your console because you paged (more, less, head, tail) a binary file, reset your console display:

1
reset
Category: Shell
Tags: > > > >
« go back — keep looking »