Background jobs with php and resque: part 6, integration into CakePHP

Using background jobs inside a php framework with php-resque is a little bit different, as the framework is imposing its own convention. Let’s see how to create background jobs in CakePHP, with the CakeResque plugin.

CakeResque is a CakePHP plugin for creating background jobs that can be processed offline later.

CakeResque is more than a wrapper to use php-resque within CakePHP. Where it really shines is the way it manages the dirty jobs of creating and stopping workers via the cake console.

Requirements

• CakePHP 2.1+
• Php 5.3+
• Redis 2.2+
• PhpRedis (optional)

Installation

Download the plugin

There a two methods to download the plugin:

By cloning the git repository

Navigate into your CakePHP Plugin directory

cd app/Plugin

Clone the CakeResque repository

git clone git://github.com/kamisama/Cake-Resque.git CakeResque

By downloading the archive

Download the latest release, and uncompress it in app/Plugin. Make sure the plugin directory is named CakeResque.

Install dependencies

This plugin requires the php-resque library, that is not shipped anymore by default. Composer is used to manage dependencies. First, let’s navigate into the CakeResque plugin folder

cd app/Plugin/CakeResque # or just cd ./CakeResque if already in the Plugin folder

Download Composer

curl -s https://getcomposer.org/installer | php

Install dependencies

php composer.phar install

Your final CakeResque folder should look like this

Load the plugin into CakePHP

In your application bootstrap ( app/Config/bootstrap.php), load the plugin:

CakePlugin::load(array( # or CakePlugin::loadAll(array(
    'CakeResque' => array('bootstrap' => true)
));

Configuration

The plugin comes with a bootstrap.php file, located inside the app/Plugin/CakeResque/Config directory. It contains all the default values necessary to connect to Redis and to create a worker. Refer to inline help for usage.

Managing workers

Workers managing is done via the Cake console. It’s very neat and easy, you don’t need to know the workers process ID, etc.. to manipulate them anymore, all is hidden via an API.

In summary, to

Start a worker

cake CakeResque.CakeResque start

Neat, isn’t it ? No need to run php resque.php with all the permission, piping and daemon stuff anymore. You can also pass various options such as the queue names, interval etc … like

cake CakeResque.CakeResque start -q default -i 15

See the bootstrap file for default values. Refer to documentation for all available options.

Stop a worker

cake CakeResque.CakeResque stop

You don’t need to know the worker PID anymore to kill the worker’s process. Instead, you’ll be presented with a list of active workers, just choose the one you want to stop, or pass -all flag to stop them all.

Worker Stats

cake CakeResque.CakeResque stats

Display a summary of all active workers, with their respective number of processed and failed jobs.

Tail Log file

cake CakeResque.CakeResque tail

To tail a log file, chosen from a list of log files currently used by your workers.

Enqueue a job

cake CakeResque.CakeResque enqueue queuenName ClassName MethodName args1 args2

Pause, Resume and CleanUp worker

cake CakeResque.CakeResque pause
cake CakeResque.CakeResque resume
cake CakeResque.CakeResque cleanup

Each command will present you with a list of workers to choose from. Use the --all flag to pause, resume or cleanup all workers.

The following 2 commands are the icing on the cake:

Restart workers

cake CakeResque.CakeResque restart

Restart all your workers. Each worker options (queue, interval, etc…) are preserved.

Load a set of predefined workers

cake CakeResque.CakeResque load

Load one or more workers defined in the bootstrap. Very useful when you have more than one worker, or when your workers take a lot of additional options. Let’s say you always need 4 workers, started with the following commands

cake CakeResque.CakeResque start -q notification
cake CakeResque.CakeResque start -q default -i 10
cake CakeResque.CakeResque start -q recommendation -i 60
cake CakeResque.CakeResque start -q cache -u www-data

To automatically load all these queues with the load command, edit the Queues section in the plugin bootstrap file

'Queues' => array(
    array           (
        'queue' => 'notification',
    ),
    array(
        'queue' => 'default',
        'interval' => 10
    ),
    array(
        'queue' => 'recommendation',
        'interval' => 60
    ),
    array(
        'queue' => 'cache',
        'user' => 'www-data'
    )
)

Queuing jobs

Queuing a job is done exactly the same way as the Group job by family method, beside the fact that you can use only string inside the third argument. It can’t be an associative array, nor can contain array.

CakeResque::enqueue('QUEUENAME', 'CLASSNAME', array('METHODNAME', 'otherArg', 'foo'));

• QUEUENAME is the queue name to queue to job in
• CLASSNAME is the job’s classname
• METHODNAME is the class’ method name to run when performing the job
• Other values after METHODNAME are optional, and depend on your job. Can only be string.

Jobs classes

Instead of creating new job classes just for Resque exclusive usage, we can write the job classes as shell classes, and re-use them elsewhere. That way, you’ll be able to execute a job manually, without using Resque.

CakePHP shell classes are located inside app/Console/Command folder. A shell class file is named ClassnameShell.php, e.g: NotificationShell.php. You’ll put all your notification related jobs inside like:

# app/Console/Command/NotificationShell.php
class NotificationShell extends AppShell
{
    public function sendToUser() {
        # Do your stuff
    }

    public function sendToAdmin() {
        # Do your stuff
    }
}

To link this Notification class example with the CakeResque::enqueue() function in the previous section:

• CLASSNAME will be NotificationShell
• METHODNAME is either sendToUser or sendToAdmin
• 'otherArg', 'foo' are available inside the Notification class, via $this->args.

$this->args = array(
    0 => 'otherArg',
    1 => 'foo'
);

Finally, to make these Shell classes usable by the workers, add that perform() method to your AppShell:

public function perform()
{
    $this->initialize();
    $this->{array_shift($this->args)}();
}

If this file doesn’t exist, create it

# app/Console/Command/AppShell.php
<?php
App::uses('AppModel', 'Model');
class AppShell extends Shell
{
    public function perform() {
        $this->initialize();
        $this->{array_shift($this->args)}();
    }
}

Since job class equals shell class, you can use them in your cake console too:

cake Notification sendToAdmin otherArg foo

Using plugin shell classes

Plugin naming scheme is also supported.

CakeResque::enqueue('default', 'PluginName.MyClass', array('methodName'));

Monitor with DebugKit

All jobs queuing activities are logged, and are available inside the DebugKitEx Resque panel.

On top of the DebugKit plugin, also install the DebugKitEx plugin and activate the Resque panel:

var $components = array('DebugKit.Toolbar' => array(
     'panels' => array('DebugKitEx.Resque')
));

New panel will contain

Next Time …

The main feature of CakeResque is its cli to manage workers. An equivalent tool not tied to any framework also exists.

In part 7, we’ll explore Fresque, a command line interface for managing php-resque workers.