Background jobs with php and resque: part 1, introduction

Background jobs are jobs that are executed outside the main flow of your program, and usually handled by a queue system.

This first tutorial of the serie will introduce what’s a background job and its importance.

What’s a background job

Let’s start with a real-world example : a social network website, where a user lambda will update his profile with a new country.

The classic workflow

User location update workflow

As PHP is not multi-threaded, all tasks will be executed one after each other, resulting in a pretty long waiting time for the user. After 3.7 seconds, user will finally get the response to his update location action. User is unhappy :(

Note that the only important step in this workflow is the first one : Update Database.

If this fail, all the other tasks will not be executed. If this pass, it’s the only “message” send back to the user, because in the end, we just send back the page saying : “Your location was updated with success” in case of success, and “An error occurred while updating your location” otherwise.

The other steps (refresh cache, send mails, etc …) are not needed to print the final page. So why should the user need to wait for them to execute before getting back a response ? Just do what the user has asked you to do, answer him immediately, and report the other unimportant steps to later.

The ideal workflow

User location update workflow with background jobs

This will make the user happy, as he just waited 0.3 second, the time taken to do what he asked. All the other tasks will be executed as background jobs, outside the main workflow.

A little analogy

If you still didn’t understand what just happens :

  • User Lambda has asked Mr. Server to update his location
  • Mr. PHP#1, who is working for Mr. Server, picked up the order
  • Mr. PHP#1 update the location in the database
  • Mr. PHP#1 write down in a post-it : “I just updated user lambda’s location, please refresh cache, send mails, notification and recommend new friends”, and pin it to the todo list board.
  • Mr. PHP#1 give the update status to Lambda : “Yeah, it’s done sir !”
  • Mr. PHP#1 is now free, and ready to pick up another order.

But, what happens to the todo list ?

  • Mr. PHP#2, another guy working for Server, will come to the todo list board each X seconds to pick up all the post-it
  • He then will return to his desk, and execute the tasks described in the post-it, from older to newest.
  • After executing all the tasks, he will wait X seconds, then come back to the post-it board to do the same thing again, and again, and again.

In the diagram above, PHP#1 is on the left, and PHP#2 on the right. They are not in communication.

Let’s call them by their name

In the right jargon, PHP#2 is called a worker, and the post-it board, a queue. The tasks (in the post-it) are called jobs.

Anyone (PHP#1, etc … or even another worker) can push new jobs in the queue.

A worker will poll the queue at a defined interval, and execute the jobs in the order they arrived.

These jobs are the background jobs. The “foreground” jobs are the tasks executed by PHP#1. This latter doesn’t know how, by who, nor when the background jobs will be executed.

Once in the queue, only the workers have control about the jobs. PHP#1 just assume they will be executed later.

Worker and Queue are not limited to one. Many worker can poll the same queue, or a worker can poll many queues. Polling time can also be customized : each 5 seconds, 15 seconds, 1 minute, etc …

A Queue system is used to manage all the jobs (save them, sort them by the order they arrived, etc …)

The importance of background jobs

Background jobs are in our everyday life.

Let’s say, you purchase a web-hosting by phone

  • The guy on the phone ask for your credit card informations
  • He confirm these infos with the bank, and the bank approved the transaction
  • He create your account, and give you your credential
  • He asks you to wait 15 minutes for the server to be online
  • He wish you a good day and hang up

Benefits

If there was no background jobs, you will have to stay on the phone for days, because that’s the time needed for the bank to send money to the web-hosting company. A background job is used here to tell the bank to send the money, the guy on the phone just verified that you card is valid. At this time, the hosting company just assume they will be payed, and will create your account.

On top of that, you will have to wait an additional 15 minutes, because the guy on the phone and you will have to wait for the technician to put your hosting online. The background job just spare not just yours, but also the guy’s time !

Looking back at the programing side, background jobs are executed in another process. It means that whatever happens in these jobs will not affect the main workflow, like exception or fatal errors.

Drawbacks

Waiting time is greatly reduced with background jobs, but you lose control on them. Since they’re just pushed in a queue, only god knows what happens with them. Do you have any workers at all ? Are they slacking off ? Are they polling the right queue ?

It’s your duty to make sure that you are pushing new jobs to a queue polled by a worker. Among the common mistakes are :

  • Pushing in the queue name “achievement”, and have a worker polling the queue “achivement”.
  • Restart your server, and forget to create your workers again
  • Not monitoring your workers properly. A Fatal error stop the process, and takes the worker with him.

Let’s take the web-hosting example again. The guy on the phone promises you that your hosting will be online in 15 minutes. How did he know ? Because in the manual, it says so. Neither he nor you are guaranteed that it’ll really be online in 15 minutes.

The worker (the technician installing your hosting) is in its own bubble, and does not communicate with the external world. What if he is overloaded with a lot of jobs, and yours is in the bottom of the list ?

To keep a progress status on jobs, you have to “instruct” you workers to report a lot of various things : “When you hit that problem, send a mail/notification to admin X”, etc … You don’t and can’t ask them anything, they have to tell you on their own.

There’s still the Fatal error issue that can’t be reported, since it’s a … fatal error. Only way to find them is to monitor the server’s php error log. They’re particularly important, since they take down the worker. Risks are you end up with no more workers at all.

Don’t abuse of background jobs

Workers execute the jobs from zero. If needed, they will have to connect to the database, fetch and recompute some datas to work with, etc … steps already done in the main workflow. When creating jobs, keep in mind that they start in another process, from nothing.

Next time …

Part 2 of the tutorial will talk about the queue system.