Unlike some other programming languages, PHP shuts down completely the moment it stops processing a specific request.

Different programming languages have very different lifespans. Your computer’s operating system, for example, runs continuously—executing tasks, consuming memory, and so on—as long as your computer’s on. Imagine if your computer was designed to shut off and boot back up in between each keypress on your keyboard!

As insane as that sounds (and would be for an operating system), it’s pretty much how PHP works. PHP was designed specifically to be run on web servers, handling one-off requests from numerous clients. Since a server talks to many (even millions of) clients at once, it’s best to free up all that client’s resources the moment its request is complete. As a result, PHP shuts down completely the moment it stops processing a specific request.

So each new request made to a PHP codebase requires PHP to rebuild itself from scratch—which is why, for example, your WordPress site builds itself, from init on up, each and every time someone wants to view a new post on your blog. It’s a bit like a hotel rebuilding itself every time there’s a visitor, but it turns out to be the best way to serve web traffic.

PHP Can’t Run Forever

PHP turns out to be quite bad at handling long-running processes.

Since a PHP process is either running or nonexistent, PHP turns out to be quite bad at handling long-running processes. If you want to use PHP to process through a database with a trillion rows, or to compress and export a 100GB video file, or to crack the human genome, you’re going to have problems. PHP simply wasn’t designed to live that long: it’s supposed to be either serving lightning-quick responses to a requesting client, or not existing at all. It doesn’t “run in the background.”

If you do write a script that’s more than a few seconds to complete, you’ll likely start to hit the following issues:

Timeout Errors

Servers set limits on the amount of time a single PHP process can take.

Servers set limits on the amount of time a single PHP process can take. This is good: If a rogue PHP process wants to keep calling itself over and over for, say, five years straight, that’s a lot of weight on the server. Better to cut the process sooner—after, say, 60 or 120 seconds.

So a PHP timeout error means that a PHP operation you’ve requested was taking too long to complete and was shut down.

Memory Limit Errors

Servers also limit the amount of server resources that PHP processes can take up.

Servers also limit the amount of server resources that PHP processes can take up. For example, a PHP process that attempts to export a database with a trillion rows could simply take all the server’s memory, as the server tries to store details about each row during its processing. Running the server out of memory this way would cause everything else it was doing to grind to a halt.

So servers have memory limits—128MB, 256MB, or another number—that restrict PHP’s ability to store a ton of information all at once.

You can work with both of these errors to some extent by adjusting your server settings. David did a fabulous job covering this topic in his article on php.ini and phpinfo(). But this only goes so far; in particular, many shared hosting plans don’t include control over your server configuration.

Sometimes you Do Have to Do Big Stuff in PHP

You can use Ajax to block a single, overlarge PHP process into a number of smaller individual processes, which then run one by one.

The world of the web is full of big, time-consuming processes, and PHP needs to touch some of them. Migrating large databases is an obvious one, and so is, say, bulk-importing hundreds of large images into your media library, which might be a perfectly reasonable client request. If you’re in WordPress, PHP is the main game in town in terms of how to create new functionality. What to do?

Today, we’re introducing one solution: Use Ajax to block a single, overlarge PHP process into a number of smaller individual processes, which then run one by one.

The Code: Using Ajax to Route Around Memory and Timeout Limits

In the example below, we can use PHP to transfer infinitely many large files into a WordPress install, without worrying about timeout and memory limits.

Sort-Of-Advanced Code Warning

The code below relies on knowledge of PHP, JavaScript, jQuery, Ajax in WordPress, WordPress’s hooks system, and wp_localize_script(). So it’s a bit on the advanced side. If you need to brush up on any of those topics, click the appropriate link, search the site for that topic, or contact us in this article’s comments and we’d be happy to help!

What the Working Code Does

The final code takes a number of large files from another site and transfers them to the root of WPShout’s WordPress installation. As it does so, it logs to the browser’s console each file that it writes, along with the filesize:

This solution could be useful if, for example, you have a client with 1,000 large images (consistently named, e.g., img001.jpg) that need to be exported from a soon-to-expire hosting account, and the client can’t find his or her FTP credentials. This way is quite a bit quicker than FTP, too, even if you do have the credentials!

General Approach and Intuition

JavaScript can run continuously even if PHP cannot, so we can write an ongoing JavaScript that makes a series of individual requests to PHP using Ajax.

The intuition here is that JavaScript can run continuously even if PHP cannot. We can use that knowledge to write an ongoing JavaScript that makes a series of individual requests to PHP using Ajax. Even if the full transfer process takes 72 hours, no individual PHP process ever runs longer than a few seconds.

Since this is a “run yourself until everything’s finished” process, we’re going to be making use of recursion: our Ajax function calls itself again, as you’ll see in the code below.

The code is wrapped into a plugin with two files:

1. wpshout-ajax-file-copier.php, which enqueues the JavaScript and provides the Ajax endpoint that does the file processing
2. wpshout-ajax-file-copier.js, which makes the recursive Ajax request and logs responses to the browser console

The full plugin is available for download here: wpshout-ajax-file-copier.zip

The PHP Code: wpshout-ajax-file-copier.php

add_action( 'wp_enqueue_scripts', 'wpshout_ajax_copier_script' );
function wpshout_ajax_copier_script() {
	if( ! is_single( 'Above and Beyond: Avoiding PHP Timeout and Memory Limit Errors with AJAX' ) ) {
		return;
	}

	wp_enqueue_script(
		'wpshout-ajax-file-copier',
		plugin_dir_url( __FILE__ ) . 'wpshout-ajax-file-copier.js',
		array( 'jquery' )
	);

	$target_files = array(
		'ebook.mobi', 'podcast.mp3', 'video.mp4'
	);

	wp_localize_script(
		'wpshout-ajax-file-copier',
		'ajax_vars',
		array(
			'ajax_url' => admin_url( 'admin-ajax.php' ),
			'target_files' => $target_files
		)
	);
}

add_action( 'wp_ajax_wpshout_copy_file_over', 'wpshout_php_file_copier' );
add_action( 'wp_ajax_nopriv_wpshout_copy_file_over', 'wpshout_php_file_copier' );
function wpshout_php_file_copier() {
	$file = $_POST['file'];
	$base = 'https://pressupinc.com/';
	$filestring = file_get_contents( $base . $file );
	$bytes = file_put_contents( get_home_path() . $file, $filestring );

	if( $bytes === false ) {
		echo '-1';
	} else {
		echo $bytes;
	}

	die;
}

Comments

The code above does two things. wpshout_ajax_copier_script() enqueues our JavaScript file, wpshout-ajax-file-copier.js. We’re also using wp_localize_script() to pass a number of key variables to the JS file we create, including an array of the filenames we want copied.

Second, we’ve got wpshout_php_file_copier(), which handles the Ajax request. It uses file_get_contents() to retrieve (as a string) the filename it’s given through $_POST, then uses file_put_contents() to write that string out into a new file, with the same filename, at the home directory of the current WordPress install. It then echos back either a failure code, or the number of bytes in the copied file.

As a note, the is_single() check at the top means that this code only runs on a specific post on the site, since we don’t want to be copying 200 MB files everytime there’s a site visitor! (Before you ask, the plugin’s deactivated; you’re not melting our server.) To really use this check, you’d create a page that no one else would be likely to find (“My Ajax Copier Page”) and set an is_page() check for that—or, even better, do something less lazy/hacky to make sure the code only runs when you push a button somewhere in the WordPress admin.

The JavaScript Code: wpshout-ajax-file-copier.js

jQuery( document ).ready(function($) {

	var i = 0;
	function processFile() {

		if( (typeof ajax_vars.target_files[i] == 'undefined') ) {
			return;
		}

		$.post(
			ajax_vars.ajax_url,
			{
				'action': 'wpshout_copy_file_over',
				'file': ajax_vars.target_files[i]
			}, 
			function( response ) {
				if( response === '-1' ) {
					console.log( 'Failed to write file ' + ajax_vars.target_files[i] + '.' );
				} else {
					console.log( 'Wrote file ' + ajax_vars.target_files[i] + ', ' + response + ' bytes in total.' );
				}

				i++;
				processFile();
			}
		);

	}

	processFile();
});

Comments

For each file it’s given through wp_localize_script(), this JavaScript file creates an Ajax request to wp_ajax_wpshout_copy_file_over(), passing along the filename as the file property of $_POST.

Next, it waits for a success response—something back from the PHP process it called on. If that’s a "-1" (our private code for “This failed”), it says so in the browser console. Otherwise, it writes a success message to the browser console.

We use recursion to, in effect, foreach through the target_files[] array, creating a new Ajax request each time.

Now a crucial part: the recursion. i++ moves us to the next element in the target_files[] array, and we call processFile() again, from within the function itself.

This has the effect of foreaching through the target_files[] array, creating a new Ajax request each time.

The typeof check at the top is responsible for stopping this process so it doesn’t result in an infinite loop. If the requested array element doesn’t exist (that is, at the end of the array, or if there’s no array to begin with), the process exits with return.

PHP: Doing Long Jobs that are Really Lots of Short Jobs

The solution above routes around PHP’s limitations by breaking a single large task into a number of smaller, discrete requests, using an actually long-running language, JavaScript, to keep the whole process humming, and Ajax to connect the two.

I hope this has your mind working on your own giant-processing-job-for-PHP problems, and thanks for reading! I’d love to hear thoughts or questions in the comments below.

Image credit: Helgi Halldórsson