Drupal 7 is an old release, which due to go end-of-life in January 2025, and which many of us are still using in production. I had set up Docker containers running PHP 7.1 and MySQL 5.7, back when those versions were what's required to host Drupal 7 sites. Since those PHP and MySQL versions are old, the Drupal team has since updated Drupal 7 to run on PHP 8.2 and MySQL 8. Before Drupal 7 goes end-of-life, I want to build a modern stack for hosting the old Drupal release required by my site.
Nearly 20 years ago I was excited by Drupal, and used it to build several websites. I've since lost my enthusiasm for Drupal, and most of those sites have been deleted. I was especially excited by the tagline that Drupal was Community Plumbing, meaning that one could build "Community" websites with Drupal. The Drupal team has since dropped that tagline, and is chasing golden rainbows of enterprise scale deployment.
My remaining Drupal website, visforvoltage.org, is a discussion forum, a.k.a. online community, for world-wide discussion of electric vehicle technology, especially focusing on the DIY side of the electric vehicle market. This site is in use by its community, and the site contains lots of useful information.
It was launched in November 2006, using an early release of Drupal 6, then upgraded to Drupal 7 in perhaps 2014. A big issue with keeping the site running is that I believed development on Drupal 7 was frozen, and therefore it required an old PHP release (7.1), and an old MySQL release (5.7). For that purpose I concocted a Docker container
robogeek/drupal-php-7.1 containing PHP 7.1 and Apache along with some useful tools for running Drupal. That includes a Drush install.
To host the website, I setup a VPS containing Docker with visforvoltage.org deployed using that container, and using mysql/mysql-server:5.7 for the database.
It's proof that with Docker it's possible to host services using really old software. The images for both PHP 7.1 and MySQL 5.7 are still there, despite being very old and unsupported.
It turns out that the
Drupal 7 maintainers have updated the code to allow us to use MySQL 8.x and PHP 8.2. That piqued my interest because both are supposed to be more secure and higher performance.
The task we're solving in this blog post is updating an old Drupal 7 site to run on the latest MySQL and PHP while still being Drupal 7. The result should be lowering the server load for hosting the site, while increasing reliability and security, and opening a path to applying patches as the Drupal team makes updates. Also, it should be possible to maintain the site as it stands for longer into the future.
Project overview
There are four stages in this project.
- Rehost your Drupal 7 site on PHP 8.2 hosting, preferably in Docker. This involves updating the core drupal to v7.101 (as of this writing), plus testing all contrib modules to make sure they work on PHP 8.2.
- Set up caching, such as FileCache or Memcache.
- Redo the database for 4-byte UTF characters.
- Convert the database to MySQL 8.x or MariaDB 11.x.
Evaluating the Drush and Drupal containers
The first task is determining what Docker container to simplify hosting a Drupal site.
I use drush make as the core step for building the code used by visforvoltage.org. Therefore, I need to use Drush. The simplest route to installing Drush is to use the container:
https://hub.docker.com/r/drush/drush
According to the drush.org compatibility chart, we use Drush 8.x with Drupal 7.x. Drush 8.x is available but unsupported, for example the
Drush 8.x documents are kept as a hold-over from earlier days.
There is also a Drupal container:
https://hub.docker.com/_/drupal
With that container we could put the following into a Compose file:
services:
drupal:
image: drupal:7-apache
ports:
- 8080:80
volumes:
- $PWD/roots/YOUR-DOMAIN/modules:/var/www/html/modules
- $PWD/roots/YOUR-DOMAIN/profiles:/var/www/html/profiles
- $PWD/roots/visforvoltage.org/themes:/var/www/html/themes
# this takes advantage of the feature in Docker that a new anonymous
# volume (which is what we're creating here) will be initialized with the
# existing content of the image at the same location
- $PWD/roots/YOUR-DOMAIN/sites:/var/www/html/sites
restart: always
That might work. What will happen is the Drupal container downloads Drupal v7 code, and then we use volume mounts to inject our code customizations into the code downloaded by the container.
I don't like that approach. Using Drush Make handles downloading Drupal 7 Core, so why should I use this Drupal container to do that? Instead I will modify
robogeek/drupal-php-7.1 to use PHP 8.2.
Building a Drupal 7 site using Drush Make, in the Docker generation
I have a shell script which is used for cloning/rebuilding my existing site. To update the site, I run the script which downloads new copies of the code and fixes things so it can run. It runs Drush Make to download pristine copies of the core Drupal software, plus pristine copies of the contributed modules. One rationale for this approach is to ensure you always have clean code to run your Drupal site.
The clone script then patches up the resulting tree by copying files over from the site being cloned.
The new version of the script is:
#!/bin/sh
NOW=roots/visforvoltage.org
BUILD=roots/dev.visforvoltage.org
# Modified to use Drush from Docker
docker run --rm --user 33 -v $(pwd):/app drush/drush make \
/app/vvd7.make.yaml /app/${BUILD}
if [ ! -d ${BUILD} ]; then
echo FAILED TO CREATE ${BUILD}
exit 1
fi
mkdir -p ${BUILD}/sites/default
(cd ${NOW}/sites/default; tar cf - files) \
| (cd ${BUILD}/sites/default; tar xfp -)
cp ${NOW}/sites/default/settings.php \
${BUILD}/sites/default
cp ${NOW}/sites/all/modules/nice_menus/nice_menus_v.css \
${BUILD}/sites/all/modules/nice_menus
cp visforvoltage.extra/views_import.drush.inc \
${BUILD}/sites/all/modules/views_import
cp ${NOW}/.htaccess ${BUILD}/.htaccess
( cd ${BUILD};
ln -s sites/default/files/ files
mv sites/all/libraries/bad_behavior sites/all/libraries/bad-behavior
mkdir -p sites/all/imports/views
)
The NOW variable names the directory containing the current running site, and the BUILD variable names the directory into which the updated copy of the site is installed. This gives me a staging site with which to test whether the code updates were correctly made.
The Drush Make step is run with --user 33 because the files must end up owned by the www-data user ID.
The bottom portion of the script is where files are copied from the existing site into the cloned site.
The file vvd7.make.yaml starts like so:
core: '7.x'
api: '2'
projects:
drupal:
version: ~
##### Access Control
acl:
version: ~
forum_access:
version: ~
chain_menu_access:
version: ~
# ...
The actual file lists the 50 modules or so used by the site, some of which have explicit version references.
The result from this script is a directory hierarchy containing what should be a running Drupal instance. It has copied over certain files like sites/default/settings.php and the files hierarchy and should be ready to go.
Updating a Docker container for running Drupal 7 under Docker
The Docker container I currently use for hosting visforvoltage.org is perfectly capable of hosting it under PHP 8.2, after making a few changes. Refer to the GitHub repository:
https://github.com/robogeek/docker-drupal-php/
The Dockerfile is parameterized for the PHP version to use. It simply needs to be rebuilt with PHP 8.2, and a few additional small changes were required.
The build command becomes:
docker --context default build -t drupal-php-8.2:latest \
-t drupal-php-8.2:3 \
--build-arg PHP_VERSION=8.2 .
The Dockerfile starts this way:
ARG PHP_VERSION=""
FROM php:${PHP_VERSION:+${PHP_VERSION}-}apache
Hence, the base container is php:8.2-apache.
The container also includes a lot of PHP extensions required for Drupal, the correct Drush version for Drupal 7, and a script apache2-reload that can be used to tell Apache to reload the configuration files.
This is configured to support multiple sites from one container. One simply creates a sites-enabled directory containing Apache configurations, and Apache will automatically setup multiple sites.
Launching Drupal 7 using Compose with PHP 8.2
Using the image built in the previous step we can launch the Drupal website using a Compose file.
services:
drupal-visforvoltage:
image: "robogeek/drupal-php-7.1"
container_name: drupal-YOUR-DOMAIN
networks:
- servernet
- dbnet
# ports:
# - "80:80"
restart: always
volumes:
- /opt/YOUR-DOMAIN/logs:/var/log/apache2:rw
- /opt/YOUR-DOMAIN/roots:/var/www:rw
- /opt/YOUR-DOMAIN/sites-enabled:/etc/apache2/sites-enabled:rw
- /opt/YOUR-DOMAIN:/server:rw
drupal-vv82:
image: "drupal-php-8.2"
container_name: drupal-vv82
networks:
- servernet
- dbnet
# ports:
# - "80:80"
restart: always
volumes:
- /opt/YOUR-DOMAIN/logs82:/var/log/apache2:rw
- /opt/YOUR-DOMAIN/roots:/var/www:rw
- /opt/YOUR-DOMAIN/sites-enabled82:/etc/apache2/sites-enabled:rw
- /opt/YOUR-DOMAIN:/server:rw
The first service is the production site currently running PHP 7.1. The second uses the newly built PHP 8.2 container.
The /opt/YOUR-DOMAIN/roots directory is the parent directory containing docroot directories for each site to be hosted by this container. The .../roots directory contains both YOUR-DOMAIN and dev.YOUR-DOMAIN, the latter being the test site containing the latest code built around Drupal 1.101 and PHP 8.2.
The servernet network connects to the NGINX Proxy Manager service which handles Lets Encrypt SSL certificates and other usefulnesses.
The dbnet network connects to the MySQL server. The database is not managed in this Compose file because my opinion is that you start a database and leave it running long-term rather than bringing it up and down every time you modify the application code. For that purpose, there is a separate Compose file in another directory. That MySQL instance is handling multiple services including NextCloud and BookStack.
Launching the new Drupal 7 site alongside the existing site
Notice that both of the Compose file services use the same roots directory but different sites-enabled directories.
The VirtualHost defined in the second is as so:
<VirtualHost *:80>
ServerName dev.YOUR-DOMAIN
ServerAdmin ...@YOUR-DOMAIN
DocumentRoot /var/www/dev.YOUR-DOMAIN
ErrorLog /var/log/v-error.log
CustomLog /var/log/v-access.log combined
<Directory "/var/www/dev.YOUR-DOMAIN">
AllowOverride All
</Directory>
</VirtualHost>
The DocumentRoot is .../roots/dev.visforvoltage.org, in this case.
This means we can start both by running docker compose up -d.
Once it's launched you must inspect the Docker system to ensure both services are running. Once you know they're running, head to the NGINX Proxy Manager dashboard and setup a new proxy host. If you're using another system like Traefik or Caddy, do whatever is required.
The result should mean you can connect to https://dev.example.com in your browser to see if the site happens to work.
Finding a broken Drupal 7 site running on PHP 8.2
In my case, I was happy to see my site immediately come up. But there were a long list of issues.
In dev.visforvoltage.org/sites/default/settings.php I did have to make this change:
$base_url = 'https://dev.visforvoltage.org'; // NO trailing slash!
This way Drupal knows the actual URL it is supposed to be using. Everything else in settings.php was left alone.
If you carefully read the compatibility writeup for Drupal 7.101 on PHP 8.2, you see the Drupal team guarantees PHP 8.2 compatibility for Core Drupal. The team cannot guarantee compatibility for the contributed modules.
In practical terms, many of the contributed modules used on visforvoltage.org have not been updated for many years. In one case the last update was in 2011! There is no way their code is compatible with PHP 8.2. Indeed, while the site comes up, there are many warnings and errors.
One positive thing I can say about Drupal is that every error is captured, and all errors are displayed in the browser in an easy-to-read summary. The error messages include a file name and line number reference, making it easy (in most cases) to find the offending line of code.
The task now is to start fixing errors and warnings.
You open the offending file in a text editor, reason your way to a solution for that error, change the code, then reload the browser. Sometimes you use the error message text to find discussion about the problem, as well as potential fixes. If you're good, and the PHP coding gods are smiling on you, the error message will go away.
The following sections describe some of the errors which came up. There were several more that I didn't record.
It's necessary to exercise as many areas of your Drupal site as possible. Some errors will happen only on certain pages.
It's necessary to keep track of changes. You need to be able to replicate those changes in the future if/when you ever update the site again. Recall that the clone script I showed earlier downloads a pristine copy of the source for Drupal Core and all the contributed modules. The pristine copies won't contain the changes you made. Further, the upstream projects are not paying attention to Drupal 7 and will never look at any change you propose. Instead, it's up to you to develop a methodology of recording these changes to simplify making future updates.
For example - keep the changed files in a directory, and in the clone script copy the changed files over the top of the file they are to replace. That strategy works so long as the upstream project has not changed a file where you have made changes.
Solving warning of function where optional parameter xxx is declared before required parameter yyy
This issue was difficult to solve. The error message, shown here, points to a line in bootstrap.inc but nowhere in that file are those parameter names mentioned.
Deprecated function: Optional parameter $decorators_applied declared before \
required parameter $app is implicitly treated as a required parameter in include_once() \
(line 3563 of /var/www/dev.visforvoltage.org/includes/bootstrap.inc).
Deprecated function: Optional parameter $relations declared before \
required parameter $app is implicitly treated as a required parameter in include_once() \
(line 3563 of /var/www/dev.visforvoltage.org/includes/bootstrap.inc).
The clue is that the line in question includes code using include_once. Therefore, the error message is coming from the code being included rather than from this line in bootstrap.inc. The trick is finding where those parameters exist.
The offending code is in a module named moopapi which contains a class structured as so;
abstract class Logger extends Decorator {
// ...
public function __construct($decorators_applied = array(), &$relations = array(), $app) {
// ...
}
// ...
}
The parent class, Decorator, has a matching constructor. The $app parameter is stored in a property of this class.
The problem is that the optional parameters decorators_applied and relations are before the non-optional parameter app. This is a basic programming fault. But, it's compounded by the fact that PHP used to not say anything about this, and now it does. Maybe PHP stopped being such a bad language?
The fix in this case is:
abstract class Logger extends Decorator {
// ...
public function __construct($decorators_applied = array(), &$relations = array(), $app = null) {
// ...
}
// ...
}
This makes $app into an optional parameter, giving it a decent default value.
Solving warning: Undefined property: stdClass::$content in advanced_forum_get_reply_link
This comes from the Advanced Forum module at a line of code reading this way:
$fragment = $node->content['comments']['comment_form']['#id'];
This says the content property doesn't exist, it is undefined.
The solution is to rewrite it as so:
$fragment = property_exists($node, 'content')
? $node->content['comments']['comment_form']['#id']
: '';
If the property exists, then go ahead with the code, otherwise substitute an empty string.
Solving many warnings in the autoload module
The
Autoload module allows modules to use PHP5 autoloading capabilities. It's a little too close to the metal for my understanding of PHP. With Drupal 7 and earlier, this module was required by several of the modules. We are instructed to install this module if requested by the maintainers of another module.
Running under PHP 8.2 we get these warning messages:
Deprecated function: Return type of ArrayContainer::offsetExists($offset) should either be \
compatible with ArrayAccess::offsetExists(mixed $offset): bool, \
or the #[\ReturnTypeWillChange] attribute should be used \
to temporarily suppress the notice in require_once() \
(line 11 of /var/www/dev.visforvoltage.org/sites/all/modules/autoload/autoload.cache.inc).
Deprecated function: Return type of ArrayContainer::offsetGet($offset) should either be \
compatible with ArrayAccess::offsetGet(mixed $offset): mixed, \
or the #[\ReturnTypeWillChange] attribute should be used \
to temporarily suppress the notice in require_once() \
(line 11 of /var/www/dev.visforvoltage.org/sites/all/modules/autoload/autoload.cache.inc).
There were many such notices complaining about several method signatures.
One issue is the signature for ArrayAccess::offsetGet(mixed $offset): mixed is incorrect. The code in question is:
/**
* {@inheritdoc}
*/
public function offsetGet($offset) {
return $this->data[$offset];
}
And as you see it doesn't include mixed in the required locations.
Change the signature like so:
public function offsetGet(mixed $offset): mixed {
return $this->data[$offset];
}
And the error message goes away.
Alternatively a function can be changed this way:
#[\ReturnTypeWillChange]
public function rewind() {
reset($this->data);
}
This annotation is also mentioned in the warning message. Its use also makes the warning messages go away.
Solving warning: Deprecated function: Creation of dynamic property backup_migrate_destination::$settings is deprecated
This warning showed up in many contributed modules. It refers to old PHP behavior where assigning a value to an object property where the property wasn't declared. In such a case PHP used to dynamically create the property. Arguably this is incorrect behavior on PHP's part, because it should enforce that properties are declared before being used. Indeed, the PHP team made this change in PHP, and this warning message is the result.
In every case there is a class, inside which is a method assigning property values where the property is not defined as part of the class definition.
Here's a concrete example.
class backup_migrate_item {
// ...
/**
* Load an existing item from an array.
*/
public function from_array($params) {
foreach ($params as $key => $value) {
if (method_exists($this, 'set_' . $key)) {
$this->{'set_' . $key}($value);
}
else {
// Error printed on this line
$this->{$key} = $value;
}
}
}
// ...
}
The statement $this->{$key} = $value triggered the error.
The solution is to annotate the class definition as so:
#[\AllowDynamicProperties]
class backup_migrate_item {
// ...
}
This informs PHP to enable dynamically assigned properties for instances of this class.
Solving warning: The following module is missing from the file system: devel_node_access.
This message is printed when Drupal attempts to find a module or theme which cannot be found.
The modules are recorded in the system table. From the contents of that table, Drupal looks for the corresponding files to use them. It can happen that those files have been deleted.
In this case, devel_node_access is a developer-only module that displays "node access" information. I must have installed it once to explore an issue. It is not needed now, and I should delete it from the table.
The full message is:
User warning: The following module is missing from the file system: devel_node_access. For information about how to fix this, see the documentation page(link is external). in _drupal_trigger_error_with_delayed_logging() (line 1184 of /var/www/dev.visforvoltage.org/includes/bootstrap.inc).
The message includes a link to a documentation page on drupal.org from which I got this command to run:
root@5fd8ae19366a:/var/www/dev.YOUR-DOMAIN# drush sql-query \
"DELETE from system where type = 'module' AND name IN ('devel_node_access');"
root@5fd8ae19366a:/var/www/dev.YOUR-DOMAIN# drush cc
Solving the issue of Drupal create content forms being truncated
In Drupal, we create or edit content using using the page https://example.com/node/add/forum. There's a lot things on this page because it contains a big multifaceted HTML form. Upon updating my site one side effect was this:
The form is cut off following the selector for the content filter types. This should have been a big clue, but I was distracted by the error message about DOMSubtreeModified in the browser console log.
I thought DOMSubtreeModified was the error. I learned that this feature has been deprecated and will be shortly removed from Google Chrome. I searched all through the Drupal source and could not find where this error existed in Drupal. Eventually I noticed that the stack trace pointed into the Google Chrome extension for viewing PDF documents. Hence, it had nothing to do with Drupal, and disabling that extension made that particular message go away.
As for the truncated create content form, it took a few days to trace the issue. The problem was this:
Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> ...
This is the HTML describing the tags which folks are allowed to use in their content. It's useful for this to be shown to users. This code is not visible in the screen capture, but it is viewable using View Page Source. The thing to notice is that these are live HTML elements. As live HTML elements, they capture everything which follows. Some of the tags like <embed> and <iframe> in the list will make things disappear.
To fix this means making those tags inert. The key is that <a> looks like <a> when rendered in the browser, but it's not an HTML tag.
The text Allowed HTML tags is generated in the core filters module, in the file modules/filter/filter.module.
The function theme_filter_guidelines outputs this HTML. That the Allowed HTML tags snippet is surrounded by a <div ..class="filter-guidelines-item.."> tag that leads us to the next function.
The function theme_filter_guidelines outputs the HTML for the guidelines which includes this Allowed HTML Tags thing.
That function calls _filter_tips to generate the HTML ... In _filter_html_tips is the code in question, which was fixed by making this change:
// This tends to have bare HTML tags which doesn't work well
// in this context. This converts the < and > to HTML entities.
// https://www.w3schools.com/php/func_string_htmlentities.asp
$output = htmlentities(t('Allowed HTML tags: @tags', array('@tags' => $allowed_html)));
This adds a call to htmlentities to convert elements like <a> to <a>. The result is to convert the tags to a sequence which looks like an HTML tag, but is nothing of the sort.
Bottom line is that once we found this line of code, the htmlentities function converted these tags into inert HTML.
Implementing Drupal 7 upload progress
This message may appear in the Drupal status report:
Upload progress Not enabled: Your server is capable of displaying file upload progress, but does not have the required libraries. It is recommended to install the PECL uploadprogress library(link is external) (preferred) or to install APC(link is external).
The links given are:
- The PECL uploadprogress library:
http://pecl.php.net/package/uploadprogress and https://github.com/php/pecl-php-uploadprogress
- The APC library:
http://us2.php.net/apc
The link for APC turns up a page saying that doesn't exist any longer. But the PECL uploadprogress library does exist at the above links.
In the Dockerfile for the container add this line:
RUN pecl install uploadprogress && docker-php-ext-enable uploadprogress
This is adapted from the Dockerfile in the GitHub repository.
Afterwards, you must rebuild the container image and redeploy the container. If successful, the message shown earlier goes away.
Implementing Drupal 7 filesystem caching on PHP 8.2
Drupal supports caching of various results to improve performance. Caching can go to the database, to a server like Memcache, or to the local filesystem.
The
FileCache serves this purpose. Its description starts with a comparison of different caching mechanisms and explains which to use for what purpose.
In my case, visforvoltage.org runs on a single instance and therefore does not require a distributed cache system. REDIS or Memcache is typically used for distributed distributed systems of multiple webservers cache servers. FileCache stores cached objects in the local filesystem, it only supports a single webserver, is claimed to be faster, and even allows hosting on shared webhosting.
After installing and enabling the FileCache module, you'll first see an error about setting a dynamic property on an object. See the solution for that presented earlier in this article, namely #[\AllowDynamicProperties]. After that you'll see this message in the status report
File Cache No cache bins are served by File Cache. Please follow the instructions in README.txt.
This includes a link to the project README on drupal.org which explains what to do.
We start by adding this to the end of sites/default/settings.php
$conf['cache_backends'] = array('sites/all/modules/filecache/filecache.inc');
$conf['cache_default_class'] = 'DrupalFileCache';
For a more extensive cache configuration see the README. If the filecache module is not in sites/all/modules you must adjust the cache_backends path.
The next step is setting up a directory in which to store the cache. It is important for security reasons that the cache files are not available by web requests. The README discusses making sure the Apache configuration blocks access to directories starting with .ht.
This is my configuration.
$conf['filecache_directory'] = '/var/www/YOUR-DOMAIN/sites/default/files/.ht.filecache';
From the FileCache README it's clear the cache could be stored in /tmp or other locations. I made this choice several years ago, I don't remember why, but I'm sticking with this because it works.
To verify the security requirement, in your browser visit https://YOUR-DOMAIN/sites/default/files/.ht.filecache. Success is receiving this message:
Forbidden You don't have permission to access this resource.
Another guage of success is that in the Drupal status report, the FileCache module shows no issues.
Implementing Drupal 7 Memcached caching on PHP 8.2
In the previous section we went over setting up FileCache, because my site has a single instance and does not need a distributed cache system. For those of you who need a distributed cache let's look at a simple setup for Memcached.
Memcached is a data storage service that can work either as a single instance, or a cluster of instances. You can learn more at
http://memcached.org/
The Drupal module for Memcached caching is:
https://www.drupal.org/project/memcache
The README for that module has detailed instructions:
https://git.drupalcode.org/project/memcache/-/blob/7.x-1.x/README.txt?ref_type=heads
The Docker image we'll use is:
https://hub.docker.com/_/memcached
In the Docker container we use for hosting Drupal, we need to install the PECL extension memcache. As with the uploadprogress extension, we add this:
RUN pecl install memcache && docker-php-ext-enable memcache
We then rebuild the container and restart the Compose file.
We also need to add a Memcached container to our system. To do so, add this to the Compose file:
memcache:
image: memcached:1.6
container_name: memcache
networks:
- dbnet
As of this writing v1.6 is the current version. This sets up a default Memcached deployment with one instance.
It will be available on dbnet with the host name memcache, meaning the default connection string is memcache:11211 because the default Memcached port is 11211.
The next step is to download and install the module. This is the way I did it:
$ docker exec -u33 -it drupal-vv82 bash
www-data@fb3400325bfb:~$ cd YOUR-DOMAIN
www-data@fb3400325bfb:~/YOUR-DOMAIN$ drush dl memcache
Project memcache (7.x-1.8) downloaded to /var/www/YOUR-DOMAIN/sites/all/modules/memcache.
Project memcache contains 2 modules: memcache_admin, memcache.
The autoloading class map has been rebuilt.
www-data@fb3400325bfb:~/YOUR-DOMAIN$ drush en memcache
The following extensions will be enabled: memcache
Do you really want to continue? (y/n): y
memcache was enabled successfully.
While it says there are two modules, we're only enabling memcache. I did enable the memcache_admin module and it doesn't do anything useful, in my humble opinion.
If you are using a Drush Makefile, make sure to add memcache.
Like with FileCache, when Memcache is first installed there's an error for which you must insert #[\AllowDynamicProperties] into the module source. Once that is accomplished you'll see the Drupal status report tell you Memcache has not been configured.
The README contains a multiple-step process for enabling Memcache, and we've already done a couple of those steps. It also describes many details about values to set in the $conf array, especially when deploying a Memcached cluster. Rather than repeat all that, I found their Example 2 was very close to what is needed for a simple single-instance deployment like we have here.
In sites/default/settings.php, add the following:
$conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
$conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
$conf['memcache_stampede_protection'] = TRUE;
$conf['cache_default_class'] = 'MemCacheDrupal';
// The 'cache_form' bin must be assigned no non-volatile storage.
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
// Don't bootstrap the database when serving pages from the cache.
$conf['page_cache_without_database'] = TRUE;
$conf['page_cache_invoke_hooks'] = FALSE;
// Important to define a default cluster in both the servers
// and in the bins. This links them together.
$conf['memcache_servers'] = array('memcache:11211' => 'default');
$conf['memcache_bins'] = array('cache' => 'default');
The first four lines are roughly the same as for FileCache. The middle few lines set values recommended by the README. The last two describe the single Memcached instance.
Namely, in $conf['memcache_servers'] one describes the Memcached cluster. This is the simplest cluster, available at host memcache and port 11211. The string default can be replaced by values giving precedence etc, and you can have more than one entry in this array if you've deployed multiple Memcached instances. The README has the details.
The $conf['memcache_bins'] value is related, and similarly defines a single instance. For more complex scenarios read the README.
At this point the Drupal status page should not show any errors, indicating the Memcache module is doing its thing. The Admin module adds a display at the bottom of the page of memory utilization, which you may want to enable to verify its behavior.
Configuring Database 4 byte UTF-8 support for Drupal 7 on PHP 8.2
Drupal 7 now supports 4-byte UTF-8 characters. A very important (supposedly) result is being able to support modern emojis. Whether or not you think emojis are important, it means Drupal 7 supports the full power of unicode text.
The Drupal status report will show a message like this if your database should be updated:
The procedures to follow are outlined on the Drupal website:
https://www.drupal.org/node/2754539
MySQL or MariaDB sites need to convert their Drupal 7 database to support utf8mb4 text. On PostgresQL or SQLITE this conversion is not required.
The task at hand is converting your database, which probably supports utf8_general_ci collation, to using utf8mb4_unicode_ci collation.
Prior to converting the database tables we must enable large indexes in MySQL. In the MySQL configuration file, typically named /etc/my.cnf, add these options:
[mysqld]
innodb_large_prefix=true
innodb_file_format=barracuda
innodb_file_per_table=true
In the my.cnf provided by the MySQL container there was already a [mysqld] section. Therefore, I commented out that line just leaving the three configuration settings.
HOWEVER - later on, I decided to go ahead and convert directly to MySQL 8. In that case, I did not create a custom /etc/y.cnf and did not change any of these settings. The recommendations here are useful for implementing MySQL customization.
To implement this in Docker, I copied the text of /etc/my.cnf to a file on the host system. The directory /opt/db holds the Docker configuration for the MySQL server. In the docker-compose.yml file, the volume mounts for the container are changed to this:
volumes:
- /opt/db/db:/var/lib/mysql
- /opt/db:/db
- /opt/db/my.cnf:/etc/my.cnf
The last maps the local file, /opt/db/my.cnf into the container. To verify
$ docker exec -it db bash
bash-4.2# cat /etc/my.cnf
...
In this example the container name is db, and we use cat to examine the config file.
The next step is to create a new database instance in which to experiment with the conversion. The conversion can potentially screw up, and it's best to leave your production database untouched until you've proven a conversion process that works for your site.
Using PHPMyAdmin, I created a new database of and a user ID with permissions to the new database. The production database name is visforvoltage_d7, so the test database was named visforvoltage_d7_utf8mb4_unicode_ci.
With the new database setup, we use mysqldump and mysql to clone the database.
$ docker exec -it db bash
bash-4.2# mysqldump -u ...USER-NAME... --password='..PASSWORD..' -h db visforvoltage_d7 >/db/vv.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.
bash-4.2# mysql -u ...USER-NAME... --password='..PASSWORD..' -h db visforvoltage_d7_utf8mb4_unicode_ci </db/vv.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
This dumps the existing database to a file, then imports it into a new database. And, yes, using the password on the command line is a bad idea. The risk is the password being recorded into shell history. However, in this case the shell is inside an ephemeral Docker container, and any shell history will vanish once the container is recreated.
Also notice that the new database is in the same MySQL server instance as the production database. This is good enough for validating the conversion process. Later on, when it was time to do the actual conversion the best path was found to be combining the MySQL 5.7==>8 conversion with the UTF8==>UTF8MB4 conversion.
The next step is to run an SQL command for generating a table of commands to alter the character set and collation for all the tables.
bash-4.2# mysql -u ...USER-NAME... --password='..PASSWORD..' -h db visforvoltage_d7_utf8mb4_unicode_ci
...
mysql> SELECT CONCAT('ALTER TABLE `', TABLE_NAME,'` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;') AS mySQL
-> FROM INFORMATION_SCHEMA.TABLES
-> WHERE TABLE_SCHEMA= "visforvoltage_d7_utf8mb4_unicode_ci"
-> AND TABLE_TYPE="BASE TABLE";
+-----------------------------------------------------------------------------------------------+
| mySQL |
+-----------------------------------------------------------------------------------------------+
| ALTER TABLE `abuse` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
| ALTER TABLE `abuse_reasons` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
| ALTER TABLE `abuse_status` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
| ALTER TABLE `abuse_status_log` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
| ALTER TABLE `abuse_warnings` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
| ALTER TABLE `access` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; |
...
This command prints, in table-output format. , the SQL commands for converting the database tables to charset utf8mb4 and utf8mb4_unicode_ci collation. We need to extract the commands from the output format. Copy/paste the lines with the commands into a text editor. I use /bin/vi for this kind of task. Two vi commands were useful
1,$s/^| //--- Deletes the|prefix on every line1,$s/\;.*$/\;/--- Removes everything after the;on every line
This can probably be done in a script with /bin/sed. I don't think Nano offers this kind of global search and replacement...?
But, instead of converting those commands to a shell script, read a little ways down to view another shell script.
That leaves an SQL script which can be executed as so:
bash-4.2# mysql -u ...USER-NAME... --password='...PASSWORD...' \
-h db visforvoltage_d7_utf8mb4_unicode_ci </db/alter.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1062 (23000) at line 154: Duplicate entry 'public://u7379/photo 1.JPG' for key 'uri'
This takes awhile to run. During execution you might use PHPMyAdmin to browse the tables and notice that not all tables have been converted.
If you're good and have fed the database gods recently this will successfully complete. It seems I'd not done so. Notice the error about a duplicate entry.
This is from the files table. The user in question has uploaded two files, one named photo 1.jpg and the other named photo 1.JPG. In the Unix filesystem these are different file names. But a case-insensitive database index will see these as the same filename.
The instructions say this:
Tables may also fail to convert if two entries of a unique key differ only in case. Use utf8mb4_bin when case-sensitive collation is required.
But, that advice only served to confuse me into trying this ALTER command:
ALTER TABLE `files` CONVERT TO CHARACTER SET utf8mb4_bin COLLATE utf8mb4_bin;
But, that gave the same error.
Searching for clues I came across a
stackoverflow posting about character set conversion from which this shell script was developed.
CHARSET="utf8mb4_bin"
DB="visforvoltage_d7_utf8mb4_unicode_ci"
USER=USER-NAME
PW='PASSWORD'
(
echo 'ALTER DATABASE `'"$DB"'` CHARACTER SET utf8mb4 COLLATE '"$CHARSET"';'
mysql -u $USER --password="$PW" "$DB" -e "SHOW TABLES" --batch --skip-column-names \
| xargs -I{} echo 'ALTER TABLE `'{}'` CONVERT TO CHARACTER SET utf8mb4 COLLATE '"$CHARSET"';'
) \
| mysql -u $USER --password="$PW" "$DB"
Compare to the above commands and you see it's roughly the same. This converts the database to use the utf8mb4 character set, and to use utf8mb4_bin for collation. This is set in both the DATABASE and in each TABLE. This script also has the advantage of not requiring an intermediate file which must be edited in /bin/vi. I suggest running this once with the last line commented-out so you can inspect the generated commands.
However - this generated the following error:
ERROR 1067 (42000) at line 252: Invalid default value for 'completed'
This error refers to a table, tasks, which has a Date column named completed.
mysql> describe tasks;
+-------------+------------------+------+-----+------------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+------------+-------+
| nid | int(10) unsigned | NO | PRI | 0 | |
| parent | int(10) unsigned | NO | | 0 | |
| assigned_to | int(10) unsigned | NO | | 0 | |
| order_by | float unsigned | NO | | 0 | |
| completed | date | NO | | 0000-00-00 | |
+-------------+------------------+------+-----+------------+-------+
The problem is described in
a Stack Overflow posting discussing how to handle a default Date value such as the one shown here.
MySQL has a sql_mode variable which is used for customizing how SQL is interpreted. The problem in this case is at least the settings: NO_ZERO_IN_DATE, NO_ZERO_DATE.
To see the current SQL mode, execute the command:
mysql> SELECT @@GLOBAL.sql_mode global, @@SESSION.sql_mode session;
There is an sql_mode for both the global and session scope, and this prints both of those strings. In my case a lot of modes were already set. Maybe some of those settings are perfectly valid. The Stack Overflow discussion said to set sql_mode to an empty string. What if some of those settings are very important? Instead, it seems better to just remove the offending settings.
To do that, execute this command:
mysql> SET GLOBAL sql_mode = 'ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
This is the same string after removing NO_ZERO_IN_DATE, NO_ZERO_DATE. The setting change can also be made in /etc/my.cnf by adding an sql-mode line.
After this, the above shell script executes correctly without error.
What we've done is a dry-run of converting the Drupal 7 database to support multi-byte UTF8 text and collation.
At this point we could have proceeded with UTF8==>UTF8MB4 conversion in the MySQL 5.7 database. Instead, I dedided to jump directly to combining that with the conversion from MySQL 5.7==>8.
Updating a Drupal 7 site from MySQL 5.7 to MySQL v8 or MariaDB v10
The
Drupal documentation on MySQL 8.x support in Drupal 7 suggests it'll be easy to convert.
Core Drupal fully supports MySQL 8. The code requires these permissions to the database: SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER. Some contributed modules will require additional permissions, such as: CREATE TEMPORARY TABLES, LOCK TABLES.
FWIW, the hosted MySQL service on my web service provider (Dreamhost) supports the first set of permissions, but not the last two. In other words, hosting Drupal on MySQL 8 on shared hosting might (or might not) work.
Given the experience earlier, we can expect some contributed modules will have some difficulties. There may be older modules, that haven't been updated, that have old unsupported SQL statements.
It does suggest this:
You might wish to simply create your database and proceed with installing Drupal, and then refer back to Drupal documentation for specific troubleshooting help if you run into problems.
After thinking all this through, I decided to host MySQL 8 on my VPS, and to perform the UTF8MB4 conversion as part of the migration to MySQL 8.
The Compose file for launching MySQL 8 for Drupal 7 is:
services:
db8:
image: "mysql:8"
container_name: db8
command: [ "mysqld",
"--character-set-server=utf8mb4",
"--collation-server=utf8mb4_unicode_ci",
"--bind-address=0.0.0.0",
"--innodb_file_per_table=true" ]
# "--innodb_large_prefix=true",
# "--innodb_file_format=barracuda",
networks:
- dbnet
volumes:
- /opt/mysql8/db:/var/lib/mysql
- /opt/mysql8:/db
# - /opt/mysql8/my.cnf:/etc/my.cnf
restart: always
environment:
MYSQL_ROOT_PASSWORD: "ROOT PASSWORD"
# MYSQL_ROOT_HOST: "10.%"
MYSQL_USER: USER-NAME
MYSQL_PASSWORD: MySQL PASSWORD FOR USER
MYSQL_DATABASE: DATABASE NAME
phpmyadmin:
image: phpmyadmin/phpmyadmin
networks:
- dbnet
- servernet
environment:
- PMA_ARBITRARY=1
ports:
- "8001:80"
networks:
dbnet:
external: true
servernet:
external: true
On my VPS, the virtual networks are managed externally to the Compose files. The dbnet network is used for databases, and is not exposed outside the server. The servernet network exposes services to NGINX Proxy Manager which in turn is the proxy for exposing a service to the public Internet.
The configuration for MySQL 8 is almost the same as for the MySQL 5.7 setup. Two settings are commented out because the MySQL server would not initialize with those settings. I'm also including PHPMyAdmin to make it easier to manage the MySQL instance. It is configured so users must login to a database server. Since it is attached to dbnet this lets me access the MySQL instance via a useful user interface.
Earlier, the dry-run UTF8MB4 conversion showed me how to do it quickly, but it had not been performed on the production database. It turns out the MySQL 5.7==>8 conversion is trivial, and combining it with the conversion worked great.
The steps I followed are:
- With the above Compose file, run
docker compose upand make sure it initializes the server correctly. Once the new database server is initialized you can rerun the Compose file with the-doption to put it in the background. - Using PHPMyAdmin, create a database and user which can access that database, giving it appropriate permissions
- Put the site in maintenance mode -- Note that the site is running on the old database server.
- Use
mysqldumpas shown earlier to make a database backup. - For the new database server, use
docker exec -it db8 bashto get into the container, then usemysqlin that container to migrate the SQL into the new database. - THIS TURNED OUT TO BE UNNECESSARY Set the global
sql_modevalue. - Run the conversion script while inside the container of the new MySQL 8 server.
- Modify
site/default/settings.phpto set the database HOST, USER, PASSWORD, and DATABASE access, and add thecharsetandcollationsettings. - While in maintenance mode, as the administrator user, check out the site. If there's a problem set
site/default/settings.phpto revert to the old database instance. - Once you're satisfied, take the site out of maintenance mode.
With the MySQL container the documentation says to setup MYSQL_USER, MYSQL_PASSWORD and MYSQL_DATABASE variables. You can certainly do that, but it limits you to one database per MySQL server instance. Since MySQL can support multiple databases on one server instance, I prefer to do so as discussed in Using multiple databases and PHPMyAdmin in a single MySQL Docker instance
In PHPMyAdmin, one therefore clicks on the Databases tab and creates a new database. Earlier it was deemed necessary to use the utf8mb4_bin collation, so that was selected during database creation. After creating the database, click on the database name which appears in the left-hand navigation, and head to the Privileges tab. On that screen is a button for creating a user ID which can access this database.
On the Privileges screen you enter a user name, password, and then select permissions. I selected every permission in the Data and Structure sections, plus the LOCK TABLES permission. This covered the permissions described in the Drupal documentation.
Migrating the SQL into the database is two steps
$ docker exec -it db bash
bash-4.2# mysqldump -u USER-NAME --password='PASSWORD' DATABASE_NAME >/db/DATABASE.sql
This dumps the existing database into an SQL file in the /db directory, which in turn is mounted from the host system into the container.
$ docker exec -it db8 bash
bash-5.1# mysql -u USER-NAME --password='PASSWORD' DATABASE_NAME </db/DATABASE.sql
This is executed inside the container for the new MySQL server instance. It inserts the SQL into that database instance.
While still inside that container, run:
bash-5.1# sh /db/convert.sh
Update the convert script as appropriate for this database instance. It performs any needed UTF8 to UTF8MB4 conversions.
I neglected to set the sql_mode value, and that particular issue did not come up. Running SELECT @@GLOBAL.sql_mode global, @@SESSION.sql_mode session; on the MySQL 8 instance show that the NO_ZERO_IN_DATE,NO_ZERO_DATE options are enabled, but the conversion did not fail.
In site/default/settings.php is the PHP object describing the database configuration. Mine is ultra simple. Your site may have a more complex configuration.
$databases = array (
'default' =>
array (
'default' =>
array (
'driver' => 'mysql',
'database' => 'DATABASE-NAME',
'username' => 'USER_NAME',
'password' => 'PASSWORD',
'host' => 'db8',
'port' => '',
'prefix' => '',
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_bin',
),
),
);
The new MySQL 8 container has the name db8 which distinguishes it from the 5.7 container whose name is just db. For the other settings you simply change to the credentials required for the new server. The charset and collation settings are to be added to this object.
At this point your production Drupal instance is still in maintenance mode, but is connected to the new database instance. You should use the site and verify that things are working.
The first task is to view the Drupal status report. In my case, it included this message, which had also been visible on the MySQL 5.7 server.
The message itself is generated in modules/system/system.install and uses functions in includes/database/mysql/database.inc. Examining code in these files will be helpful for debugging this issue.
This message comes up if the Drupal variable drupal_all_databases_are_utf8mb4 is not TRUE. The value of that variable is set here:
$connection = Database::getConnection();
if ($connection->utf8mb4IsConfigurable() && $connection->utf8mb4IsActive()) {
variable_set('drupal_all_databases_are_utf8mb4', TRUE);
}
It's possible to dive deeper into the code, which I did. The key is setting drupal_all_databases_are_utf8mb4 to TRUE. The code to do this is in modules/system/system.install, but I don't know (er.. remember) enough Drupal to know if that code can be executed. The values for the modules/system/system.install and utf8mb4IsActive functions do return TRUE, So, if this was to be executed this variable would be set to TRUE.
Since my Docker container contains Drush, it's possible to use it to set the variable. Specifically:
$ docker exec -it drupal-vv82 bash
root@3ca108b98628:/var/www/DOMAIN-NAME# drush vget drupal_all_databases_are_utf8mb4
No matching variable found.
root@3ca108b98628:/var/www/DOMAIN-NAME# drush vset drupal_all_databases_are_utf8mb4 TRUE
drupal_all_databases_are_utf8mb4 was set to TRUE. [success]
root@3ca108b98628:/var/www/DOMAIN-NAME# drush vget drupal_all_databases_are_utf8mb4
drupal_all_databases_are_utf8mb4: true
After this, go to the Drupal status report, and you'll see the error is no longer visible.
So long as you do not find any bugs or errors, you can bring the site out of maintenance mode. At which point you'll have successfully converted your Drupal site to run on MySQL 8 and also use UTF8MB4 characters.
Conclusion
I had given up on updating my Drupal 7 site for many years, and was resigned to it staying on a crumbling old version of Drupal and MySQL. As long as the old Docker images were available I would be able to keep the site running and crossed fingers everything would be okay.
Fortunately the Drupal 7 team is still actively supporting that old version. By taking these steps to update to the latest Drupal core, the latest PHP version, and the latest MySQL version, one can stretch out the lifetime for your Drupal 7 site.
In my case, because the Drupal ecosystem seems to have abandoned the idea of using Drupal for online community, my next task is to convert this site to another system. There are plenty of "online forum" packages available. I wonder how horrendously difficult it will be to do such a conversion? My mind is predicting a many month-long process which is not at all enticing. What was so wrong with being Community Plumbing that the Drupal ecosystem has abandoned support for online community websites?
For sites where it's desired to remain on Drupal, you should be planning a transition to Drupal 10 or whatever is the latest thing.
