Skip to content

Migrate from php8.0 to php8.1 #2309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Migrate from php8.0 to php8.1 #2309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
8884f41
add logic to migrate from php8.0 to php8.1
kiekerjan Sep 29, 2023
5cd36d2
merge upstream
kiekerjan Jul 23, 2024
3559488
z-push admin and top update with PHP_VER change
matidau Aug 16, 2024
f7386d1
Merge remote-tracking branch 'upstream/main' into upgradephp81
kiekerjan Feb 10, 2025
cac493f
Hardcode PHP version
kiekerjan Feb 12, 2025
87deed6
Use local variable to indicat PHP version to be used
kiekerjan Feb 15, 2025
36dc5ff
Fix missing $
kiekerjan Feb 16, 2025
c559974
make sure php cleanup runs on nextcloud upgrade
kiekerjan Feb 22, 2025
42d1857
Merge pull request #21 from matidau/z-push-admin-top-php-ver-change
kiekerjan Feb 22, 2025
0a45999
Make sure to unhold packages to prevent future issues
kiekerjan Feb 22, 2025
5c02ea3
Add comments on aligning used php version
kiekerjan Feb 27, 2025
8e7b4b5
Removed unneeded sudo call
kiekerjan Feb 27, 2025
da6a204
add quotes around variable
kiekerjan Mar 2, 2025
6adc494
Add quotes
kiekerjan Mar 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conf/nginx-top.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@
## your own --- please do not ask for help from us.

upstream php-fpm {
server unix:/var/run/php/php8.0-fpm.sock;
server unix:/var/run/php/php{{phpver}}-fpm.sock;
}

7 changes: 4 additions & 3 deletions management/backup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import rtyaml
from exclusiveprocess import Lock

from utils import load_environment, shell, wait_for_service
from utils import load_environment, shell, wait_for_service, get_php_version

def backup_status(env):
# If backups are disabled, return no status.
Expand Down Expand Up @@ -266,6 +266,7 @@ def get_target_type(config):

def perform_backup(full_backup):
env = load_environment()
php_fpm = f"php{get_php_version()}-fpm"

# Create an global exclusive lock so that the backup script
# cannot be run more than one.
Expand Down Expand Up @@ -301,7 +302,7 @@ def service_command(service, command, quit=None):
if quit:
sys.exit(code)

service_command("php8.0-fpm", "stop", quit=True)
service_command(php_fpm, "stop", quit=True)
service_command("postfix", "stop", quit=True)
service_command("dovecot", "stop", quit=True)
service_command("postgrey", "stop", quit=True)
Expand Down Expand Up @@ -339,7 +340,7 @@ def service_command(service, command, quit=None):
service_command("postgrey", "start", quit=False)
service_command("dovecot", "start", quit=False)
service_command("postfix", "start", quit=False)
service_command("php8.0-fpm", "start", quit=False)
service_command(php_fpm, "start", quit=False)

# Remove old backups. This deletes all backup data no longer needed
# from more than 3 days ago.
Expand Down
5 changes: 5 additions & 0 deletions management/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,11 @@ def wait_for_service(port, public, env, timeout):
return False
time.sleep(min(timeout/4, 1))

def get_php_version():
# Gets the version of PHP used by Mail-in-a-Box
# To modify PHP version, also update variable PHP_VER in functions.sh
return "8.1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you leave behind a new comment for the next developer that incrementing the version here alone isn't safe? (Like adding instructions how to increment PHP versions the safe way?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I add a comment referencing the other location where this is defind

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pardon, I can't find it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's here


def get_ssh_port():
port_value = get_ssh_config_value("port")

Expand Down
4 changes: 3 additions & 1 deletion management/web_update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from mailconfig import get_mail_domains
from dns_update import get_custom_dns_config, get_dns_zones
from ssl_certificates import get_ssl_certificates, get_domain_ssl_files, check_certificate
from utils import shell, safe_domain_name, sort_domains
from utils import shell, safe_domain_name, sort_domains, get_php_version

def get_web_domains(env, include_www_redirects=True, include_auto=True, exclude_dns_elsewhere=True):
# What domains should we serve HTTP(S) for?
Expand Down Expand Up @@ -82,6 +82,8 @@ def read_conf(conf_fn):

# Build an nginx configuration file.
nginx_conf = read_conf("nginx-top.conf")
# Hardcode php version in the nginx configuration file
nginx_conf = re.sub("{{phpver}}", get_php_version(), nginx_conf)

# Load the templates.
template0 = read_conf("nginx.conf")
Expand Down
3 changes: 2 additions & 1 deletion setup/functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
# -o pipefail: don't ignore errors in the non-last command in a pipeline
set -euo pipefail

PHP_VER=8.0
# To modify PHP version, also update method get_php_version() in utils.py
PHP_VER=8.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dito. Like mentioned before, it is safe to increment the version from here alone?

I'd add another inline comment here, that the version number is defined in two places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment about versioning.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved.


function hide_output {
# This function hides the output of a command unless the command fails
Expand Down
65 changes: 53 additions & 12 deletions setup/nextcloud.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,15 @@ user_external_hash=280d24eb2a6cb56b4590af8847f925c28d8d853e
# 5.3 You still can create, edit and delete users
# 5.4 Go to Administration > Logs and ensure no new errors are shown

# Set a local variable for the PHP version
NC_PHP_VER=$PHP_VER

# Clear prior packages and install dependencies from apt.
apt-get purge -qq -y owncloud* # we used to use the package manager

apt_install curl php"${PHP_VER}" php"${PHP_VER}"-fpm \
php"${PHP_VER}"-cli php"${PHP_VER}"-sqlite3 php"${PHP_VER}"-gd php"${PHP_VER}"-imap php"${PHP_VER}"-curl \
php"${PHP_VER}"-dev php"${PHP_VER}"-gd php"${PHP_VER}"-xml php"${PHP_VER}"-mbstring php"${PHP_VER}"-zip php"${PHP_VER}"-apcu \
php"${PHP_VER}"-dev php"${PHP_VER}"-xml php"${PHP_VER}"-mbstring php"${PHP_VER}"-zip php"${PHP_VER}"-apcu \
php"${PHP_VER}"-intl php"${PHP_VER}"-imagick php"${PHP_VER}"-gmp php"${PHP_VER}"-bcmath

# Enable APC before Nextcloud tools are run.
Expand Down Expand Up @@ -138,23 +141,23 @@ InstallNextcloud() {
if [ -e "$STORAGE_ROOT/owncloud/owncloud.db" ]; then
# ownCloud 8.1.1 broke upgrades. It may fail on the first attempt, but
# that can be OK.
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ upgrade
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ upgrade
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking ahead to when we upgrade to later versions after 8.1, would it make more sense for nc_php_ver to be a parameter for this function in the same way the hashes are? We could also then include the installation of earlier versions as part of the function instead of the main body of the code. i.e. where nc_php_ver not equals PHP_VER.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did think of that, but didn´t want to make more changes then necessary (I like optimizing changes like this, so difficult for me 😉 )
If we make this a function parameter it does make the code clearer I think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have already thought about it, All good by me then.

E=$?
if [ $E -ne 0 ] && [ $E -ne 3 ]; then
echo "Trying ownCloud upgrade again to work around ownCloud upgrade bug..."
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ upgrade
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ upgrade
E=$?
if [ $E -ne 0 ] && [ $E -ne 3 ]; then exit 1; fi
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ maintenance:mode --off
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ maintenance:mode --off
echo "...which seemed to work."
fi

# Add missing indices. NextCloud didn't include this in the normal upgrade because it might take some time.
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ db:add-missing-indices
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ db:add-missing-primary-keys
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ db:add-missing-indices
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ db:add-missing-primary-keys

# Run conversion to BigInt identifiers, this process may take some time on large tables.
sudo -u www-data php"$PHP_VER" /usr/local/lib/owncloud/occ db:convert-filecache-bigint --no-interaction
sudo -u www-data php"$NC_PHP_VER" /usr/local/lib/owncloud/occ db:convert-filecache-bigint --no-interaction
fi
}

Expand Down Expand Up @@ -214,10 +217,27 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc
return 0
fi

# Hint: whenever you bump, remember this:
# - Run a server with the previous version
# - On a new if-else block, copy the versions/hashes from the previous version
# - Run sudo ./setup/start.sh on the new machine. Upon completion, test its basic functionalities.
# Install php 8.0 for older versions of nextcloud that don't support 8.1
if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^2[0123] ]]; then
# Version 20 is the latest version from the 18.04 version of miab. To upgrade to version 21, install php8.0. This is
# not supported by version 20, but that does not matter, as the InstallNextcloud function only runs the version 21 code.
# We need php 8.0 for nextcloud 21-23, as php 8.1 is supported starting nextcloud 24

# Prevent installation of old packages
apt-mark hold php7.0-apcu php7.1-apcu php7.2-apcu php7.3-apcu php7.4-apcu

# Install php version 8.0
apt_install php8.0 php8.0-fpm php8.0-apcu php8.0-cli php8.0-sqlite3 php8.0-gd php8.0-imap \
php8.0-curl php8.0-dev php8.0-xml php8.0-mbstring php8.0-zip

# set php version 8.0 as default
NC_PHP_VER=8.0

# Make sure apc is enabled
tools/editconf.py /etc/php/$NC_PHP_VER/mods-available/apcu.ini -c ';' \
apc.enabled=1 \
apc.enable_cli=1
fi

if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^20 ]]; then
InstallNextcloud 21.0.7 f5c7079c5b56ce1e301c6a27c0d975d608bb01c9 4.0.7 45e7cf4bfe99cd8d03625cf9e5a1bb2e90549136 3.0.4 d0284b68135777ec9ca713c307216165b294d0fe
Expand All @@ -235,10 +255,31 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc
InstallNextcloud 24.0.12 7aa5d61632c1ccf4ca3ff00fb6b295d318c05599 4.1.0 697f6b4a664e928d72414ea2731cb2c9d1dc3077 3.2.2 ce4030ab57f523f33d5396c6a81396d440756f5f 3.0.0 0df781b261f55bbde73d8c92da3f99397000972f
CURRENT_NEXTCLOUD_VER="24.0.12"
fi
if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^24 ]]; then

if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^2[456] ]]; then
Copy link
Contributor

@matidau matidau Mar 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we look look at a function for the removal of the PHP versions, we will need to call an identical or nearly identical code for 8.1 when we move to a higher version. In my mind this would keep the if [[ ${CURRENT_NEXTCLOUD_VER} sections closer to their current format of: check condition, call function.

Note this also could be a future improvement instead of being included in this PR, just thought I would raise it now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of doing a separate version, but then I wondered if all dependencies are the same, so I didn´t put in the time to optimize that, just kept it as is.
Again, having two separate functions to install php dependencies and uninstall them, would make the code cleaner / clearer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, perfect later.

But then one of us, or a new developer, might forget? How about adding a TODO comment about it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don´t think we should put TODOs in the code. But I'm only one person 😉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don´t think we should put TODOs in the code. But I'm only one person 😉

Was just about to post this.

I think as soon as 8.1 is merged we should put together a PR for 8.2 and we can include this as an improvement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don´t think we should put TODOs in the code. But I'm only one person 😉

Great discussion. It's open source, developers come and leave. Any inline comments, TODO comments or whatever, are precious leftovers for the next developer to pick up.

I've done the same, for example this merged PR of mine, last year when I added plenty of comments:
https://github.com/mail-in-a-box/mailinabox/pull/2268/files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think as soon as 8.1 is merged we should put together a PR for 8.2 and we can include this as an improvement.

Ah, PHP v8.2, tell me, why do you think about it? And are there any documentation or TODO comments on how to bump a PHP version?

Copy link
Contributor Author

@kiekerjan kiekerjan Mar 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we might be able to skip a php version:

  • Create one PR containing upgrades of versions 27, 28 and 29. All are supporting php 8.1
  • Create another PR containing upgrade to php 8.3. Supported by nextcloud 28 and further.
  • Create another PR optimizing the code (see earlier discussions in this thread)

Update: In hindsight, I'm going too fast. There's are also other applications that depend on PHP, e.g. Roundcube and zpush. And for instance Roundcube dependencies (see stremlau/html5_notifier#57) are also to be taken into account

I use a little table for supported versions:

Nextcloud version supported php versions
20 7.2, 7.3, 7.4
21 7.3, 7.4, 8.0
22 7.3, 7.4, 8.0
23 7.3, 7.4, 8.0
24 7.4, 8.0, 8.1
25 7.4, 8.0, 8.1
26 8.0, 8.1, 8.2
27 8.0 (d), 8.1, 8.2 (r)
28 8.0 (d), 8.1, 8.2 (r), 8.3
29 8.0 (d), 8.1, 8.2 (r), 8.3
30 8.1 (d), 8.2, 8.3 (r)
31 8.1 (d), 8.2, 8.3 (r), 8.4

Also take into account dependency on the user_external addon

user_external version Nextcloud version supported
2.1.0 21-22
3.0.0 22-24
3.1.0 22-25
3.2.0 25-27
3.3.0 25-28
3.4.0 25-29

This actually leads me to the next bump in the road: support by user_external. Support for newer Nextcloud versions seems to stall But at least by going to version 29 of Nextcloud we get some air in keeping up with the development tempo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent summary. We should put these tables somewhere better, maybe under CONTRIBUTING.md?

Once this PR gets merged, I'll create incremental PRs to slowly upgrade our NC versions step by step.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the interests of moving this PR forward, can we agree that any additional steps can be left to after the PR?

# From nextcloud 24 and higher, php8.1 is supported, so we can now remove the php8.0 ppa and packages

# Reset the default php version used
NC_PHP_VER=8.1

# Remove older php version
apt-get purge -qq -y php8.0 php8.0-fpm php8.0-apcu php8.0-cli php8.0-sqlite3 php8.0-gd \
php8.0-imap php8.0-curl php8.0-dev php8.0-xml php8.0-mbstring php8.0-zip \
php8.0-common php8.0-opcache php8.0-readline

# Unhold packages
apt-mark unhold php7.0-apcu php7.1-apcu php7.2-apcu php7.3-apcu php7.4-apcu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that held packages might prevent the quiet apt installation commands that mailinabox issues, I think specifically in case that the hold package can be updated. To prevent future issues, unhold these packages to be sure.

fi

if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^24 ]]; then
InstallNextcloud 25.0.7 a5a565c916355005c7b408dd41a1e53505e1a080 5.3.0 4b0a6666374e3b55cfd2ae9b72e1d458b87d4c8c 4.4.2 21a42e15806adc9b2618760ef94f1797ef399e2f 3.2.0 a494073dcdecbbbc79a9c77f72524ac9994d2eec
CURRENT_NEXTCLOUD_VER="25.0.7"
fi

# Hint: whenever you bump, remember this:
# - Run a server with the previous version
# - On a new if-else block, copy the versions/hashes from the previous version
# - Run sudo ./setup/start.sh on the new machine. Upon completion, test its basic functionalities.
fi

InstallNextcloud $nextcloud_ver $nextcloud_hash $contacts_ver $contacts_hash $calendar_ver $calendar_hash $user_external_ver $user_external_hash
Expand Down
18 changes: 9 additions & 9 deletions setup/zpush.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,18 @@ if [ $needs_update == 1 ]; then
mv /tmp/z-push/*/src /usr/local/lib/z-push
rm -rf /tmp/z-push.zip /tmp/z-push

# Create admin and top scripts with PHP_VER
rm -f /usr/sbin/z-push-{admin,top}
echo '#!/bin/bash' > /usr/sbin/z-push-admin
echo php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php '"$@"' >> /usr/sbin/z-push-admin
chmod 755 /usr/sbin/z-push-admin
echo '#!/bin/bash' > /usr/sbin/z-push-top
echo php"$PHP_VER" /usr/local/lib/z-push/z-push-top.php '"$@"' >> /usr/sbin/z-push-top
chmod 755 /usr/sbin/z-push-top

echo $VERSION > /usr/local/lib/z-push/version
fi

# Create admin and top scripts with PHP_VER
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious here, why did you move this code out of the if-else block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @matidau noted in #2431, this code updates the used php version of zpush, but the code is only run on an update of zpush. Because PHP_VER is changed, the scripts should be updated even without a zpush update.

rm -f /usr/sbin/z-push-{admin,top}
echo '#!/bin/bash' > /usr/sbin/z-push-admin
echo php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php '"$@"' >> /usr/sbin/z-push-admin
chmod 755 /usr/sbin/z-push-admin
echo '#!/bin/bash' > /usr/sbin/z-push-top
echo php"$PHP_VER" /usr/local/lib/z-push/z-push-top.php '"$@"' >> /usr/sbin/z-push-top
chmod 755 /usr/sbin/z-push-top

# Configure default config.
sed -i "s^define('TIMEZONE', .*^define('TIMEZONE', '$(cat /etc/timezone)');^" /usr/local/lib/z-push/config.php
sed -i "s/define('BACKEND_PROVIDER', .*/define('BACKEND_PROVIDER', 'BackendCombined');/" /usr/local/lib/z-push/config.php
Expand Down
5 changes: 3 additions & 2 deletions tools/owncloud-restore.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#
# This script will restore the backup made during an installation
source /etc/mailinabox.conf # load global vars
source ./setup/functions.sh

if [ -z "$1" ]; then
echo "Usage: owncloud-restore.sh <backup directory>"
Expand All @@ -26,7 +27,7 @@ if [ ! -f "$1/config.php" ]; then
fi

echo "Restoring backup from $1"
service php8.0-fpm stop
service "php$PHP_VER-fpm" stop

# remove the current ownCloud/Nextcloud installation
rm -rf /usr/local/lib/owncloud/
Expand All @@ -45,5 +46,5 @@ chown www-data:www-data "$STORAGE_ROOT/owncloud/config.php"

sudo -u www-data "php$PHP_VER" /usr/local/lib/owncloud/occ maintenance:mode --off

service php8.0-fpm start
service "php$PHP_VER-fpm" start
echo "Done"
3 changes: 2 additions & 1 deletion tools/owncloud-unlockadmin.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
# purpopses only. After this point you are on your own.

source /etc/mailinabox.conf # load global vars
source ./setup/functions.sh

ADMIN=$(./mail.py user admins | head -n 1)
ADMIN=$(./management/cli.py user admins | head -n 1)
test -z "$1" || ADMIN=$1

echo "I am going to unlock admin features for $ADMIN."
Expand Down