Upgrade iRedMail from 0.9.6 to 0.9.7

Attention

Check out the lightweight on-premises email archiving software developed by iRedMail team: Spider Email Archiver.

Upgrade iRedMail from 0.9.6 to 0.9.7

Paid Remote Upgrade Support

We offer remote upgrade support if you don't want to get your hands dirty, check the details and contact us.

ChangeLog

Nov 12, 2017: Add Fail2ban jail nginx-http-auth.
Jul 3, 2017: Mention how to upgrade uwsgi (OpenBSD only), iRedAdmin and iRedAPD.
Jul 2, 2017: Mention Roundcube 1.3.0 requires PHP 5.4.
Jul 1, 2017: Initial publish.

General (All backends should apply these steps)

Update `/etc/iredmail-release` with new iRedMail version number

iRedMail stores the release version in /etc/iredmail-release after installation, it's recommended to update this file after you upgraded iRedMail, so that you can know which version of iRedMail you're running. For example:

0.9.7

Upgrade iRedAPD (Postfix policy server) to the latest stable release (2.1)

Please follow below tutorial to upgrade iRedAPD to the latest stable release: Upgrade iRedAPD to the latest stable release

Upgrade iRedAdmin (open source edition) to the latest stable release (0.8)

Please follow this tutorial to upgrade iRedAdmin open source edition to the latest stable release: Upgrade iRedAdmin to the latest stable release

Upgrade Roundcube webmail to the latest stable release (1.3.0)

Roundcube 1.3

Roundcube 1.3 requires at least PHP 5.4. If your server is still running PHP 5.3 and cannot upgrade to 5.4, please upgrade Roundcube to the latest 1.2 branch (1.2.5) instead.
Roundcube 1.3 no longer supports IE < 10 and old versions of Firefox, Chrome and Safari.
Roundcube 1.3 uses jQuery 3.2 and will not work with current jQuery mobile plugin. If you use any third-party plugin, please check its website to make sure it's compatible with Roundcube 1.3 before upgrading.

With the release of Roundcube 1.3.0, the previous stable release branches 1.2.x and 1.1.x will switch in to LTS low maintenance mode which means they will only receive important security updates but no longer any regular improvement updates.

There're several security fixes in Roundcube 1.2.4 and 1.2.5, all users are encouraged to upgrade it as soon as possible. For more details about this release, please check Roundcube release notes:

1.2.4

1.2.5

Please follow Roundcube official tutorial to upgrade Roundcube webmail to the latest stable release immediately:

How to upgrade Roundcube.

Fixed: improper order of Postfix HELO restriction rules.

iRedMail-0.9.6 and earlier releases didn't configure Postfix to apply custom HELO restriction rule before FQDN helo hostname check and DNS verification, this way you cannot whitelist some bad HELO hostnames. Please follow steps below to fix it.

Open file /etc/postfix/main.cf (Linux/OpenBSD) or /usr/local/etc/postfix/main.cf (FreeBSD), find parameter smtpd_helo_restrictions like below:

smtpd_helo_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    reject_non_fqdn_helo_hostname
    reject_unknown_helo_hostname
    check_helo_access pcre:/etc/postfix/helo_access.pcre

Move the check_helo_access line after permit_sasl_authenticated:

smtpd_helo_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    check_helo_access pcre:/etc/postfix/helo_access.pcre
    reject_non_fqdn_helo_hostname
    reject_unknown_helo_hostname

Reloading or restarting Postfix service is required.

Fixed: incorrect owner and permission for rotated Dovecot log files

Attention

This is applicable to Linux, not FreeBSD and OpenBSD.

iRedMail-0.9.6 and earlier releases have an incorrect logrotate setting for Dovecot log file, it causes all Dovecot log files are empty due to no required permission to open log files. Please follow steps below to fix it.

Please open file /etc/logrotate.d/dovecot, find line below:

    create 0600 vmail vmail

Remove above line and save the change.

Fixed: incorrect session.save_path in php-fpm pool config file on RHEL/CentOS

Attention

This is applicable to RHEL/CentOS system, and Nginx web server.

iRedMail-0.9.6 doesn't set path for session.save_path parameter in php-fpm pool config file /etc/php-fpm.d/www.conf, please fix it with steps below:

Open file /etc/php-fpm.d/www.conf, find line:

php_value[session.save_path] = "/var/lib/php/session"

The directory name should be sessions (ends with s), not session. So please change it to:

php_value[session.save_path] = "/var/lib/php/sessions"

Restarting php-fpm service is required:

service php-fpm restart

Fixed: incorrect freshclam setting `UpdateLogFile`

Attention

This is applicable to RHEL/CentOS system.

With iRedMail-0.9.6, freshclam program cannot update ClamAV signatures due to improper log file permission, please open its config file /etc/freshclam.conf or /etc/clamav/freshclam.conf, comment out setting UpdateLogFile to use syslog for logging.

#UpdateLogFile ...          # <- Comment out this parameter
LogSyslog true              # <- Make sure you have this line. If not present, please add it manually.

Fail2ban: fixes an improper filter and add new filter rule

iRedMail-0.9.7 fixes an improper filter for Dovecot log file which may cause incorrect ban, and adds a new filter for Roundcube log file to help ban bad client while Roundcube is running behind a proxy server.

On Linux:

cd /etc/fail2ban/filter.d/
rm -f dovecot.iredmail.conf roundcube.iredmail.conf
wget https://github.com/iredmail/iRedMail/raw/1.0/samples/fail2ban/filter.d/dovecot.iredmail.conf
wget https://github.com/iredmail/iRedMail/raw/1.0/samples/fail2ban/filter.d/roundcube.iredmail.conf

Restarting Fail2ban service is required.

Fail2ban: Add new jail for Nginx

Attention

This is applicable if you run Nginx as web server.

Let's add a new jail to stop bad clients which tried to perform http basic auth but failed.

Create file /etc/fail2ban/jail.d/nginx-http-auth.local with content below:

Attention

If directory /etc/fail2ban/jail.d/ doesn't exist, you can append content below in file /etc/fail2ban/jail.local instead.

[nginx-http-auth]
enabled     = true
filter      = nginx-http-auth
action      = iptables-multiport[name=nginx, port="80,443", protocol=tcp]
logpath     = /var/log/nginx/error.log

Restarting Fail2ban service is required.

NEW: New backup script for SOGo

Attention

This is not applicable to SOGo-2.x because it doesn't support backing up all users' data with command sogo-tool backup /path/to/backup/dir ALL.

iRedMail has script /var/vmail/backup/backup_mysql.sh (or backup_pgsql.sh) to backup SOGo database by dumping whole database to a plain SQL file as backup. It's not ideal because:

it's hard to restore single user's data with a plain SQL file.
if SOGo changes some SQL structure, it's hard to restore all data.

This new script does backup with sogo-tool backup command to avoid issues mentioned above, you can restore a single user's data or all users data with sogo-tool restore.

Please follow steps below to setup this daily cron job.

Download backup script. We store it under /var/vmail/backup, if you prefer a different directory, feel free to change the directory name used in commands below:

cd /var/vmail/backup/
wget https://github.com/iredmail/iRedMail/raw/1.0/tools/backup_sogo.sh
chmod 0400 backup_sogo.sh

This script will create new directory under /var/vmail/backup like below to store backup files:

/var/vmail/backup
            |- sogo/
                |- 2017/                # <- year
                    |- 03/              # <- month
                        |- 22.tar.bz2   # <- day (file name is: <day>.tar.bz2)

If you want to use another directory to store backup files, please open file backup_sogo.sh, update variable BACKUP_ROOTDIR with the new directory.

Run command crontab -e -u root to setup root user's cron job. Add content below as new job:

# SOGo: backup all users' data at 3:05AM everyday.
5   3   *   *   *   bash /var/vmail/backup/backup_sogo.sh

OpenBSD: Upgrade uwsgi to the latest 2.0.15

uwsgi is the interface between Nginx and iRedAdmin, so if you're running iRedAdmin, it's recommended to upgrade uwsgi to the latest version, 2.0.15.

Steps: Download the latest uwsgi, compile it, then restart uwsgi service.

cd /root/
ftp https://projects.unbit.it/downloads/uwsgi-2.0.15.tar.gz
tar zxf uwsgi-2.0.15.tar.gz
cd uwsgi-2.0.15
python setup.py install

uwsgi should be succesfully installed, then restart uwsgi service:

rcctl restart uwsgi

OpenLDAP backend special

Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

[CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

Open the daily MySQL backup script, it's /var/vmail/backup/backup_mysql.sh by default. if you use different storage directory during iRedMail installation, you can find the base directory with command postconf virtual_mailbox_base.
Find variable name CMD_MYSQLDUMP like below:

export CMD_MYSQLDUMP="mysqldump ..."

Make sure it has argument --skip-comments like below:

export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

Save your change. That's it.

MySQL/MariaDB backend special

SQL structure change in `vmail.alias` SQL table

We've made some changes to vmail.alias SQL table for easier account management. This change introduces 2 new SQL tables (forwardings, alias_moderators), and (optionally) dropped few columns in vmail.alias table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Warning

Please backup SQL database vmail before you run any SQL commands below.

Create required new SQL tables

Please connect to MySQL server as MySQL root user, and execute SQL commands below to create required new tables:

USE vmail;

CREATE TABLE IF NOT EXISTS alias_moderators (
    id BIGINT(20) UNSIGNED AUTO_INCREMENT,
    address VARCHAR(255) NOT NULL DEFAULT '',
    moderator VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    dest_domain VARCHAR(255) NOT NULL DEFAULT '',
    PRIMARY KEY (id),
    UNIQUE INDEX (address, moderator),
    INDEX (domain),
    INDEX (dest_domain)
) ENGINE=InnoDB;

-- Forwardings. it contains
--  - members of mail alias account
--  - per-account alias addresses
--  - per-user mail forwarding addresses
CREATE TABLE IF NOT EXISTS forwardings (
    id BIGINT(20) UNSIGNED AUTO_INCREMENT,
    address VARCHAR(255) NOT NULL DEFAULT '',
    forwarding VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    dest_domain VARCHAR(255) NOT NULL DEFAULT '',
    -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
    is_list TINYINT(1) NOT NULL DEFAULT 0,
    -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
    is_forwarding TINYINT(1) NOT NULL DEFAULT 0,
    -- defines whether it's a per-account alias address. 0=no, 1=yes.
    is_alias TINYINT(1) NOT NULL DEFAULT 0,
    active TINYINT(1) NOT NULL DEFAULT 1,
    PRIMARY KEY (id),
    UNIQUE INDEX (address, forwarding),
    INDEX (domain),
    INDEX (dest_domain),
    INDEX (is_list),
    INDEX (is_alias)
) ENGINE=InnoDB;

Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

Attention

If you're running Python 3, please download this file instead: https://github.com/iredmail/iRedMail/raw/master/tools/migrate_sql_alias_table.py, and run it with command python3 migrate_sql_alias_table.py.

cd /root/
wget https://github.com/iredmail/iRedMail/raw/1.0/tools/migrate_sql_alias_table.py
python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user vmailadmin:

/opt/www/iredadmin/settings.py (Debian/Ubuntu)
/var/www/iredadmin/settings.py (CentOS/OpenBSD)
/usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
/usr/local/www/iredadmin/settings.py (FreeBSD)

Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is /usr/local/etc/postfix/mysql.

cd /etc/postfix/mysql/
perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

Drop unused SQL columns and records in `vmail.alias` table

Warning

Make sure you have a backup of SQL database vmail.
Please also upgrade iRedAPD and iRedAdmin-Pro, they need the new SQL structure too.

After migration, vmail.alias table contains few sql columns we will never use, also old records (accounts) will cause ghost accounts if we don't remove them.

Please connect to MySQL server as MySQL root user, then execute SQL commands below:

USE vmail;

-- Remove non-mail-alias account
DELETE FROM alias WHERE islist <> 1;

-- Remove per-domain catch-all account
DELETE FROM alias WHERE address=domain;

-- Drop unused columns
ALTER TABLE alias DROP COLUMN goto;
ALTER TABLE alias DROP COLUMN moderators;
ALTER TABLE alias DROP COLUMN islist;
ALTER TABLE alias DROP COLUMN is_alias;
ALTER TABLE alias DROP COLUMN alias_to;

Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

[CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

Open the daily MySQL backup script, it's /var/vmail/backup/backup_mysql.sh by default. if you use different storage directory during iRedMail installation, you can find the base directory with command postconf virtual_mailbox_base.
Find variable name CMD_MYSQLDUMP like below:

export CMD_MYSQLDUMP="mysqldump ..."

Make sure it has argument --skip-comments like below:

export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

Save your change. That's it.

PostgreSQL backend specific

SQL structure change in `vmail.alias` SQL table

We've made some changes to vmail.alias SQL table for easier account management, this change introduces 2 new SQL tables (forwardings, alias_moderators), and (optionally) dropped few columns in vmail.alias table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Warning

Please backup SQL database vmail before you run any SQL commands below.

Create required new SQL tables

Please run shell commands below to connect to PostgreSQL server as vmailadmin user first:

su - postgres
psql -U vmailadmin -d vmail

Then execute SQL commands below to create required new tables:

CREATE TABLE alias_moderators (
    id SERIAL PRIMARY KEY,
    address VARCHAR(255) NOT NULL DEFAULT '',
    moderator VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    dest_domain VARCHAR(255) NOT NULL DEFAULT ''
);

CREATE INDEX idx_alias_moderators_address ON alias_moderators (address);
CREATE INDEX idx_alias_moderators_moderator ON alias_moderators (moderator);
CREATE UNIQUE INDEX idx_alias_moderators_address_moderator ON alias_moderators (address, moderator);
CREATE INDEX idx_alias_moderators_domain ON alias_moderators (domain);
CREATE INDEX idx_alias_moderators_dest_domain ON alias_moderators (dest_domain);

CREATE TABLE forwardings (
    id SERIAL PRIMARY KEY,
    address VARCHAR(255) NOT NULL DEFAULT '',
    forwarding VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    dest_domain VARCHAR(255) NOT NULL DEFAULT '',
    -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
    is_list INT2 NOT NULL DEFAULT 0,
    -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
    is_forwarding INT2 NOT NULL DEFAULT 0,
    -- defines whether it's a per-account alias address. 0=no, 1=yes.
    is_alias INT2 NOT NULL DEFAULT 0,
    active INT2 NOT NULL DEFAULT 1
);
CREATE INDEX idx_forwardings_address ON forwardings (address);
CREATE INDEX idx_forwardings_forwarding ON forwardings (forwarding);
CREATE UNIQUE INDEX idx_forwardings_address_forwarding ON forwardings (address, forwarding);
CREATE INDEX idx_forwardings_domain ON forwardings (domain);
CREATE INDEX idx_forwardings_dest_domain ON forwardings (dest_domain);
CREATE INDEX idx_forwardings_is_list ON forwardings (is_list);
CREATE INDEX idx_forwardings_is_forwarding ON forwardings (is_forwarding);
CREATE INDEX idx_forwardings_is_alias ON forwardings (is_alias);

-- Grant required privilege to vmail user
GRANT SELECT ON TABLE forwardings to vmail;
GRANT SELECT ON TABLE alias_moderators to vmail;

Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

cd /root/
wget https://github.com/iredmail/iRedMail/raw/1.0/tools/migrate_sql_alias_table.py
python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user vmailadmin:

/opt/www/iredadmin/settings.py (Debian/Ubuntu)
/var/www/iredadmin/settings.py (CentOS/OpenBSD)
/usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
/usr/local/www/iredadmin/settings.py (FreeBSD)

Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is /usr/local/etc/postfix/pgsql.

cd /etc/postfix/pgsql/
perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

Drop unused SQL columns in `vmail.alias` table

Warning

Make sure you have a backup of SQL database vmail.
Please also upgrade iRedAPD and iRedAdmin-Pro, they need the new SQL structure too.

After migration, few columns in vmail.alias table are not used anymore, it's ok to drop them. But it's strongly recommended to keep them for few more days until you can confirm all features are working as expected.

su - postgres
psql -d vmail

-- Remove non-mail-alias account
DELETE FROM alias WHERE islist <> 1;

-- per-domain catch-all account
DELETE FROM alias WHERE address=domain;

-- Drop unused columns
ALTER TABLE alias DROP COLUMN goto;
ALTER TABLE alias DROP COLUMN moderators;
ALTER TABLE alias DROP COLUMN islist;
ALTER TABLE alias DROP COLUMN is_alias;
ALTER TABLE alias DROP COLUMN alias_to;

Upgrade iRedMail from 0.9.6 to 0.9.7

ChangeLog

General (All backends should apply these steps)

Update /etc/iredmail-release with new iRedMail version number

Upgrade iRedAPD (Postfix policy server) to the latest stable release (2.1)

Upgrade iRedAdmin (open source edition) to the latest stable release (0.8)

Upgrade Roundcube webmail to the latest stable release (1.3.0)

Fixed: improper order of Postfix HELO restriction rules.

Fixed: incorrect owner and permission for rotated Dovecot log files

Fixed: incorrect session.save_path in php-fpm pool config file on RHEL/CentOS

Fixed: incorrect freshclam setting UpdateLogFile

Fail2ban: fixes an improper filter and add new filter rule

Fail2ban: Add new jail for Nginx

NEW: New backup script for SOGo

OpenBSD: Upgrade uwsgi to the latest 2.0.15

OpenLDAP backend special

Fixed: Avoid possible backdooring mysqldump backups

MySQL/MariaDB backend special

SQL structure change in vmail.alias SQL table

Create required new SQL tables

Migrate mail accounts

Update Postfix config files

Drop unused SQL columns and records in vmail.alias table

Fixed: Avoid possible backdooring mysqldump backups

PostgreSQL backend specific

SQL structure change in vmail.alias SQL table

Create required new SQL tables

Migrate mail accounts

Update Postfix config files

Drop unused SQL columns in vmail.alias table

Update `/etc/iredmail-release` with new iRedMail version number

Fixed: incorrect freshclam setting `UpdateLogFile`

SQL structure change in `vmail.alias` SQL table

Drop unused SQL columns and records in `vmail.alias` table

SQL structure change in `vmail.alias` SQL table

Drop unused SQL columns in `vmail.alias` table