Migrating from the Python release

This guide moves an existing ProjectWikit installation that runs the Python release to this release. pwikit takes over the existing PostgreSQL database in place: pages, history, accounts, passwords, roles and site settings stay where they are. The uploaded files are copied into pwikit's data directory.

The move needs a maintenance window. The two releases must never run against the same database at the same time.

Requirements

• The old installation runs the last Python release of ProjectWikit. This release takes over only a database with that release's layout. With a database from an earlier Python release, pwikit applies part of its database changes and then stops with an error.
• PostgreSQL 14 or newer.
• pwikit can reach the PostgreSQL server and connects as the account that owns the database, which is the account the old installation uses.
• The migrated instance holds only the old installation's site, because pwikit takes over the old database itself. Adding a Python release's site to an instance that already has sites is not supported, and restoring a backup likewise replaces the whole database; see Operations.

Check the database layout before you start. Connected to the old database with psql, run:

SELECT EXISTS (
  SELECT 1 FROM information_schema.columns
  WHERE table_schema = 'public' AND table_name = 'web_user' AND column_name = 'email_verified_at'
) AS ready;

If the result is f, stop here: this release cannot take over that database. pwikit migrate status does not replace this check, because it reports every database made by the Python release the same way.

Before you start

1. Tell your users when the site will be unavailable.

2. Stop the old release. Leave its PostgreSQL running. With the Docker Compose setup of the Python release, run in its project directory:

   docker compose stop web updater

3. Back up the database with PostgreSQL's own tools:

   pg_dump -Fc -h <host> -U <user> -d <database> -f projwikit-before-pwikit.dump

   With the Docker Compose setup, use the user and database name from DB_PG_USERNAME and DB_PG_DATABASE in its .env file (admin and projwikit by default):

   docker compose exec -T postgres pg_dump -Fc -U admin projwikit > projwikit-before-pwikit.dump

4. Keep the old files/ directory unchanged. The steps below copy from it, so it remains available for a rollback.

5. Write down the old settings: SECRET_KEY, the DB_PG_ values, the EMAIL_ values, the upload limits and GOOGLE_TAG_ID. With Docker Compose they are in .env.

6. Install pwikit as described in Deployment, but do not start pwikit serve yet.

Connect pwikit to the existing database

The Compose file of the Python release does not publish PostgreSQL's port. For pwikit on the same machine to connect, add a port mapping to the postgres service in docker-compose.yaml:

  postgres:
    ports:
      - "127.0.0.1:5432:5432"

Then recreate the container. The data stays in ./postgresql:

docker compose up -d postgres

Put the connection string into pwikit.toml in pwikit's data directory. Create the file if it does not exist yet:

database = "postgres://admin:change-me-please@127.0.0.1:5432/projwikit"

Special characters in the password must be percent-encoded. The pwikit commands that use the database, and the system service, read this setting. DATABASE_URL and -database also work, but a system service does not see environment variables set in your shell. See Configuration.

Check the database without changing it:

./pwikit migrate status

Without a database setting, DATABASE_URL or -database, this command uses the bundled PostgreSQL and creates a new, empty database in pgdata/ instead of reading yours.

A database from the Python release shows existing 0001_baseline.sql on the first line and pending on the rest:

STATUS    MIGRATION
existing  0001_baseline.sql
pending   0002_admin_log_and_addresses.sql
...

The statuses are explained in Command line.

If the first line reads pending 0001_baseline.sql instead of existing, pwikit is not looking at a database made by the Python release. Check the connection string before going on.

Apply the database changes

./pwikit migrate up

The command prints adopted 0001_baseline.sql, then one applied <name> line for each change it applies.

pwikit serve applies the same changes on start, unless it is started with -no-migrate. Running migrate up first shows the result before the site goes live.

Each change is applied completely or not at all. If one fails, pwikit stops and names it, and the changes before it remain applied. Roll back as described in Rolling back before trying again.

Note Once this release has changed the database, do not start the old release against it. To return to the old release, restore the backup taken before the move.

Carry over the secret key

The secret key signs sign-in sessions and the links sent by email, such as invitations, claim links and password resets.

• With the old SECRET_KEY, links that were sent but not yet used keep working. Signed-in users stay signed in if the old site's sign-in cookie was named pwikit_sessionid; with any other cookie name, everyone signs in again. The browser's developer tools show the cookie name on the old site.
• With a different key, everyone signs in again and links that were already sent stop working.
• If the old installation never set SECRET_KEY, or still uses the example value from .env.example, its key is publicly known. Do not carry it over. pwikit creates a new key on its first start, and everyone signs in again.

To carry the key over, write it into secrets/session-key in the data directory before starting pwikit:

mkdir -p secrets
printf '%s\n' 'old-secret-key' > secrets/session-key
chmod 600 secrets/session-key

On Windows, in PowerShell:

New-Item -ItemType Directory -Force secrets | Out-Null
Set-Content -Path secrets\session-key -Value 'old-secret-key' -Encoding ascii

The file must not start with a byte order mark, which is why the command above writes it as ASCII. pwikit reads the file on every start, so replacing a key it has already created takes effect after a restart. SECRET_KEY and -secret-key take precedence over the file. See The secrets directory.

Move the uploaded files

The Python release kept uploads in files/ in its project directory. This release reads them from files/ in its data directory:

Path	Contents
files/media/	Page attachments
files/-/	Site icons, avatars, role icons and files uploaded in the admin panel
files/theme/<site identifier>/	Theme style sheets, one directory per site

1. Copy media, - and theme from the old files/ directory. The old files/symlinks/ directory is not used by this release; leave it out.

   mkdir -p /opt/pwikit/files
   cp -a /opt/ProjectWikit/files/media /opt/ProjectWikit/files/- /opt/ProjectWikit/files/theme /opt/pwikit/files/

   On Windows, in PowerShell:

   Copy-Item -Recurse C:\ProjectWikit\files\media, C:\ProjectWikit\files\-, C:\ProjectWikit\files\theme -Destination C:\pwikit\files\

2. Move the theme style sheets into a directory named after the site identifier. pwikit site list shows it:

   ./pwikit site list
   mkdir -p files/theme/wikit-wiki
   mv files/theme/*.css files/theme/wikit-wiki/

   Alternatively, open each theme under Themes in the admin panel and save it, which writes its style sheet again.

3. Make sure the account that runs pwikit can read and write everything under files/. Files written by the containers of the Python release can belong to root:

   sudo chown -R <account>: /opt/pwikit/files

Settings

Put the settings into pwikit.toml. The environment variables of the same name are read too, but a system service does not see them.

Python release	This release
SECRET_KEY	secrets/session-key, see Carry over the secret key
DB_PG_HOST, DB_PG_PORT, DB_PG_DATABASE, DB_PG_USERNAME, DB_PG_PASSWORD	Not read. Combine them into database
EMAIL_ENGINE	[mail] engine
EMAIL_HOST	[mail] host
EMAIL_PORT	[mail] port
EMAIL_USERNAME	[mail] username
EMAIL_PASSWORD	[mail] password
EMAIL_USE_TLS	[mail] use_tls
EMAIL_DEFAULT_FROM	[mail] from
MEDIA_UPLOAD_LIMIT	[server] upload_limit
ABSOLUTE_MEDIA_UPLOAD_LIMIT	[server] storage_limit
GOOGLE_TAG_ID	[analytics] google_tag_id
WEB_PORT	No equivalent. See Domains and HTTPS
MEDIA_HOST, ARTICLE_SOURCE_LIMIT, ARTICLE_REPLACE_CONFIG, ARTICLE_IMPORT_REPLACE_CONFIG, DEBUG, LOGLEVEL, COMPOSE_PROJECT_NAME, HOST_PROJECT_DIR, UPDATE_REPO, UPDATE_BRANCH, UPDATE_POLL_INTERVAL	Not read

The mail defaults differ. Without host, pwikit writes mail into its log instead of sending it, and the default port is 587. Set host and port explicitly even if the old installation relied on its defaults. See Configuration.

Domains and HTTPS

The domain and media domain of each site are stored in the database and carry over unchanged. What changes is who handles HTTPS.

• pwikit handles HTTPS. When a site is bound to a public domain and none of listen, trusted_proxies or a TLS mode is configured, pwikit listens on ports 80 and 443 and obtains certificates from Let's Encrypt. Stop the web server that handled HTTPS for the old release so that these ports are free. See Automatic HTTPS.

• The reverse proxy stays. Set the address the proxy forwards to and the proxy's address. This turns automatic HTTPS off:

  [server]
  listen = "127.0.0.1:8000"
  trusted_proxies = ["127.0.0.1"]

  Point the proxy at that address. See Behind a reverse proxy.

Start pwikit and check the site

1. Start pwikit, in a terminal or as a system service, as described in Deployment:

   ./pwikit serve

2. Sign in and open several pages, attachments and forum threads. Check that the theme is applied.

3. In the admin panel, open Site settings and review Site language and Site time zone. The time zone starts as UTC.

4. Take a first backup with pwikit: ./pwikit backup create. See Operations. pwikit's backups do not include secrets/, so keep a copy of that directory as well.

Administrators

• The admin interface of the Python release is replaced by the admin panel, at the same address /-/admin/. See Site administration.
• Superusers remain superusers. Members of a role with Can enter the admin panel can still open it.
• Existing passwords keep working.
• To give an account every right, run ./pwikit admin grant -name "<name>". To take them back, run ./pwikit admin revoke -name "<name>". See Command line.

Rolling back

1. Stop pwikit.

2. Restore the database from the dump into an empty database:

   dropdb -h <host> -U <user> <database>
   createdb -h <host> -U <user> <database>
   pg_restore -h <host> -U <user> -d <database> projwikit-before-pwikit.dump

   With the Docker Compose setup:

   docker compose exec -T postgres dropdb -U admin projwikit
   docker compose exec -T postgres createdb -U admin projwikit
   docker compose exec -T postgres pg_restore -U admin -d projwikit < projwikit-before-pwikit.dump

3. The old files/ directory was only copied from, so it is unchanged.

4. Start the old release. With Docker Compose: docker compose up -d.

Everything that changed in pwikit after the move, such as new pages, accounts and uploads, is lost.