Trying to get Plone working on OpenBSD (part deux) - SUCCESS! (update: well, mostly)

Years ago I fought with trying to get Plone to work on OpenBSD, to no avail. Recently I was able to get Plone 4.3 working on OpenBSD 6.0, but not Plone 5.0.6.. I was finally able to get Plone 5.0.7 Unified Installer working on OpenBSD 6.1!

UPDATE 20170925:

While the install works, and I can create new plone subsites, and edit their front pages, copy and edit themes, when I try to add new content such as a new page, the Plone instance completely restarts!

I haven't had time to fire up the logs and debug the issue yet. And probably won't until after the current ZOE projects wraps up. But when I do have time, I will update this page accordingly.

Then I hope to clean up this page to be more readable for someone trying to follow the correct steps. For now it is showing all of my trial and error (some of it some pretty dumb things I tried and/or missed, but hopefully helps others).

Once the bugs are worked out and it is operating fully. I will clean up this page, and then submit it for consideration to add to the docs.plone.org site, so that others can have the joys of Plone 5 on openBSD 6!

I am so glad to be getting Plone working on OpenBSD!

-Hawke

==================================

So, progress is finally being made to get Plone working on OpenBSD, but there is significant version lag.

I was able to, with some effort (but not as egregious as previous years), get 4.3 working on OpenBSD 6.0.

But 4.x is being steadily phased out, so need to get Plone 5 working.

Unfortunately I ran into along series of unresolvable depndency issues with Plone 5.0.6 on OpenBSD 6.0.

Going to try again now with Plone 5.0.7 on OpenBSD 6.1 and see if any better progress, and share the information here.

UPDATE 20170914

Now trying Plone 5.0.7 (latest currently) on OpenBSD 6.1. Problem with libjpeg header path not found even though installed.

Testing /usr/local/bin/python2.7 for Zope/Plone requirements....
/usr/local/bin/python2.7 looks OK. We will use it.

Unable to find libjpeg library and headers. These are required to build Plone. Please use your system package or port manager to install libjpeg dev. (Debian/Ubuntu libjpeg-dev) Exiting now.
#

So, need to manually add path variables:


# export CPPFLAGS="-I/usr/local/include $CPPFLAGS"
# export LDFLAGS="-L/usr/local/lib $LDFLAGS"
# env
LOGNAME=root
HOME=/root
CPPFLAGS=-I/usr/local/include 
LDFLAGS=-L/usr/local/lib 
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/sbin:/usr/local/bin
SHELL=/bin/sh
TERM=xterm-256color
USER=root
# echo $LDFLAGS
-L/usr/local/lib
# echo $CPPFLAGS
-I/usr/local/include
# ./install.sh zeocluster --target="/arc/plone/5" --password="<password>" --clients=2

Now, next issue is difference in scripting in syntax how users are added with useradd, groupadd, etc. My be a Bash versus Sh issue, but will try to get this working with native OpenBSD Sh instead of install Bash agian. Last time I swapped all to bash to work, the next upgrade I couldn't login since the upgrade swapped things back but didn't update the user variables, and I had to rescue the system from the console. As detailed here.

esting /usr/local/bin/python2.7 for Zope/Plone requirements....
/usr/local/bin/python2.7 looks OK. We will use it.

Root install method chosen. Will install for use by users:
ZEO & Client Daemons:      plone_daemon
Code Resources & buildout: plone_buildout

Detailed installation log being written to /arc/downloads/Plone-5.0.7-UnifiedInstaller/install.log
Installing Plone 5.0.7 at /arc/plone/5

Using useradd and groupadd to create users and groups.
plone_group already exists; no need to create it.
User plone_daemon already exists. No need to create it.
Adding user plone_daemon to group plone_group.
usage: usermod [-moUvZ] [-c comment] [-d home-directory] [-e expiry-time]
               [-f inactive-time] [-G secondary-group[,group,...]]
               [-g gid | name | =uid] [-L login-class] [-l new-login]
               [-p password] [-S secondary-group[,group,...]]
               [-s shell] [-u uid] user
User plone_buildout already exists. No need to create it.
Adding user plone_buildout to group plone_group.
usage: usermod [-moUvZ] [-c comment] [-d home-directory] [-e expiry-time]
               [-f inactive-time] [-G secondary-group[,group,...]]
               [-g gid | name | =uid] [-L login-class] [-l new-login]
               [-p password] [-S secondary-group[,group,...]]
               [-s shell] [-u uid] user
Creating Python virtual environment.
New python executable in /arc/plone/5/zeocluster/bin/python2.7
Also creating executable in /arc/plone/5/zeocluster/bin/python
ERROR: The executable /arc/plone/5/zeocluster/bin/python2.7 could not be run: [Errno 13] Permission denied
Installing zc.buildout in virtual environment.
Unable to install zc.buildout in virtual environment.

Installation has failed.
See the detailed installation log at /arc/downloads/Plone-5.0.7-UnifiedInstaller/install.log
to determine the cause.

While the default Plone Unified Installer script is able to create the plone_group group, it does not have matching syntax for creating users and putting them in the right groups for OpenBSD. You cold hack the install script in ./Plone-5.0.7-UnifiedInstaller/helper_scripts/user_group_utilities.sh

Or more quickly create the users yourself. If I skimmed it correctly the script wants that they should be nologin nohome users. So something like this should address that:

# useradd -d /nonexistent -s /sbin/nologin -g plone_group plone_daemon
# useradd -d /nonexistent -s /sbin/nologin -g plone_group plone_buildout
Verify the users now exist in /etc/passwd

You should see something like:

plone_daemon:*:1001:1001::/nonexistent:/sbin/nologin
plone_buildout:*:1002:1001::/nonexistent:/sbin/nologin

And check if they have been added to the /etc/group listing for group plone_group

If not listed, then add them manually to correct so that line looks like this:

plone_group:*:1001:plone_daemon,plone_buildout

Now, though it still has auser/group add error, they are there, and should work. And looking at <install_dir>/zeocluster the permission appear to match ownership as normal.

Now the next issue, ../zeocluster/bin/python2.7 could not be run: [Error 13] Permission denied.

This is because pip is missing. I forgot to get the other dependencies in place.

from the install.log error log:

"Detailed installation log
Starting at Thu Sep 14 16:50:36 PDT 2017
helper_scripts/main_install_script.sh: line 831: /arc/plone/5/zeocluster/bin/pip: No such file or directory
"

Other dependencies to consider installing:

pkg_add readline

pkg_add py-pip (for python PIP)

pkg_add libzip (is this the same as libz?)

libexpat now in xbase?

pkg_add openssl

pkg_add py-lxml (for libxml2 and libxslt)

pkg_add py-Pillow (for Python Pillow)

Can't use --build-python with openbsd:

--build-python

helper_scripts/main_install_script.sh: line 484: $'echo\n***Aborting***\nSorry, but the Unified Installer cannot build a Python 2.7 for OpenBSD.\nThere are too many platform-specific patches required.\nPlease consider installing the Python 2.7 port and re-run installer.\n': command not found

So try with --with-python=/usr/local/bin/python2.7

./install.sh zeocluster --target="/arc/plone/5" --password="<password>" --clients=2 --with-python=/usr/local/bin/python2.7

Detailed installation log
Starting at Thu Sep 14 19:34:03 PDT 2017
helper_scripts/main_install_script.sh: line 831: /arc/plone/5/zeocluster/bin/pip: No such file or directory

Manually edited line 831, remmed out original: #"${PY_HOME}/bin/pip" install "$PKG"/zc.buildout* >> "$INSTALL_LOG" 2>&1

and changed to: "/usr/local/bin/pip" install "$PKG"/zc.buildout* >> "$INSTALL_LOG" 2>&1

helper_scripts/main_install_script.sh

Also need to add plone_group and related users to /etc/sudoers list to run python:

# visudo

Move to bottom of file and added:

# Plone 5 install
%plone_group ALL=/arc/plone/5/zeocluster/bin/python

Still not working:

Creating Python virtual environment.
New python executable in /arc/plone/5/zeocluster/bin/python2.7
Also creating executable in /arc/plone/5/zeocluster/bin/python
ERROR: The executable /arc/plone/5/zeocluster/bin/python2.7 could not be run: [Errno 13] Permission denied
Installing zc.buildout in virtual environment.
Unpacking buildout cache to /arc/plone/5/buildout-cache

So tried this (temporarily for install, don't want to leave it this way):

# Plone 5 install
#%plone_group   ALL=/arc/plone/5/zeocluster/bin/python
%plone_group    ALL=(ALL) NOPASSWD: SETENV: ALL
plone_buildout ALL=(ALL) NOPASSWD: SETENV: ALL
plone_daemon    ALL=(ALL) NOPASSWD: SETENV: ALL

But still:

ERROR: The executable /arc/plone/5/zeocluster/bin/python2.7 could not be run: [Errno 13] Permission denied

#################### i THINK I FOUND IT: As of OpenBSD 6.0 mandatory W^X enforcement is implemented.#########

https://unix.stackexchange.com/questions/308179/mark-binaries-writable-and-executable-in-openbsd

Now I realize that I can do this on /arc, because that is not a separate partition from root, so moving to /home/arc instead

Still:

Creating Python virtual environment.
New python executable in /home/arc/plone/5/zeocluster/bin/python2.7
Also creating executable in /home/arc/plone/5/zeocluster/bin/python
ERROR: The executable /home/arc/plone/5/zeocluster/bin/python2.7 could not be run: [Errno 13] Permission denied
Installing zc.buildout in virtual environment.
Unpacking buildout cache to /home/arc/plone/5/buildout-cache

Checking fstab mount options for /home ....

cat /etc/fstab
0248eed6db75d0dd.b none swap sw
0248eed6db75d0dd.a / ffs rw 1 1
0248eed6db75d0dd.k /home ffs rw,nodev,nosuid 1 2
0248eed6db75d0dd.d /tmp ffs rw,nodev,nosuid 1 2
0248eed6db75d0dd.f /usr ffs rw,nodev 1 2
0248eed6db75d0dd.g /usr/X11R6 ffs rw,nodev 1 2
0248eed6db75d0dd.h /usr/local ffs rw,wxallowed,nodev 1 2
0248eed6db75d0dd.j /usr/obj ffs rw,nodev,nosuid 1 2
0248eed6db75d0dd.i /usr/src ffs rw,nodev,nosuid 1 2
0248eed6db75d0dd.e /var ffs rw,nodev,nosuid 1 2

Sure enough, /home does not have the wxallowed option. Probably not a good idea to enable this for /home, but for the sake of testing, I will give it a shot. Really I should setup a separate partition instead I bet. I could install to appropriate /usr/local, but alas only 8 GB partition. Does OpenBSD 6 support dynamic partition resizing (doubt it). Okay, lets go ahead with /usr/local instead of mucking around with /home inappropriately. And then see about resizing /home to be smaller, and either /usr/local larger, or creating a new /arc or /opt partition that is much larger than the currently small 8gb defaults from obsd default install partitioning.

So install parameters now:

./install.sh zeocluster --target="/usr/local/plone/5" --password="<password>" --clients=2

For resizing partitions, this info should be useful:

http://scie.nti.st/2013/3/4/how-to-resize-an-openbsd-root-partition/

https://www.openbsd.org/faq/faq14.html

##################

progress, but still another error:

Creating Python virtual environment.
New python executable in /usr/local/plone/5/zeocluster/bin/python2.7
Also creating executable in /usr/local/plone/5/zeocluster/bin/python
Installing setuptools, pip, wheel...done.
Installing zc.buildout in virtual environment.
Unpacking buildout cache to /usr/local/plone/5/buildout-cache
Copying Plone-docs
Copying buildout skeleton
Building Zope/Plone; this takes a while...
Buildout returned an error code: 127; Aborting.
Buildout failed. Unable to continue

Installation has failed.
See the detailed installation log at /home/arc/downloads/Plone-5.0.7-UnifiedInstaller/install.log
to determine the cause.

Detailed installation log
Starting at Sat Sep 16 07:31:35 PDT 2017
Processing ./packages/zc.buildout-2.5.3.tar.gz
Requirement already satisfied (use --upgrade to upgrade): zc.buildout==2.5.3 from file:///home/arc/downloads/Plone-5.0.7-Unifie
dInstaller/packages/zc.buildout-2.5.3.tar.gz in /usr/local/lib/python2.7/site-packages
Requirement already satisfied: setuptools>=8.0 in /usr/local/lib/python2.7/site-packages (from zc.buildout==2.5.3)
/bin/sh: /usr/local/plone/5/zeocluster/bin/buildout: not found

Needed to manually create /usr/local/plone/5 dirs, and set ownership for plone/5 to plone_buildout:plone_group, then run unified installer.

Woot! Installation completed:

Testing /usr/local/bin/python2.7 for Zope/Plone requirements....
/usr/local/bin/python2.7 looks OK. We will use it.

Root install method chosen. Will install for use by users:
ZEO & Client Daemons:      plone_daemon
Code Resources & buildout: plone_buildout

Detailed installation log being written to /home/arc/downloads/Plone-5.0.7-UnifiedInstaller/install.log
Installing Plone 5.0.7 at /usr/local/plone/5

Using useradd and groupadd to create users and groups.
plone_group already exists; no need to create it.
User plone_daemon already exists. No need to create it.
Adding user plone_daemon to group plone_group.
usage: usermod [-moUvZ] [-c comment] [-d home-directory] [-e expiry-time]
               [-f inactive-time] [-G secondary-group[,group,...]]
               [-g gid | name | =uid] [-L login-class] [-l new-login]
               [-p password] [-S secondary-group[,group,...]]
               [-s shell] [-u uid] user
User plone_buildout already exists. No need to create it.
Adding user plone_buildout to group plone_group.
usage: usermod [-moUvZ] [-c comment] [-d home-directory] [-e expiry-time]
               [-f inactive-time] [-G secondary-group[,group,...]]
               [-g gid | name | =uid] [-L login-class] [-l new-login]
               [-p password] [-S secondary-group[,group,...]]
               [-s shell] [-u uid] user
Creating Python virtual environment.
New python executable in /usr/local/plone/5/zeocluster/bin/python2.7
Also creating executable in /usr/local/plone/5/zeocluster/bin/python
Installing setuptools, pip, wheel...done.
Installing zc.buildout in virtual environment.
Unpacking buildout cache to /usr/local/plone/5/buildout-cache
Copying Plone-docs
Copying buildout skeleton
Building Zope/Plone; this takes a while...
Buildout completed

#####################################################################

###################### Installation Complete ######################

Plone successfully installed at /usr/local/plone/5
See /usr/local/plone/5/zeocluster/README.html
for startup instructions.

Use the account information below to log into the Zope Management Interface
The account has full 'Manager' privileges.

Username: admin
Password: <password>

This account is created when the object database is initialized. If you change
the password later (which you should!), you'll need to use the new password.

Use this account only to create Plone sites and initial users. Do not use it
for routine login or maintenance.- If you need help, ask in IRC channel #plone on irc.freenode.net. - The live support channel also exists at http://plone.org/chat - You can also ask for help on https://community.plone.org - Submit feedback and report errors at https://github.com/plone/Products.CMFPlone/issues (For install problems, https://github.com/plone/Installers-UnifiedInstaller/issues)

Started successfully, and able to view and create plone site.

So, the above needs cleanup, but now Plone 5 works on OpenBsd 6.1.

Installing Plons 4.3.15 on Openbsd 6.1

cd ../Plone-4.3.15-UnifiedInstaller/

./install.sh --target=/usr/local/plone/4.x/ --with-python=/usr/local/bin/python --password=<password> zeo

Creating python virtual environment, no site packages, no setuptools.
New python executable in /usr/local/plone/4.x/Python-2.7/bin/python
Installing pip, wheel...done.
Compiling and installing jpeg local libraries ...
Unpacking buildout cache to /usr/local/plone/4.x/buildout-cache
Setting /usr/local/plone/4.x ownership to plone_buildout:plone_group
Copying buildout skeleton
Fixing up bin/buildout
Building Zope/Plone; this takes a while...
Buildout completed

#####################################################################
###################### Installation Complete ######################

Plone successfully installed at /usr/local/plone/4.x
See /usr/local/plone/4.x/zeocluster/README.html
for startup instructions

Use the account information below to log into the Zope Management Interface
The account has full 'Manager' privileges.

Username: <adminusername>
Password: <password>

This account is created when the object database is initialized. If you change
the password later (which you should!), you'll need to use the new password.

Use this account only to create Plone sites and initial users. Do not use it
for routine login or maintenance.

- If you need help, ask the mailing lists or #plone on irc.freenode.net.
- The live support channel also exists at http://plone.org/chat
- You can read/post to the lists via http://plone.org/forums

- Submit feedback and report errors at http://dev.plone.org/plone
(For install problems, specify component "Installer (Unified)")

vi startcluster.scr
#!/bin/sh
sudo -u plone_daemon /usr/local/plone/4.x/zeocluster/bin/plonectl start

vi stopcluster.scr
#!/bin/sh
sudo -u plone_daemon /usr/local/plone/4.x/zeocluster/bin/plonectl stop

vi restartcluster.scr
#!/bin/sh
sudo -u plone_daemon /usr/local/plone/4.x/zeocluster/bin/plonectl restart

./startcluster.scr
zeoserver: . .
daemon process started, pid=89926
client1: . .
daemon process started, pid=14245
client2: . . .
daemon process started, pid=81377

Depending on configuration, may also need to open the firewall port for the instance. Default is 8080, I setup for 4040 in this instance.