More Notes to the Future

Instead of talking about image analysis, this post will document the setup of a multiuser multisite ELN based on WordPress. Lots of the hard work was done by Steve Royle and detailed on his blog here and here. I found his technical post immeasurably helpful so to pay it forward, I wanted to share the details of our setup.

Apologies to anyone not dealing with the sysadmin side of things. You may want to skip this one.

Background

You can read Steve’s posts (links above) and see the video of a talk I gave covering most of the salient points here (slide deck here).

If you want to see the final product in action, here is a video demonstrating to people how to make their first post.

All set? On with the show! I’m assuming a fair degree of comfort using GNU/Linux and bash and that you’re operating on a firewalled network. Remember, if you want to play along, building a Virtual Machine is a great choice. For most VM applications, I use VirtualBox.

Operating System

Pretty much all my servers run Ubuntu 16.04 LTS and this is supported until April 2019 so that’s a no-brainer.

Download and install the server version, pick all the defaults and make a new user (we’ll call them ‘user‘) with a strong password. You’ll be asked for a hostname so we’ll use eln.

The only thing I include during the install is an OpenSSH server as I almost exclusively work remotely.

You should have received updates during install but just in case run the following to update your cache and all your existing packages.

user@eln:~$ sudo apt-get update && sudo apt-get upgrade

You can also install some helper applications:

user@eln:~$ sudo apt-get install -y zip unzip wget screen tree git

LAMP Stack

Apache2

Linux WordPress runs on a LAMP stack (Linux, Apache2, MySQL, PHP), so let’s install those. Apache first:

user@eln:~$ sudo apt-get install -y apache2

We also need to make sure that the apache rewrite engine is enabled (this is how wordpress is going to deal with multisite):

user@eln:~$ sudo a2enmod rewrite

Finally, edit the config file to add the servername and enable URL rewriting :

user@eln:~$ sudo nano /etc/apache2/apache2.conf

And add the following line at the bottom (if you don’t have a domain address, just add the IP):

ServerName eln.example.com

In the same file find the Directory root and make sure overrides are allowed:

<Directory /var/www/>
 Options Indexes FollowSymLinks
 AllowOverride All
 Require all granted
</Directory>

MySQL

When you install MySQL you’ll be asked for a root password. Make sure it’s strong!

user@eln:~$ sudo apt-get install -y mysql-server

Optionally run

mysql_secure_installation

Which will let you set the following (my choices in parentheses):

  • Password Validation (no)
  • Root password (no – already set strong)
  • Prevent anonymous users (yes)
  • Limit root access to localhost (yes)
  • Remove test database (yes)

PHP

As well as my HTML preprocesser, I’m going to be using LDAP for authentication and some plugins, so install the following PHP packages:

user@eln:~$ sudo apt-get install -y php libapache2-mod-php php-mcrypt php-mysql php-gd php-ssh2 php-ldap php-mbstring

Now move .php files up in the load level (for example, so that index.php is loaded before index.html)

user@eln:~$ sudo nano /etc/apache2/mods-enabled/dir.conf
<IfModule mod_dir.c>
 DirectoryIndex index.php index.html index.cgi index.pl index.xhtml index.htm
</IfModule>

The upload limits for WordPress are largely controlled via PHP so if you want to allow larger files to be uploaded you need to set two values in /etc/php/7.0/cli/php.ini

upload_max_filesize = 25M
post_max_size = 25M

 Setup the Database

WordPress stores almost everything in the database so we need to create it and also make a wordpress user to access that DB. The red bits are the bits you should change.

user@eln:~$ mysql -u root -p
mysql> create database wp_db;
mysql> CREATE USER wp_user@localhost IDENTIFIED BY 'strongPasswordHere';
mysql> GRANT ALL PRIVILEGES ON wp_db.* TO wp_user@localhost;
mysql> quit

Download WordPress Files

user@eln:~$ cd
user@eln:~$ wget http://wordpress.org/latest.tar.gz
user@eln:~$ tar xzvf latest.tar.gz

Make the WebRoot writeable to our user and copy the files over

user@eln:~$ sudo chown -R user /var/www/html
user@eln:~$ rm /var/www/html/*
user@eln:~$ rsync -avP wordpress/ /var/www/html

Now reboot your machine.

user@eln:~$ sudo reboot

Configure WordPress

At this point you will need to navigate to the master WordPress config file. In our case we can get there with

user@eln:~$ sudo nano /var/www/html/wp-config.php

And you’ll need to fill in the MySQL details you created earlier:

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'wp_db');

/** MySQL database username */
define('DB_USER', 'wp_user');

/** MySQL database password */
define('DB_PASSWORD', 'strongPasswordHere');

/** MySQL hostname */
define('DB_HOST', 'localhost');

You should also navigate down and fill in the random seed salts from this site. This will give you something that looks like this (don’t use these!):

define('AUTH_KEY', 'sTJL&;eg{ux$T[.F 2sHp$$>%DUigd$HTgXOj+-l2I]&Xynf/48)fRaB*rfqk/be');
define('SECURE_AUTH_KEY', 'i;A|,n2n@2}w|Z heeTRf7=Ar?-[E%r:j}12xA_pt&*_z|IkL}>`)uK(>7Q%<+s ');
define('LOGGED_IN_KEY', 'PMrQR%<Ai@|E:-^L-y<5N|O^86N+J%rZby]<xmE_e_LOSD<,}]K]]T;_`d!RrV/`');
define('NONCE_KEY', '3xW:U|#`7gHj=u+yFJ0Wj0JGF;9d<Uu[68!DpG-B^(a|7{!FaHj[fZLebhBKMrs ');
define('AUTH_SALT', 't<EPOsQw0o8dHXk=zee5uu:+3%rY|7*w]^;3_TY|::RV-3jYiOt*pgKAds93C9~,');
define('SECURE_AUTH_SALT', '}m zJ,7kzGAfbSA/#^Z)|&z[T{$}4wGb*XkZN&E7Qf{|+rjcdfQCui>,oa[0j;Uu');
define('LOGGED_IN_SALT', 'zFXmC@|&9u5Nc_3w#vE<rZy5%?PBhdk:QT_oj@h4)a,g:+y>6]aHw8#.T;@7hgby');
define('NONCE_SALT', 'js3e0}v6ps},)99dWf%FgYX+/)I7ozN/+k*Yn.{-h=w,:.|aut+0<E)0`osq`LB6');

Save the file.

Install WordPress

Make sure your web user has access to the files (note, we’ll sort out more secure permissions and access later):

user@eln:~$ sudo chown -R www-data:www-data /var/www/html

Use a browser to navigate to your Web Document Root and check the config details.

You will also be asked to name your site (we’ll just call it ELN), and to make an admin account (we’ll go with superadmin). This will become our network admin.

Aside: Firewalls and connection problems

If you can’t get to the config page, It’s likely a web server issue. First things first check your server is running:

user@eln:~$ sudo service apache2 status

If that’s fine, there may be a firewall issue. You can check the status of your firewall by running.

user@eln:~$ sudo ufw status verbose
To                         Action      From
--                         ------      ----
22                         ALLOW IN    Anywhere

Notice in this example, we’re not allowing anything to access port 80 (HTTP) or 443 (HTTPS). Let’s correct that by adding a rule:

user@eln:~$ sudo ufw allow proto tcp from any to any port 80,443

This will give access to those two ports from any IP address. At this point, I would highly recommend limiting access to a subset of IP addresses. Lots more info on the ufw man page, but for example:

user@eln:~$ sudo ufw allow proto tcp from 192.168.0.0/16 to any port 80,443

will only allow access to addresses in the range 192.168.0.0 -> 192.168.255.255. You can use this to lock down access to your institutional range, but don’t forget to include any other subdomains (like VPN, wireless, eduroam &c) as needed.

WordPress!

At this point you should be able to login with your superadmin account and have access to a wordpress blog. Congratulations!

Making it Multisite

Now that we have a basic working WordPress, we want to make it multisite so that we can have multiple independent ELNs. To do this, go back to the wp_config.php file in the root and add the following before the line about not editing any more.

/* Multisite */
define( 'WP_ALLOW_MULTISITE', true );

Save the file but keep it open as we’ll need to add some more lines.

Back in your browser, refresh, and navigate to your dashboard and Tools > Network Setup. Here I chose subdirectories (partly because I don’t know if we have wildcard DNS records). I also called the network ELN, but if you’re feeling more original, call it whatever you want.

This will give you guidelines about adding lines to the config, and to update any existing .htaccess files in the wordpress root (this will deal with redirecting the browser to each of the subsites). Make these updates then login again.

That’s it (sort of)! You can navigate the network via the admin bar, which will now have a ‘My Sites’ link:

Themes and Plugins

Themes

I forked a wordpress theme from Steve and made some changes of my own. Largely the changes were:

  • Remove the post ‘snippet’ from the listings. Usually everything before the “Read More” tag is shown, but I didn’t want people to have to remember to add this.
  • Change where and what is displayed at the top of a list of posts and in a post (IE, no tags in the former and the author added to the latter).
  • Removed post status and visibility options from Publish box: don’t want people getting confused about Public vs Private posts.
  • Added some nice formatting to various header elements (h1 & h2)

WordPress comes bundled with the latest theme and this is also the default, but we don’t want this, so navigate to the themes directory, clone the theme into a child directory, and set the permissions:

user@eln:~$ cd /var/www/html/wp-content/themes
user@eln:/var/www/html/wp-content/themes$ git clone https://github.com/dnmason/gitsta.git gitsta-child
user@eln:/var/www/html/wp-content/themes$ sudo chown www-data gitsta-child

We should also edit the <code>wp-config.php</code> file again and set this theme to be the default for new sites. Do this by adding the following line note the position just above the settings line.

define( 'WP_DEFAULT_THEME', 'gitsta-child' );

Our base site, is still going to be using twenty-seventeen or whatever, so in your browser, go back to the site list, select appearance and switch the theme to gitsta-child. You can also now delete the other themes on the network.

Plugins

A lot of the functionality we’re looking to get from WordPress as an ELN come from plugins. In rough order of importance to the ELN, I have:

  • My Private Site (David Gewirtz): The only two options for which I use this are to prevent viewing of the site unless you’re logged in and to make sure people can only view the site if they’re on the user list (this doesn’t affect their ability to login for the first time using LDAP).

  • Capability Manager Enhanced (Jordi Canals et al.): The key to controlling who can do what. Through this plugin, I set up two working roles, neither can delete, but Author can write and edit posts. manage tags and assign categories to posts. Editors can do the same but also can also manage categories and edit pages. Broadly, it’s something like this:

There seem to be some conflicting options however, as sometimes when the settings are made, they will revert. Not clear, but empirically checking the permissions, it behaves as wanted.

  • Next Active Directory Integration (NeosIT GmbH): This is the plugin I settled for for LDAP authentication. Once you figure it out, it actually works really well for a multisite install. The way I have it setup is to make a profile with all the settings set, and the options being either invisible or uneditable by the blog admin except the permissions page (below).

So now each site on the network can have a different security group (thus can individually control access). One other nice thing is that this security group can be used to allocate new users registered through LDAP into the ‘Author’ role using Role Equivalent Groups. This saves a lot of headaches trying to wrangle new users into place. Editors are still set manually by the superadmin (but there are rarely more than one or two of these).

Also installed the following which don’t really take any setup but add nice functionality.

  • Authors Widget (Gavriel Fleischer)
  • Easy Footnotes (Jason Yingling)
  • MathJax-LaTeX (Phillip Lord et al)
  • Simple Content Templates (Clifton Griffin)

The other plugins can be deleted or deactivated if not needed.

Security and Permissions

From the server side, it’s important to set the right file/directory permissions. The aim of the permissions is to have

  • Our user account (user) may read and modify files
  • WordPress (via apache) may read and modify our scripts
  • WordPress may create, modify or delete files and folders (so that uploads work)
  • No one else can see the database credentials in <code>wp-config.php</code>

We do this by issuing:

user@eln:/var/www/html$ sudo chown -R user:www-data ../html
user@eln:/var/www/html$ sudo find . -type f -exec chmod 664 {} +
user@eln:/var/www/html$ sudo find . -type d -exec chmod 775 {} +
user@eln:/var/www/html$ sudo chmod 660 wp-config.php

When I need to do updates or install new plugins through the web interface, I temporarily give the apache user ownership then revert it when I’m done updating.

user@eln:/var/www/html$ sudo chown -R www-data ../html
... do your update in the browser ...
user@eln:/var/www/html$ sudo chown -R user ../html

Adding a New Site

When setting up a new notebook, these are the things that need doing:

  • Go to Network Admin > Sites > Add New Site. Give it a name, and put the superadmin email address into the admin box.
  • Enable Private Site on the new ELN
  • Add the security group to the LDAP extension page and make sure that the role equivalent group is set to make group members ‘Author’
  • Adjust the appearance as necessary including any custom widgets (I have Authors, Calendar, Recent Posts, Categories & Archives on the sidebar)
  • Delete the default post and page that come with a new site (looking into ways to do this automatically)
  • Add a static welcome page is requested with some guidelines about ELN usage
  • Update commenting preferences so that comments do not require approval.

Backups

The astute among you will have noticed an auspicious lack of backup plug-ins in the list above. Backups are absolutely critical, but I’m a firm believer on being reliant on as little as possible when planning backup strategy.

WordPress has two main parts: the files (all kept at /var/www/html) and the SQL database. I like to keep backups of both so my basic backup strategy is to compress the files in the webroot into a zip file and to dump the contents of the SQL database into a file daily. These are kept for two weeks after which only one backup per week are kept (see script below).

The backups are copied to remote file servers and mirrored, so at any time, I have a live and three copies of every day for the last fortnight and weekly before that.

#!/bin/bash
#-- Script to backup wordpress database and files
#-- Cron this to run daily
DATE=$(date "+%Y%m%d_%H%M%S")
OUT_DIR=/srv/eln-backup

#-- backup files
zip -qr ${OUT_DIR}/${DATE}-wp-files.zip /var/www

#-- dump the database
#-- Password is stored in ~/.my.cnf so -p flag is not required at the command line
DB="wp_db"
DBUSER="wp_user"

mysqldump -u ${DBUSER} $DB > ${OUT_DIR}/${DATE}-wp-db.sql

#-- Tidy up the archives directory by deleting everything older than two weeks except for those created on a Monday
find $OUT_DIR -type f -mtime +14 -exec sh -c 'test $(date +%u -r "$1") = 1 ||rm "$1"' -- {} \;

#-- sync the backups to another server (ssh key-based login must be setup)
rsync -a --delete $OUT_DIR/ user@remote:/remote/location

With both of those things, a full copy of the ELN can be recovered in less than an hour. It’s worth noting that some of the entries in the database are hardcoded with the hostname of the blog, so if you’re recovering to a different machine than the original, you will need to search and replace the IP addresses or domain names using a tool like SRDB. Don’t forget to also update the:

  1. ServerName property in the apache2.conf
  2. The value in the multisite settings of the <code>wp-config.php</code> (otherwise the Network Admin will not be available

Talking points and final thoughts

Below are some things to think about when installing / running a wordpress ELN.

More admin rights for group leaders?

The way this is setup is with one superadmin who deals with pretty much all the site admin, themes, plugins, updates etc. I can see some value to devolving admin rights to group leaders, but there’s also something to be said for offering a consistent experience for end users. If people are going to go to the superadmin with troubleshooting, it’s much easier to deal with problems if you know the environment.

Category Management and the super-Author

It seems to vary between groups as to whether group leaders want group members to be able to manage (IE create) categories. I see this as something that is not created that often to deal with individual projects (which are about the only thing that are not categorised by tags (although they could be), authors or dates).

There’s always the option of promoting group members to a third role that can manage categories but not edit pages. Pretty trivial to do.

Working groups?

How do you deal with cross-lab working groups? The same idea applies to visiting students who you may not want to give full access to the existing ELN (although I’m not sure why). This is fairly easily solved by just creating a new site on the network, but then how do you integrate that into the existing ELN once they leave?

More space for uploads

On smaller servers or virtual machines, the uploads may start to be take up all your space.  The uploaded files are stored at:

./wp-content/uploads

The simplest solution if you can’t expand the size of your disk is to symlink another drive or network share into this directory. As long as you don’t break the location, you can keep adding space.

Exporting notebooks

There are really two situations where you might want to export notebooks. Here’s how we can deal with them:

  1. Group Leader moving to a different institution
  2. Researcher changing jobs

If in either of these cases the person wants to set up another ELN, they can (yay open source!), then the relevant parts of the database can be exported and imported again into another multisite or a single site wordpress.

A great alternative I found, that also works for project students who need to show their lab notebooks to someone else, is Kalin’s PDF Creation Station. This plugin has not been updated in ages, but still works brilliantly. You can select users, categories or dates and export posts as a fully hyperlinked PDF, with title page, table of contents and links back to the original ELN.

The only small downside is that by default, it’s enabled for admins only, but because we have the code, this can be altered by editing the file

./wp-content/plugins/kalins-pdf-creation-station/kalins-pdf-creation-station.php

And in the function:

kalins_pdf_configure_pages()
   Change
$kPDFtoolPage = add_submenu_page('tools.php', 'Kalins PDF Creation Station', 'PDF Creation Station', 'manage_options', 'kalins-pdf-tool', 'kalins_pdf_tool_page');
   To
$kPDFtoolPage = add_submenu_page('tools.php', 'Kalins PDF Creation Station', 'PDF Creation Station', 'edit_pages', 'kalins-pdf-tool', 'kalins_pdf_tool_page');

This gives anyone with the editor role (actually the ‘edit_pages’ capability) the option to export PDFs.

UPDATE: When I presented this, the question was asked whether comments could also be exported (for student reports, there needs to be some indication that the supervisor has ‘signed-off’ on the work). While this doesn’t happen by default, within the PDF Creation Options, you can add the following to the “HTML to insert after every post” box:

[post_comments before="<h1>Comments ([comment_count])</h1>" after=""]

When you export (if there are any comments) you get something like this in the PDF:

2017-10-17.png

Better searching

For blog posts, the default wordpress search is OK. For information rich notebooks though, the search is somewhat lacking. Especially as a whole bunch of experiments are likely to have the same tags, category and author.

I’m still looking for an advanced search that will allow you to filter by category, tag, date, author and keywords.

Future Expansion

There are dozens of reasons why an ELN is superior to paper notes. It will be interesting to see how scalable is the platform in its current guise. Clearly wordpress has no problem scaling to millions of sites.

Advertisements

Comment!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.