Essun

Essun

Essun is the mediaserver of Calafou local network. It is running Debian, the Universal Operating System. It is providing services for the people who are on the territory of Calafou. It is 8Gb ram and has 4 discs in it: 160Gb for system, 1Tb mirror discs, 3T for mediateca. You can connect here http://essun.calafou.org/

What does "Essun" mean?

Essun is one of the names of the protagonist of the Broken Earth trilogy by N. K. Jemisin. Essun belongs to a small group of people, called origenes, who have a special power to shape and redirect energy. Origenes are feared and hated by their society because of their powers, so if they are not murdered as children, they are carefully controlled, enslaved and exploited.

As the world ends, Essun is forced to join together with others. As she learns the strength of community and solidarity, she learns to use her powers without shame. And she learns that resistance is possible, even if it means causing the destruction of the world that had previously been her home.

Boot sequence: What happens when you turn on the machine?

1. BIOS
   1. Here you can press F12 to choose boot device. The default is to boot the system on the hard drive. This is useful if you want to boot a rescue or install image from a pendrive, for example.
   2. Here you can press F2 to enter the BIOS configuration, but we have to find out the BIOS password. At the moment, we have no idea what is the BIOS password! We have to search how to reset this password...
2. GRUB
   1. Here you can choose to start the system as usual, or to enter rescue mode. It is also possible to modify the kernel boot parameters. This is useful if you forgot the root password and want to reset it, for example.
3. Initramfs
   1. Here cryptsetup asks for the LUKS encryption password. The password can be typed in using a keyboard connected to essun, or through SSH if your SSH keys are installed in initramfs (in /etc/dropbear-initramfs/authorized_keys cf this. For the second option, use the unlock-essun.sh script. The actual password can be found in the calafou keyringer repository, and you can see it from your computer like this: keyringer calafou decrypt lan.essun.luks.system
   2. To connect with ssh to ssh dropbear server at boot :

ssh root@172.31.17.23

once in busybox, type :

cryptroot-unlock

and type the luks passphrase

1. Systemd
   1. The hard drives ("block devices") specified in /etc/crypttab are unlocked, usually using key files in /etc/luks/.
   2. The hard drives ("block devices") specified in /etc/fstab are mounted, such as /mnt/3t (a big hard drive for mediateca) and /mnt/mirror (a mirrored drive for services).
   3. Systemd manages the services, so when the computer starts, it tries to turn on the web server (apache2), etc.
2. System
   1. You can log in if your SSH key is installed in the system: ssh $USERNAME@essun.calafou.org (where $USERNAME is your username).
3. Services to re-launch : (don't know why they doesn't start well at boot time)

systemctl restart etherpad-lite.service
systemctl restart apache2.service
systemctl status omnioussonification.service

1. Container to start (as root)

lxc-start umap
lxc-start bibretime
lxc-ls -f

Services

Most of these services are only available from the local network!

Web Server

http://essun.calafou.org/

systemctl status apache2

/etc/apache2/sites-available/essun.calafou.org

The web server shows a page that explains Essum and links to the other services.

Pads

http://etherpad.calafou.org

systemctl status etherpad

/opt/etherpad-lite/settings.json

The etherpad is providing real-time multi-user editable shared persistent documents.

A few plug-ins have been installed from this framasoft list (all did not instal ...) https://github.com/YunoHost-Apps/etherpad_mypads_ynh

Streaming

http://essun.calafou.org:8000

systemctl status icecast2

/etc/icecast2/icecast.xml

Icecast2 is providing a stream ("radio") that you can play on your computer, for example with mplayer or mpv:

mplayer http://essun.calafou.org:8000

mpv http://essun.calafou.org:8000

Libros

Los libros en la antigua mediateca se pueden visualizar a traves de calibre-web.

Se pueden subir libros desde el enlace anterior o a traves del directorio mediateca en nuvol.

El indice y los metadatos de los libros se actualizan cada noche.

Peliculas, Musica, Fotos...

Las peliculas, musica, fotos, etc.. se pueden visualizar a traves de jellyfin (calaflix).

File server

Se puede acceder a tanto a los ficheros de la antigua mediateca a traves de nuvol

NextCloud

We could try to install it like this:

https://docs.nextcloud.com/server/17/admin_manual/installation/source_installation.html#example-installation-on-ubuntu-18-04-lts-server

We tried with this : https://www.howtoforge.com/tutorial/how-to-install-nextcloud-on-debian-10/ it seems to work (change the version of nextcloud to download)

More Nextcloud config

• To make the visio application Talk work : you have to use https for the website (see below) but it is limited to 5-6 persons at the same time, see https://github.com/nextcloud/spreed/issues/2515#issuecomment-598808719
• To make the ONLY OFFICE application work : you have to change the url > https of the ONLY OFFICE server in the settings (as an admin of the nextcloud)
• To integrate external storage as part of Nextcloud files
  • Possible to use the "Local Storage" application : activate it > then see as an admin in the settings the local storage
• To allow people to put file with anonymous FTP + via Nextcloud : https://serverfault.com/questions/184548/ftp-and-apache-permission-issues or more recently https://serverfault.com/questions/938143/proftpd-and-www-data-group

So it was :

groupadd www-ftp
usermod -a -G www-ftp www-data
usermod -a -G www-ftp anonftp
chown -R :www-ftp /mnt/3t/ftp/MEDIATECA
chmod -R g+rw /mnt/3t/ftp/MEDIATECA

Now it is config on proftpd side :

DefaultRoot /mnt/3t/ftp/MEDIATECA/
<Anonymous /mnt/3t/ftp/MEDIATECA/>
<Limit LOGIN>
    AllowAll
</Limit>
Group www-data
User www-data
UserAlias anonymous anonftp www-data
GroupOwner www-data
UserOwner www-data
Umask 007
<Directory *>
    <Limit ALL>
        AllowAll
    </Limit>
</Directory>
</Anonymous>

and on MEDIATECA side

chmod -R 0775 MEDIATECA/ 
chmod g+w+s MEDIATECA/
chown -R www-data: MEDIATECA/

Even more config to optimize things

As seen in https://nuvol.calafou.org/index.php/settings/admin/overview there were things that needed to be optimized :

cd /var/www/html/nextcloud
sudo -u www-data php occ db:add-missing-indices
apt install php-intl php-imagick
sudo -u www-data php occ maintenance:mode --on
sudo -u www-data php occ db:convert-filecache-bigint
sudo -u www-data php occ maintenance:mode --off

Activate cache

apt install php-apcu
phpenmod intl
nano config/config.php

Add this in the file before );

'memcache.local' => '\OC\Memcache\APCu',

Restart Apache after all this

systemctl restart apache2

Rescan the files

sudo -u www-data php occ files:scan --all -v 

If problem while scanning with unlocked files :

mysql -u root -p
connect nextclouddb
DELETE FROM oc_file_locks WHERE 1;

or more wild

TRUNCATE TABLE oc_file_locks;

As of 07082021, if "Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)" then :

 nano /etc/php/7.3/mods-available/apcu.ini

add this line :

 apc.enable_cli=1

restart apache2

 systemctl restart apache2

then this command shoud work

 root@essun:/var/www/html/nextcloud# sudo -u www-data php occ db:add-missing-indices

cf https://github.com/nextcloud/vm/issues/2039#issuecomment-876833425

Set https certificate on each subdomain.calafou.org with a certbot wildcard

One certificate for all the subdomains

• Certbot wildcard tab : https://certbot.eff.org/lets-encrypt/debianbuster-apache
• Plug in Gandi for automatic renew https://github.com/obynio/certbot-plugin-gandi
• How to get the Gandi API key https://doc.livedns.gandi.net/

Install Gandi plug-in

apt remove python-cryptography
pip install cryptography
pip install certbot-plugin-gandi
certbot certonly -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials gandi.ini -d \*.calafou.org --server https://acme-v02.api.letsencrypt.org/directory

Activate mod ssl for Apache

a2enmod ssl
systemctl restart apache2

Create the virtual hosts :

<IfModule mod_ssl.c>
<VirtualHost *:443>
       ServerName essun.calafou.org
       ServerAdmin webmaster@localhost
       DocumentRoot /var/www/essun.calafou
       ErrorLog ${APACHE_LOG_DIR}/essun-error.log
       CustomLog ${APACHE_LOG_DIR}/essun-access.log combined
      SSLEngine on
      SSLProxyEngine On
      SSLProxyVerify none
      SSLProxyCheckPeerCN off
      SSLProxyCheckPeerName off
      SSLProxyCheckPeerExpire off
SSLCertificateFile /etc/letsencrypt/live/calafou.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/calafou.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</<IfModule mod_ssl.c>

Renew auto

crontab -e
0 0 * * 0 certbot renew -q -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /etc/letsencrypt/gandi/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory

New method 2022 to create a certificate for a subdomain (as the previous one seems to cause trouble while renewing)

certbot certonly -d subdomain.calafou.org -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /etc/letsencrypt/gandi/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory

LXC container por Libretime y umap

About Libretime et Umap

Installation of the containers

• https://wiki.debian.org/LXC
• https://archives.flockport.com/lxc-networking-guide/

Use of lxcbr0 bridge to allow autostart

Create a lxc-net default config file :

sudo nano /etc/default/lxc-net

USE_LXC_BRIDGE="true"
LXC_BRIDGE="lxcbr0"
LXC_ADDR="10.0.3.1"
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="10.0.3.0/24"
LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
LXC_DHCP_MAX="253"

Restart lxc network

 systemctl restart lxc-net.service

Adapt the config of each container

nano /mnt/mirror/lxc/bibretime/config

# Network configuration
lxc.net.0.type = veth
lxc.net.0.hwaddr = 00:16:3e:cf:c1:dd
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up

# Autostart
lxc.start.delay = 10
lxc.start.auto = 1

Use the container

list containers

lxc-ls -f

start

lxc-start bibretime

stop

lxc-stop bibretime

attach to a container to have a root prompt

lxc-attach bibretime

• https://wiki.debian.org/LXC#Networking

Onduleur Salicru SPS ONE

• https://www.salicru.com/fr/onduleurs/sps-one.html
• Official Software : https://www.salicru.com/files/software/13/installviewpowerhtml_linux_text_x86_64.tar.gz

The software provided is ugly (tomcat java), so let's use NUT to replace it, see :

• https://www.jormc.es/2014/05/11/raspi-conectando-un-sai-salicru-sps-one-900va/comment-page-1/
• http://2tazasdelinux.blogspot.com/2015/11/moniitorizando-nuestro-sai-con-nut.html
• https://blog.shadypixel.com/monitoring-a-ups-with-nut-on-debian-or-ubuntu-linux/

Essun backup

Essun backup is being stored gpg-encrypted in nusse.

The software used is https://github.com/sukria/Backup-Manager/

We created the user `backup` both in essun and nusse to create/store the backups.

The user backup in essun has a gpg keyring with a public key.

The private key it is not in the servers and it's needed only to decrypt the backups.

There's only 1 configuration file, `/etc/backup-manager.conf`.

These are the changed lines:

   export BM_REPOSITORY_ROOT="/mnt/mirror/backup"
   export BM_REPOSITORY_USER="backup"
   export BM_REPOSITORY_GROUP="backup"
   export BM_ARCHIVE_METHOD="tarball-incremental"
   export BM_ENCRYPTION_METHOD="gpg"
   export BM_ENCRYPTION_RECIPIENT="264128C3E9B4F451C0105757350C60B958D846EC"
   export BM_TARBALL_DIRECTORIES="/mnt/mirror/calafou_actual"
   export BM_TARBALL_BLACKLIST="/mnt/mirror/backup"
   export BM_UPLOAD_METHOD="ssh-gpg"
   export BM_UPLOAD_HOSTS="172.31.17.202"
   export BM_UPLOAD_DESTINATION="/mnt/backup1T/backup"
   export BM_UPLOAD_SSH_USER="backup"
   export BM_UPLOAD_SSH_KEY="/mnt/mirror/backup/.ssh/id_ed25519"
   export BM_UPLOAD_SSH_HOSTS="172.31.17.202"
   export BM_UPLOAD_SSH_PORT="22"
   export BM_UPLOAD_SSH_DESTINATION="/mnt/backup1T/backup"
   export BM_UPLOAD_SSH_TTL="23"
   export BM_UPLOAD_SSHGPG_RECIPIENT="264128C3E9B4F451C0105757350C60B958D846EC"

It runs as often as configured in `/etc/cron.d/backup`

Also a pre-script is run to mount /var/www on the piweb server as a directory /mnt/ynh via sshfs to backup also the var/www folder on this small machine. For this a backupmirror user was created on the pi/yunohost machine to allow sshfs from essun via an authorized_key indicated in /etc/fstab in Essun