Advanced installation of a Raspberry Pi with Raspbian Bullseye

When installing a Raspberry Pi, I have a checklist of steps I take each time to ensure my Raspberry Pi's are (mostly) configured in the same way. They have the same way to backup their data, use the same user configurations (ntp, syslog, sendmail...) and have the same security provisioning. We will also introduce logs into memory with Log2Ram, to avoid too much SD card writing/wearing, which will eventually break your RPi. Feel free to comment on any step that is documented here. Some steps might be optional or unnecessary in your case.

1. Do the physical installation, plugin the network and HDMI cables (except the power cable of course) and screw your RPi into a cover or box.
2. Prepare SD card on Mac with Raspberry Pi Imager
3. Plugin the SD card into your RPi and now also plugin the power cable. Boot your RPi for the first time now. Create a user with password for using later. (e.g. user:pi, password:raspberry)
4. When booted, you'll be provided with a prompt to login for the first time. Mind the QWERTY keyboard layout.
5. Run the setup tool
   

   sudo raspi-config

6. Configure the setup tool
1. Set the hostname (1 System Options > S4 Hostname)
2. Expand Filesystem (6 Advanced Options > A1 Expand file system)
3. Change Timezone, set Keyboard Layout (if needed) and change Wifi Country (5 Localization Options > L2 Change Timezone, L3 Change Keyboard Layout, L4 Change Wi-fi Country)
4. Enable SSH (3 Interfacing Options > I2 SSH)
5. Press 'Finish' and Reboot
7. After reboot, login again via SSH and change your user password:
   passwd
8. Generate a SSH key-gen pair, which is more robust than the default one.
   

   ssh-keygen -o -a 100 -t ed25519

9. Change the root password
   

   sudo passwd root

10. Set the ETH0 IP address to a fixed IP. I hardly ever use the Wifi module in a Raspberry Pi
    

    sudo vi /etc/network/interfaces
    

    Add at the end of the file the following:
    

    # Added by user on 2023-XX-XX
    auto eth0
    iface eth0 inet static
            address 192.168.0.240/24
            network 192.168.0.0
            broadcast 192.168.0.255
            gateway 192.168.0.1
            dns-nameservers 192.168.0.1 8.8.8.8
    # End of Addition

    sudo systemctl restart networking.service

    And test with
    

    ip add show

    Reboot your RPi again (or do it later if you plan to reboot anyway)
    
11. Check for updates & upgrades for Bullseye, but first become root. Don't forget to reboot if kernel patches were installed.
    

    sudo -i
    apt-get update -y && apt-get upgrade -y

12. Fix a common issue with Syslog flooding your logs
    

    sudo sed -i '/# The named pipe \/dev\/xconsole/,$d' /etc/rsyslog.conf
    sudo service rsyslog restart

13. Alternatively, you could also install Syslog-NG
    

    sudo apt-get install -y syslog-ng

14. Install Git
    

    sudo apt-get install -y git dirmngr

15. Install Log2Ram as this will allow us to keep logs in memory and reduce the SD card writing significantly. From time to time, the logs are still made persistent to disk.
    

    cd /home/pi
    git clone https://github.com/azlux/log2ram.git
    cd log2ram
    chmod +x install.sh
    sudo ./install.sh
    Change the log size value to 128M
    sudo vi /etc/log2ram.conf

    Reboot
16. Install Sendmail and configure to work with a local mail relay server, or alternatively Gmail.
    

    sudo apt-get install -y sendmail mailutils sendmail-bin
    sudo mkdir -m 700 /etc/mail/authinfo/
    sudo cd /etc/mail/authinfo/

    Create a Sendmail authentication file:
    

    sudo vi sendmail-auth

    And paste the following info:
    

    AuthInfo: "U:root" "I:YOUR LOGIN" "P:YOUR PASSWORD"

    Save and exit vi. Next do the makemap:
    

    sudo makemap hash sendmail-auth < sendmail-auth
    sudo chmod 400 sendmail-auth

    Change the Sendmail configuration now
    

    sudo vi /etc/mail/sendmail.mc
    Add the following below right above first "MAILER_DEFINITIONS" line:
    # Added by yourname on 2018-XX-XX
    define(`SMART_HOST',`[192.168.Y.XX]')dnl
    define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
    define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
    define(`confAUTH_OPTIONS', `A p')dnl
    TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
    define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
    FEATURE(`authinfo',`hash -o /etc/mail/authinfo/sendmail-auth.db')dnl
    # End of Addition

    Apply the changes to the configuration and restart Sendmail:
    

    sudo make -C /etc/mail
    sudo /etc/init.d/sendmail reload

    Test if you can send an email to yourself:
    

    echo "Just testing my Sendmail email relay" | mail -s "Sendmail email relay" you@here.com

17. Setup NTP sync
    

    sudo apt-get install -y ntp ntpdate
    sudo vi /etc/ntp.conf

    And replace the XX with your country code
    

    0.XX.pool.ntp.org
    sudo /etc/init.d/ntp stop

    And query to see NTP being in sync
    

    sudo ntpd -gq
    sudo /etc/init.d/ntp start
    sudo ntpd -pn

18. Setup SNMP
    

    sudo apt-get install snmp snmpd
    sudo vi /etc/snmp/snmpd.conf

    And put the following configuration lines
    

    agentAddress udp:161
    rocommunity public 192.168.X.0/24

    Restart your SNMP daemon
    

    sudo /etc/init.d/snmpd restart

    And test on your local machine
    

    snmpwalk -Os -c public -v 1 localhost

19. Setup NFS backup share, install a backup tool, rsnapshot and configure
    Fix rpcbind issue (Make yourself root first)
    

    su -
    cat >/etc/systemd/system/nfs-common.service <<\EOF
    [Unit]
    Description=NFS Common daemons
    Wants=remote-fs-pre.target
    DefaultDependencies=no
    
    [Service]
    Type=oneshot
    RemainAfterExit=yes
    ExecStart=/etc/init.d/nfs-common start
    ExecStop=/etc/init.d/nfs-common stop
    
    [Install]
    WantedBy=sysinit.target
    EOF

    
    

    cat >/etc/systemd/system/rpcbind.service <<\EOF
    [Unit]
    Description=RPC bind portmap service
    After=systemd-tmpfiles-setup.service
    Wants=remote-fs-pre.target
    Before=remote-fs-pre.target
    DefaultDependencies=no
    
    [Service]
    ExecStart=/sbin/rpcbind -f -w
    KillMode=process
    Restart=on-failure
    
    [Install]
    WantedBy=sysinit.target
    Alias=portmap
    EOF

    
    

    cat >/etc/tmpfiles.d/rpcbind.conf <<\EOF
    #Type Path        Mode UID  GID  Age Argument
    d     /run/rpcbind 0755 root root - -
    f     /run/rpcbind/rpcbind.xdr 0600 root root - -
    f     /run/rpcbind/portmap.xdr 0600 root root - -
    EOF
    
    systemctl enable rpcbind.service
    systemctl enable nfs-common 

    Install raspiBackup  (from this website)
    

    sudo mkdir -p /backup 

    Avoid accidental file storage, when folder is not mounted
    And put the following configuration lines
    

    sudo chattr +i /backup
    sudo vi /etc/fstab 

    And add
    

    server.yourdomain.com:/volume1/backups/host.yourdomain.com/backup      nfs     rsize=8912,wsize=8912,timeo=14     0       0
    sudo mount /backup

    Now install the raspiBackup tool
    

    curl -s https://raw.githubusercontent.com/framps/raspiBackup/master/installation/install.sh | sudo bash

    Go through the configuration tool, later on you can go back to it via: raspiBackupInstallUI.sh
    -Backup versions: smart strategy
    -Backup to tar
    -No compression
    -Backup mode standard
    -Email notification set
    Uncomment the crontab (backup will run every Sunday at 5am):
    sudo vi /etc/cron.d/raspiBackup 
    And finally test
    sudo raspiBackup

20. Generate an SSH keypair for easy login
    ssh-keygen
    ssh-copy-id -p 22 admin@server.yourdomain.com 
    Log into your server, make yourself root and copy the public key into the raspberry

    cat /root/.ssh/id_rsa.pub | ssh user@hhost.yourdomain.com "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys" 

    Test if it's working by using:

    ssh user@host.yourdomain.com 

21. Setup unattended upgrade based on this tutorial
    sudo apt update
    sudo apt install unattended-upgrades 
    Configure unattended upgrades and uncomment:

    sudo vi /etc/apt/apt.conf.d/50unattended-upgrades
    
    "origin=Debian,codename=${distro_codename}-updates";
    "origin=Debian,codename=${distro_codename}-proposed-updates";
    "origin=Debian,codename=${distro_codename},label=Debian";
    "origin=Debian,codename=${distro_codename},label=Debian-Security";
    "origin=Debian,codename=${distro_codename}-security,label=Debian-Security"; 

    And uncomment:
    

    Unattended-Upgrade::Remove-Unused-Dependencies "false";

    Now enable Automatic Updates (and press Yes)
    

    sudo dpkg-reconfigure --priority=low unattended-upgrades

    To view the unattended upgrades:
    sudo systemctl status unattended-upgrades.service
    -

Installing LibreNMS Docker image on Synology

At some point in time when you start having many devices in your home network, and you want to have some insight in things like bandwidth consumption, whether devices are alive, memory usage is ok... You have the need for a monitoring tool. I went for LibreNMS as it seemed user friendly with lots of features that are handy for a low-effort setup like auto-discovery. I went for the docker image from here. There are some excellent blogpost from other bloggers on how to setup this, but I ran into a few issues that I want to share with you. One issue that is still unresolved is running the web UI with https.

1. Login to your NAS via SSH and make yourself root
2. Follow the regular step by creating an APP_KEY in base64
3. Create the MySQL user and database

CREATE DATABASE librenms;
CREATE USER 'librenms'@'%' IDENTIFIED BY 'yourpassword';
CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'yourpassword';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'%';
FLUSH PRIVILEGES;

4. I could only start the container with the following command which does not make use of the UID, GID and SSL config. I choose TCP port 30080 as my Synology is already serving other things over TCP port 80.

docker run \
 -d \
 -h librenms \
 -p 30080:80 \
 -e APP_KEY="base64:your-app-key" \
 -e DB_HOST=192.168.0.Y \
 -e DB_NAME=librenms \
 -e DB_USER=librenms \
    -e TZ=Europe/Amsterdam \
 -e DB_PASS=yourpassword \
 -e BASE_URL=http://192.168.0.X:30080 \
    -e POLLERS=16 \
 -v /volume1/docker/librenms/logs:/opt/librenms/logs \
 -v /volume1/docker/librenms/rrd:/opt/librenms/rrd \
 -v /volume1/docker/librenms/ssl:/opt/librenms/ssl:ro \
 --name librenms \
 jarischaefer/docker-librenms

5. When the container is started, from the command line you can monitor the logs from the container.
6. Find the container ID

docker ps

7. Monitor the logs using the container ID

docker logs container-id

8. I did see these errors, which seems related to this Docker setup issue explained at Stackoverflow.. As the Docker setup on Synology is somewhat custom and probably prone to errors when you change something, I decided to ignore this.

setfacl: /opt/librenms/bootstrap/cache: Operation not supported
setfacl: /opt/librenms/logs: Operation not supported
setfacl: /opt/librenms/rrd: Operation not supported
...

9. I also had to remove the UID and GID from the Docker command as this was giving me the following error:

failed to acquire lock schema

10. So the posted command to launch the Docker container under step 4 is the one that worked for me.
11. Next, setup the database, create an admin account for your user and login to the container to edit the configuration.

docker exec librenms setup_database
docker exec librenms create_admin
docker exec librenms php /opt/librenms/adduser.php admin admin 10 you@here.com
docker exec -it librenms bash

Edit the LibreNMS config:

vi /opt/librenms/conf.d/custom.php

Set the SNMP community tag:

$config['snmp']['community'][] = "public";
$config['nets'][] = '192.168.0.0/24';
$config['discovery_by_ip'] = true;

Scan for SNMP devices:

/opt/librenms/snmp-scan.py -r 192.168.0.0/24

12. Normally, you should be able now to login into the web UI on http://192.168.0.X:30080. (note the default username/password for LibreNMS is admin:admin)
13. The next thing I want to do is to figure out how Nginx can help serve LibreNMS over HTTPS with a reverse proxy. The current issue is the fact that base_url is being used as an environmental variable, which is causing all URLs to be always rewritten back to HTTP. No, this is not the expected behaviour. :-)

Make your Raspberry Pi read-only for reducing SD wear/corruption

Everyone using Raspberry Pi's knows that SD cards are actually not made to serve as storage, especially when storage intensive applications (e.g. database, log server...) are being put in-there. To limit this SD wearing (which will lead to SD  corruption at some point), you can put Raspian in a read-only mode and only swith to read-write when really needed.

All kudos go to this blog for the excellent write up. I assume you have an RPi3 with Wheezy installed onto it. All commands are executed as root.

Update your RPi

Make yourself root, update your Raspian and reboot.

su -
apt-get update && apt-get upgrade
reboot

Remove some unwanted packages

apt-get remove --purge wolfram-engine triggerhappy anacron dphys-swapfile xserver-common lightdm
insserv -r x11-common; apt-get autoremove --purge

Replace the Rsyslog with the Busybox one

apt-get install busybox-syslogd; dpkg --purge rsyslog

Disable swap and filesystem check and set it to read-only

Edit the file: /boot/cmdline.txt and add the three words
fastboot noswap ro

Move some system files to the temp filesystem

rm -rf /var/lib/dhcp/ /var/run /var/spool /var/lock /etc/resolv.conf
ln -s /tmp /var/lib/dhcp
ln -s /tmp /var/run
ln -s /tmp /var/spool
ln -s /tmp /var/lock
touch /tmp/dhcpcd.resolv.conf; ln -s /tmp/dhcpcd.resolv.conf /etc/resolv.conf

Change the dhcpd lock file to the temp filesystem

vi /etc/systemd/system/dhcpcd5

And be sure to change the line with PIDFile=/run/dhcpcd.pid to PIDFile=/var/run/dhcpcd.pid

On Debian Jessie and Wheezy move random-seed to a writable location

rm /var/lib/systemd/random-seed
ln -s /tmp/random-seed /var/lib/systemd/random-seed

Since this file is on tmpfs, it will not be created upon reboot, but we can still do this with some magic of the systemd system service.
To create file on the tmp area at bootup before starting the random-seed service, just edit the file service file to add a pre-command to execute:

vi /lib/systemd/system/systemd-random-seed.service

Add the line: ExecStartPre=/bin/echo "" >/tmp/random-seed under the service section.

Do not use touch instead of echo, it won’t work because we'll be checking a read-only filesystem.
Execute the following to tell systemd we made changes.

systemctl daemon-reload

Setup the Internet clock sync

If (still) needed install NTP

apt-get install ntp

And be sure to configure your time zone, with raspi-config tool.

raspi-config

Then go to menu "Internationalisation Options" and change "Change Timezone" and select your time zone.

Edit the hourly cron script that saves the clock every hour

vi /etc/cron.hourly/fake-hwclock

And change it to allow saving the clock.

#!/bin/sh
#
# Simple cron script - save the current clock periodically in case of
# a power failure or other crash

if (command -v fake-hwclock >/dev/null 2>&1) ; then
  mount -o remount,rw /
  fake-hwclock save
  mount -o remount,ro /
fi

Edit the file /etc/ntp.conf and set to redirect driftfile to the writable zone /var/tmp

vi /etc/ntp.conf

Remove some startup scripts and edit fstab

insserv -r bootlogs; insserv -r console-setup

Edit fstab and add the ro option. Add the tmpfs parts as well.

vi /etc/fstab

tmpfs           /tmp            tmpfs   nosuid,nodev            0       0
tmpfs           /var/log        tmpfs   nosuid,nodev            0       0
tmpfs           /var/tmp        tmpfs   nosuid,nodev            0       0

Reboot

reboot

If all went fine, you're pi will be up again. Test if the filesystem is read-only now.

Switching from read-only mode to read-write and back

Now you’re in read-only mode, it’s fine and safe, but if you need to install, write or modify files, upgrade, or whatever that need write access, you'll need to be able to do this.

To set system to read-write:

mount -o remount,rw /

And to set it back to read-only:

mount -o remount,ro /

If you want to have two simple commands like: ro for setting mode to read-only and: rw to enable read-write mode. I also want to know on which mode I am in, on the command prompt.

Add fancy indicating features

Edit the file bash.bashrc

vi /etc/bash.bashrc

At the end add the following lines:

# set variable identifying the filesystem you work in (used in the prompt below)
set_bash_prompt(){
    fs_mode=$(mount | sed -n -e "s/^\/dev\/.* on \/ .*(\(r[w|o]\).*/\1/p")
    PS1='\[\033[01;32m\]\u@\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
}

alias ro='sudo mount -o remount,ro / ; sudo mount -o remount,ro /boot'
alias rw='sudo mount -o remount,rw / ; sudo mount -o remount,rw /boot'

# setup fancy prompt"
PROMPT_COMMAND=set_bash_prompt

Execute this new file and look at the magic! The prompt has changed and shows the file system mode.

. /etc/bash.bashrc

Test by typing ro or rw to switch between modes.

Use logout to save history and force read-only mode

To be sure to avoid setting back to read-only at logout, add the following line to the file
/etc/bash.bash_logout. (maybe you'll need to create it)

vi /etc/bash.bash_logout

Add:

mount -o remount,rw /
history -a
fake-hwclock save
mount -o remount,ro /
mount -o remount,ro /boot

Installing Crashplan docker image to Xpenology or Synology

Following the excellent tutorial by Mike Tabor, the only difference to my setup, was the need to change my local config for this location: C:\Users\username\AppData\Local\CrashPlan\.ui_info instead of the mentioned one.

Building custom Linux kernel (howto)

Just summarizing what I did to build my own custom Debian Linux kernel. As I needed a few modules which where not shipped in the standard Debian kernel. All is based on this tutorial which I ran on a Debian 8.6 with 3.16 kernel. Make sure you're root.

mkdir /root/custom-kernel
cd /root/custom-kernel
apt-get install fakeroot linux-source-3.16 kernel-package libncurses5-dev
tar xf /usr/src/linux-source-3.16.tar.xz
cd linux-source-3.16
make menuconfig
make-kpkg clean
fakeroot make-kpkg --initrd --revision=001
dpkg -i linux-image-3.16.39_001_i386.deb
shutdown -r now