Fulcrum on a RaspiBlitz

FAQ
- Do I need to stop Electrs?
- Database corrupted
Automated setup
Docker setup
- Generate TLS credentials
Sources:

This is a rough overview, the guide is a work in progress.

Tested environments:

Raspberry Pi4 8GB 64bit RaspberryOS with SSD and ZRAM
First sync took 48h. Can expect 2 - 2.5 days.
Raspberry Pi4 4GB 64bit RaspberryOS with SSD and 10GB ZRAM
First sync took 3 days.
See RPi-specific settings under heading “Create a config file”.
FAQ

Do I need to stop Electrs?
Don’t really need to, Electrs (and also Fulcrum) are very light once synched. Chugging through the 450GB transaction history poses the challenge for the RPi. Best is to stop all services you don’t use, but testing is valuable in any circumstance.
Database corrupted

Received an error like:

FATAL: Caught exception: It appears that Fulcrum was forcefully killed in the middle of committing a block to the db.
We cannot figure out where exactly in the update process Fulcrum was killed, so we cannot undo the inconsistent state caused by the unexpected shutdown. Sorry!
The database has been corrupted. Please delete the datadir and resynch to bitcoind.

Delete the database and restart Fulcrum to start the sync from scratch

sudo rm -rf /mnt/hdd/app-storage/fulcrum/db
sudo systemctl restart fulcrum

Automated setup

Issue: https://github.com/rootzoll/raspiblitz/issues/2924
PR: https://github.com/rootzoll/raspiblitz/pull/2966
- Install script https://github.com/rootzoll/raspiblitz/blob/dev/home.admin/config.scripts/bonus.fulcrum.sh : ```
  download
  
  wget -O bonus.fulcrum.sh https://raw.githubusercontent.com/rootzoll/raspiblitz/dev/home.admin/config.scripts/bonus.fulcrum.sh
check

cat bonus.fulcrum.sh

run with debug

bash -x bonus.fulcrum.sh on ```
if running RaspiBlitz v1.7.2 and above the script is part of the image:
```
config.scripts/bonus.fulcrum.sh on
```
Display the Tor address
```
sudo cat /mnt/hdd/tor/fulcrum/hostname
```
Connect
TCP port 50021
SSL 50022
the IP address is your Raspiblitz IP
consider using Zerotier or Tailscale for remote connection though clearnet

Monitor

```
sudo journalctl -fu fulcrum
```
Restart

sudo systemctl restart fulcrum

Edit the config

sudo nano /home/fulcrum/.fulcrum/fulcrum.conf

Manual setup

Prepare bitcoind

To avoid errors like

503 (content): Work queue depth exceeded

set in the /mnt/hdd/bitcoin/bitcoin.conf:

txindex=1
whitelist=download@127.0.0.1
rpcworkqueue=512
rpcthreads=128
zmqpubhashblock=tcp://0.0.0.0:8433

Restart bitcoind
```
sudo systemctl bitcoind restart
```
If the txindex was not built before WAIT until it finishes (takes ~7 hours).
Monitor the bitcoin debug.log).
```
sudo tail -n 100 -f /mnt/hdd/bitcoin/debug.log | grep txindex
```

Prepare the system and directories

# Create a dedicated user
sudo adduser --disabled-password --gecos "" fulcrum
cd /home/fulcrum

# sudo -u fulcrum git clone https://github.com/cculianu/Fulcrum
# cd fulcrum

# Install dependencies
# sudo apt install -y libzmq3-dev
sudo apt install -y libssl-dev # was needed on Debian Bullseye

# Set the platform
if [ $(uname -m) = "aarch64" ]; then
  build="arm64-linux"
elif [ $(uname -m) = "x86_64" ]; then
  build="x86_64-linux-ub16"
fi

# Download the prebuilt binary
sudo -u fulcrum wget https://github.com/cculianu/Fulcrum/releases/download/v1.7.0/Fulcrum-1.7.0-${build}.tar.gz
sudo -u fulcrum  wget https://github.com/cculianu/Fulcrum/releases/download/v1.7.0/Fulcrum-1.7.0-${build}.tar.gz.asc
sudo -u fulcrum  wget https://github.com/cculianu/Fulcrum/releases/download/v1.7.0/Fulcrum-1.7.0-${build}.tar.gz.sha256sum

# Verify
# Get the PGP key
curl https://raw.githubusercontent.com/Electron-Cash/keys-n-hashes/master/pubkeys/calinkey.txt | sudo -u fulcrum gpg --import

# Look for 'Good signature'
sudo -u fulcrum  gpg --verify Fulcrum-1.7.0-${build}.tar.gz.asc

# Look for 'OK'
sudo -u fulcrum sha256sum -c Fulcrum-1.7.0-${build}.tar.gz.sha256sum

# Decompress
sudo -u fulcrum tar -xvf Fulcrum-1.7.0-${build}.tar.gz

# Create the database directory in /mnt/hdd/app-storage (on the disk)
sudo mkdir -p /mnt/hdd/app-storage/fulcrum/db
sudo chown -R fulcrum:fulcrum /mnt/hdd/app-storage/fulcrum

# Create a symlink to /home/fulcrum/.fulcrum
sudo ln -s /mnt/hdd/app-storage/fulcrum /home/fulcrum/.fulcrum
sudo chown -R fulcrum:fulcrum /home/fulcrum/.fulcrum

Create a config file

https://github.com/cculianu/Fulcrum/blob/master/doc/fulcrum-example-config.conf

Can paste the this as a block to create the config file, but fill in the PASSWORD_B (Bitcoin Core RPC password):

PASSWORD_B="your-password-here"

echo "\
datadir = /home/fulcrum/.fulcrum/db
bitcoind = 127.0.0.1:8332
rpcuser = ${RPC_USER}
rpcpassword = ${PASSWORD_B}

# RPi optimizations
# avoid 'bitcoind request timed out'
bitcoind_timeout = 300
# reduce load (4 cores only)
bitcoind_clients = 1
worker_threads = 1
db_mem=1024
# for 4GB RAM
db_max_open_files=200
fast-sync = 1024
# server connections
# disable peer discovery and public server options
peering = false
announce = false
tcp = 0.0.0.0:50021
# ssl via nginx
" | sudo -u fulcrum tee /home/fulcrum/.fulcrum/fulcrum.conf

The ports 50021 and 50022 are used to not interfere with a possible Electrs or ElectrumX instance.
Note the different settings for 4 and 8 GB RAM
Edit afterwards with sudo nano /home/fulcrum/.fulcrum/fulcrum.conf

Create a systemd service

Can paste this as a block to create the fulcrum.service file:

echo "\
[Unit]
Description=Fulcrum
After=network.target bitcoind.service

[Service]
ExecStart=/home/fulcrum/Fulcrum-1.7.0-${build}/Fulcrum /home/fulcrum/.fulcrum/fulcrum.conf
User=fulcrum
LimitNOFILE=8192
TimeoutStopSec=30min
Restart=on-failure

[Install]
WantedBy=multi-user.target
" | sudo tee /etc/systemd/system/fulcrum.service

Start

Depending on the available RAM it is a good idea to keep at least 10GB swap: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-debian-10 can consider ZRAM:
https://haydenjames.io/raspberry-pi-performance-add-zram-kernel-parameters/
https://github.com/rootzoll/raspiblitz/issues/2905
```
sudo systemctl enable fulcrum
sudo systemctl start fulcrum
```

Monitor

sudo journalctl -fu fulcrum
sudo systemctl status fulcrum

Open the firewall

sudo ufw allow 50021 comment 'Fulcrum TCP'
sudo ufw allow 50022 comment 'Fulcrum SSL'

Set up SSL

Paste this code as a block to make Fulcrum available on the port 50022 with SSL ncryption through Nginx

cd /home/fulcrum/.fulcrum

# Create a self signed SSL certificate
sudo -u fulcrum openssl genrsa -out selfsigned.key 2048

echo "\
[req]
prompt             = no
default_bits       = 2048
default_keyfile    = selfsigned.key
distinguished_name = req_distinguished_name
req_extensions     = req_ext
x509_extensions    = v3_ca

[req_distinguished_name]
C = US
ST = Texas
L = Fulcrum
O = RaspiBlitz
CN = RaspiBlitz

[req_ext]
subjectAltName = @alt_names

[v3_ca]
subjectAltName = @alt_names

[alt_names]
DNS.1   = localhost
DNS.2   = 127.0.0.1
" | sudo -u fulcrum tee localhost.conf

sudo -u fulcrum openssl req -new -x509 -sha256 -key selfsigned.key \
    -out selfsigned.cert -days 3650 -config localhost.conf


# Setting up the nginx.conf
    isConfigured=$(sudo cat /etc/nginx/nginx.conf 2>/dev/null | grep -c 'upstream fulcrum')
    if [ ${isConfigured} -gt 0 ]; then
            echo "fulcrum is already configured with Nginx. To edit manually run \`sudo nano /etc/nginx/nginx.conf\`"

    elif [ ${isConfigured} -eq 0 ]; then

            isStream=$(sudo cat /etc/nginx/nginx.conf 2>/dev/null | grep -c 'stream {')
            if [ ${isStream} -eq 0 ]; then

            echo "
stream {
        upstream fulcrum {
                server 127.0.0.1:50021;
        }
        server {
                listen 50022 ssl;
                proxy_pass fulcrum;
                ssl_certificate /home/fulcrum/.fulcrum/selfsigned.cert;
                ssl_certificate_key /home/fulcrum/.fulcrum/selfsigned.key;
                ssl_session_cache shared:SSL-fulcrum:1m;
                ssl_session_timeout 4h;
                ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
                ssl_prefer_server_ciphers on;
        }
}" | sudo tee -a /etc/nginx/nginx.conf

            elif [ ${isStream} -eq 1 ]; then
                    sudo truncate -s-2 /etc/nginx/nginx.conf
                    echo "
        upstream fulcrum {
                server 127.0.0.1:50021;
        }
        server {
                listen 50022 ssl;
                proxy_pass fulcrum;
                ssl_certificate /home/fulcrum/.fulcrum/selfsigned.cert;
                ssl_certificate_key /home/fulcrum/.fulcrum/selfsigned.key;
                ssl_session_cache shared:SSL-fulcrum:1m;
                ssl_session_timeout 4h;
                ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
                ssl_prefer_server_ciphers on;
        }
}" | sudo tee -a /etc/nginx/nginx.conf

            elif [ ${isStream} -gt 1 ]; then
                    echo " Too many \`stream\` commands in nginx.conf. Please edit manually: \`sudo nano /etc/nginx/nginx.conf\` and retry"
                    exit 1
            fi
    fi

# Test nginx
sudo nginx -t

# Restart nginx
sudo systemctl restart nginx

Create a Tor .onion service

On RaspiBlitz v1.7.2 run:

/home/admin/config.scripts/tor.onion-service.sh fulcrum 50021 50021 50022 50022

Previous versions:

/home/admin/config.scripts/network.hiddenservice.sh fulcrum 50021 50021 50022 50022

To set up manually see the guide here.

Remove the Fulcrum user and installation (not the database)

sudo systemctl disable fulcrum
sudo systemctl stop fulcrum
sudo userdel -rf fulcrum

# Remove Tor service
/home/admin/config.scripts/tor.onion-service.sh off electrs

# Close ports on firewall
sudo ufw deny 50021
sudo ufw deny 50022

# To remove the database directory
# sudo rm -rf /mnt/hdd/app-storage/fulcrum/db

Docker setup

The database persisted and serving on ports tcp: 50025, ssl: 50027.

Will mount the existing database from (can be any directory to sync new in < 10h on an SSD):

/mnt/hdd/hdd-snapshot-clone/app-storage/fulcrum/db

bitcoind is running on the localhost.

Adapted config in the ./fulcrum.conf

bitcoind = 127.0.0.1:8332
rpcuser = RPCUSER
rpcpassword = RPCPASSWORD
bitcoind_timeout = 600
bitcoind_clients = 1
worker_threads = 1
db_mem=1024
db_max_open_files=200
fast-sync = 1024
peering = false
announce = false
tcp = 0.0.0.0:50025
ssl = 0.0.0.0:50027

Generate TLS credentials

openssl req -newkey rsa:2048 -sha256 -nodes -x509 -days 3650 -subj "/O=Fulcrum" -keyout "tls.key" -out "tls.cert"

Start the image

adapt the values as needed

docker image pull cculianu/fulcrum:latest
docker run \
--network="host" \
-p 50025:50025 \
-p 50027:50027 \
-v "$(pwd)"/fulcrum.conf:/fulcrum.conf \
-v "$(pwd)"/tls.key:/tls.key \
-v "$(pwd)"/tls.cert:/tls.cert \
-v /mnt/hdd/hdd-snapshot-clone/app-storage/fulcrum/db:/db \
-e DATA_DIR=/db \
-e SSL_CERTFILE=/tls.cert \
-e SSL_KEYFILE=/tls.key \
openoms/fulcrum \
Fulcrum /fulcrum.conf

Docker compose snippet

  fulcrum:
    image: cculianu/fulcrum:latest
    depends_on: [bitcoind]
    volumes:
      - ${PWD}/fulcrum.conf:/fulcrum.conf
      - ${PWD}/tls.key:/tls.key
      - ${PWD}/tls.cert:/tls.cert
    environment:
      - DATA_DIR=/db
      - SSL_CERTFILE=/tls.cert
      - SSL_KEYFILE=/tls.key
    command: [ "Fulcrum", "/fulcrum.conf" ]

Sources:

https://github.com/cculianu/Fulcrum
https://sparrowwallet.com/docs/server-performance.html

Fulcrum on a RaspiBlitz

FAQ

Do I need to stop Electrs?

Database corrupted

Automated setup

download

check

run with debug

Display the Tor address

Connect

Monitor

Restart

Edit the config

Manual setup

Prepare bitcoind

Prepare the system and directories

Create a config file

Create a systemd service

Start

Monitor

Open the firewall

Set up SSL

Create a Tor .onion service

Remove the Fulcrum user and installation (not the database)

Docker setup

Generate TLS credentials

Start the image

Docker compose snippet

Sources: