John Lines's Debian blog

Snikket – is an Internet chat service, based on open standards, providing private and secure communication while not depending on phone numbers or a central provider, and can communicate with any other open standard internet chat service. It is available as an app for Android and iPhone, but can be used through any software which supports XMPP.

This article describes how to set up Thunderbird chat with a snikket account, for a user fred.bloggs@snikket.example.org with a password of loquaci0us (you should use a stronger password than this, and specifically do not use this).

Note that Thunderbird does not support some of the features of XMPP, such as OMEMO encryption, or voice of video chats, but it does still allow for basic one to one messaging. In particular the lack of OMEMO encryption means that you are unlikely to be able to participate in Group Chats as they will almost certainly require OMEMO.

In Thunderbird select Edit/Account Settings which will show a screen with information on your current email account(s) and a button ‘Account Actions’

This gives an option ‘Add chat account…’

Select the XMPP option and click Next

Fill this in as (changing the name and domain as appopriate)

Click next and enter your password on the next screen.

You will be shown some extra options

These allow for special cases, which you can ignore, so just press next and see:

Click Finish and you are there.

This post describes how I monitor for mail delivery issues, in particular mails where delivery is blocked by recipients policy, on a small server running Debian Bookworm, with Postfix and the monitoring-plugins-check-logfiles package.

If the system you are delivering to is RFC compliant, i.e. follows the normal internet rules, then if it does not accept an email from you due to policy (such as denying email from your IP address), it should return a status code which starts with 5.7 This will show up in your /var/log/mail.log as a status containing

'dsn=5.7'

I run an icinga2 system to monitor systems for errors, and whenever I discover a problem which had not been picked up by monitoring, and it was one which could have been detected earlier, I don’t consider the issue resolved until I have added a check to look for it in future. For this particular case I use check_logfiles – which can be found in the Debian package monitoring-plugins-check-logfiles, installed on the outgoing mail server, with these configuration items.

Mail server with /var/log/mail.log

Here are the items I added to a mail server running postfix with logs going to /var/log/mail.log, with other items already being monitored.

/etc/nagios/nrpe.d/check_logfiles_dsn57.cfg

command[check_logfiles_dsn57]=/usr/bin/sudo /usr/local/bin/check_logfiles_dsn57

/etc/sudoers.d/nagios_check_logfiles_dsn57

nagios ALL=(root) NOPASSWD:/usr/local/bin/check_logfiles_dsn57

/usr/local/bin/check_logfiles_dsn57

#/bin/sh

/usr/lib/nagios/plugins/check_logfiles --tag=dsn57 --criticalpattern="dsn=5.7" --logfile=/var/log/mail.log

Mail server with journald

Transitioning away from traditional syslog, with postfix logs in journald needs a slight tweak to the monitoring check_logfiles, which I patched as shown in the patch attached to Debian bug #1060859. I also changed the local script to be

/usr/local/bin/check_logfiles_dsn57

#/bin/sh

/usr/lib/nagios/plugins/check_logfiles_journal_identifier --tag=dsn57 --type=journald:identifier='postfix/smtp' --criticalpattern="dsn=5.7"

Icinga2 server configuration

The service check for check_logfiles_dsn57 is likely not to find a new adminstratively denied mail the next time it is run (I have a check_interval of 1h), so I use the following non standard parameters for the nrpe service which does the check

  check_command = "nrpe"
  check_interval = 1h
  retry_interval = 24h
  volatile = true
  vars.nrpe_command = "check_logfiles_dsn57"

The ‘retry_interval = 24h’ leaves the critical alert for the problem visible until it has been investigated, and will then be cleared by re-running the test.

I have ‘nagstamon‘ (from the Debian package) running on my desktop as a constant overview which needs less screen space than the icingaweb page, which provides more information for a detailed investigation.

This is a work in progress and will be updated as I get time. Initially it simply documented what I did for my own server, but now it reflects a build on a test server. I intend to refine it iteratively.

My first installation of the Pleroma server social.paladyn.org, was through a Yunohost server, which I had set up on a QEMU Virtual Machine, which runs Debian Bullseye. This worked quite well to try, but had several drawbacks, including that it shared the same external IPv4 address with several other systems, which introduced complexities.

Pleroma, Mastodon or what ?

Mastodon is the best known name in the ActivityPub Federated area, and my first venture into the Fediverse was as @JohnLines@mstdn.io – but I wanted to understand the platform well enough that I could contribute to making it more widely available – probably as part of a Small Organisation Server.

Debian has a team working on providing Social Networking for Developers, and another working on packaging. Through the Social team I am @jlines@pleroma.debian.social which steered me towards Pleroma, particularly as Pleroma and Mastodon work together.

That working together, but not being Mastodon, is important as for a protocol to be an IETF Standard it must have two independent, interoperating implementations. Diversity in implementing the Fediverse makes for a healthier ecosystem.

Yunohost

Yunohost is an all-in-one solution, which is well worth investigating if you have a system which is not already set up doing other things, but in my case I decided to migrate to running Pleroma directly on a Debian Bullseye system which was already running Nginx, but did not have much running which would conflict with Pleroma.

Pleroma from Source

The official Pleroma instructions for install on Debian have two methods, one is what they call an OTP release – where they supply a pre-built binary image for many popular computer architectures, the other is to build it from source.

As my end goal is a Debian package, I went down the build from source route. In the following instructions I have adhered to the Pleroma instructions, rather than Debian standards for file locations etc.

Picking a name and setting up DNS.

You will need a name for your Federated Social Media System. Although this document is about installing Pleroma you may want to consider a more generic name, for example setting up social.example.com, in the same way as you might use www.example.com, rather than apache.example.com to allow a future switch to nginx, or vice versa. If you set up the name pointing to your target system now then there is more chance for it to propagate before you need it. You will also be asked an admin email address later, and an address for sending email notifications. You may need to do some preparation for these too.

Preparing for the installation.

In order to make sure the target system knows the dependencies which should be installed I have started a Debian package called pleroma-installer.

pleroma-installer

Note that in these commands I have a blank line between each command. They may wrap round when shown in your browser, but should be entered as one line. I have put parts you will need to change for your system in italics.

At the moment this simply asks for dependencies, and can be downloaded from https://paladyn.org/john/tmp/pleroma-installer_0.0.1_all.deb – install it by

wget -nc https://paladyn.org/john/tmp/pleroma-installer_0.0.1_all.deb

sudo apt install  ./pleroma-installer_0.0.1_all.deb

This will install the dependencies, and apt will know that they were installed for a reason, and not try to clean them up.

The plan is to create an apt repository for this installer package to enable it to be updated by apt, however as it is an installer, not a real package this is not a high priority.

Create a user and directories

On installation the package will create a pleroma user and a directory /opt/pleroma which is the main place the files are kept. If /opt/pleroma exists it will be moved to /opt/pleroma.yyyymmddhhmm and the pleroma files will be fetched again.

There will be a warning about Federation not working until it is configured.

At present this is as far as the installer package goes, partly as the next section will have some questions to answer.

Generate the instance

cd /opt/pleroma

sudo -Hu pleroma MIX_ENV=prod mix pleroma.instance gen 

Say Y to install rebar3. You will be asked for the name and email addresses you chose above. Otherwise I used the defaults, except for answering ‘y’ to Do you want to store the configuration in the database.

The installation should complete, saying ‘All files successfully written!’

sudo -Hu pleroma mv config/{generated_config.exs,prod.secret.exs}

sudo -Hu postgres psql -f config/setup_db.psql

The above steps put the configuration where pleroma expects it, and do the initial database setup. As I chose the option to store the configuration in the database I did:

sudo -Hu pleroma MIX_ENV=prod mix ecto.migrate

You should see logs about lots of files being compiled.

At this stage pleroma should be installed and ready to run, start it with

sudo -Hu pleroma MIX_ENV=prod mix phx.server

You will end up with pleroma running as a background process.

Set up nginx

If you started with an empty server then the installer should have pulled nginx, but if you already had apache2 installed then it will not switch web servers on you. I have not tested pleroma behind apache2. but there is a sample apache configuration in /opt/pleroma/installation

Check if nginx is running by

systemctl status nginx

and stop it by

systemctl stop nginx

Now install an SSL certificate for your new subdomain – note that it will need to exist by this stage.

sudo certbot certonly --email myuser@example.org -d social.example.org --standalone

This can fail if some other program is using port 80 (the non SSL secured web port on your computer) – for example if or apache is running, of if your domain is mistyped, or does not exist yet.

sudo cp /opt/pleroma/installation/pleroma.nginx /etc/nginx/sites-available/pleroma.nginx

sudo ln -s /etc/nginx/sites-available/pleroma.nginx /etc/nginx/sites-enabled/pleroma.nginx

Edit /etc/ngix/sites-available/pleroma.nginx to change all occurrences of sample.tld to your server name e.g. social.example.com

sudo cp /opt/pleroma/installation/pleroma.service pleroma.service

sudo systemctl enable --now pleroma.service

You should now be able to access your server at https://social.example.com/ – check here if this does not work.

Create your first user

sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new myuser myuser@example.com --admin

Part of the output from the pleroma.user command will be a password reset URL – enter the link into a web browser, set your password and you should have set up a pleroma server !

Feedback on this post is welcome, either by comments on this post (note that they are moderated so will not appear instantly) or via federated social media to @jlines@pleroma.debian.social.

Troubleshooting

Pleroma has quite a number of pieces, and there are stages where the setup may run into problems. Here are some of the ones which people have encountered, together with what to do about them.

Certbot unable to set up the SSL certificate

Nginx (or apache) showing the home page, not Pleroma

If you went to the web page of the site you set up, for example https://social.example.org/ – but see a default web page then web server configuration has not taken notice of the new site you added. The first thing to do is restart nginx

systemctl restart nginx

and look again, if you see the home page still then you will need to look further.

The sites which are live can be found in /etc/ngnix/sites-enabled/ – as symbolic links to the real configurations.

You can check by

ls -l /etc/nginx/sites-enabled

You should see a line like

lrwxrwxrwx 1 root root 40 Nov 3 18:39 pleroma.nginx -> /etc/nginx/sites-available/pleroma.nginx

Nginx not running

Removing pleroma

When there is a real Debian package this should just a a matter of

apt remove pleroma

but at present it does not have an uninstall script, and what it should do to tidy up is not obvious.

Removing the database

You might want to do this in order to do a re-install cleanly

systemctl stop pleroma.service

sudo -Hu postgres psql -c 'DROP DATABASE pleroma; '

sudo -Hu postgres psql -c 'DROP USER pleroma; '

Removing the files

All the files are created under /opt/pleroma, but if you run the installer a second time it moves /opt/pleroma in order to have a clean install, but not lose anything if you want to go back, or compare configuration files.

For anything continue to operate, it has to be, in some form, financially viable. (I regard economics as one of the key decision making tools available to people – see How do we decide ?). I am also suspicious of systems which try to conceal their economic model. (see Transparancy and Trust, and Who pays for WhatsApp).

Monolithic Social Media

Western Commercial Social Media (CSM), by which I mean systems such as Facebook, YouTube, TikTok, WhatsApp etc has two primary funding streams.

1. Selling advertising space on their own platform
2. Selling information about their users.

As some users are starting to find the advertisements when using the platform intrusive, they are developing a new stream, where users pay to use the platform without seeing any advertising, but this does not interfere with stream 2, the gathering of data about their user base.

The Chinese Social Networking system WeChat also has an eCommerce system, WeChat Pay, which gives it another revenue stream, an area in which Meta group and Alphabet have struggled to establish a foothold.

Federated Social Media

Federated social media systems have the following funding options:

Donations from users

This is the model which funds, in theory, many Federated social media sites. Generally they are free to use, and the system relies on some people being sufficiently generous to keep the site in operation. The sites are generally run by someone with a good technical understanding. The users, prior to joining the site, did not have a particular relationship with the person/group who runs it, although it may be set up for some general class of users, for example those with an interest in the environment.

Funding by Governments and large companies

The main present example of this is the European Commission. For a government or large organisation with a Public Relations department, or similar, funding a Federated site is a trivial expense and has the potential to extend their reach, particularly as it is unlikely to be impacted by controversies related to that platform. (for example people may like what you do, but stop following you on, say Twitter – if they do not like what you do they will not follow you on any platform anyway).

Some large organisations may fund their own social media system, for example Truth Social, generally with a particular political slant.

Journals, Newspapers, TV channels

Large media organisations i.e. newspapers and television channels are a natural fit for Federated Social Media, as I point out in Federated Social Media and Journalism. They also have a role as a conduit for funding from readers (if those readers pay some form of subscription to the journal or media outlet) to the journalists. Other means of financially rewarding journalists, writers and artists exist, such as Patreon, but they too involve a middle-man. The journals add value for the reader by associating, and risking, their reputation on the quality of the writing they publish, and take a cut for that. The relationship between the size of the cut and the value added too complex to discuss here, but compared to the other costs involved the technical side of running servers is (probably) small.

Funding by small organisations and individuals.

If the technical knowledge needed to run a social media system could be reduced then the cost or running one for the benefit of members is within the reach of many small organisations. The advantage of running their own system and federating with others is that the content can be more focused. By funding directly, without advertising, there is less risk that the social media feed will end up carrying content which does not match the goals of that organisation.

Companies providing an excellent service

If a company has a good reputation then having a Fediverse presence allows customer testimonials to be shared in a way which is more more natural than Commercial Social Media advertising. For example Mythic Beasts and Olimex can easily be referred to in conversations where they are relevant.

Most advertising on CSM is intrusive and jarring, as it would be if you were out with friends in some social setting like a pub or cafe, chatting about a holiday, and some stranger butted in, trying to sell you a holiday.

Selling Advertising

Although nobody, as far as I know, is doing this yet, it should be possible to support a social media site through advertising sold through a broker, without needing that broker (which in many ways is what Facebook etc are) needing to own the site outright. Being federated users could post content onto those sites, which would be visible to their followers on other sites, who would also see some advertisements. Google, Facebook, and Amazon already act as advertising brokers and have mechanisms to place advertising on third party web sites. It might require and extension to the ActivityPub protocol to insert the advertising posts.

Sponsorship

People who make a living being an influencer could run their own site, and, clothing and lifestyle companies, who currently sponsor them could do so directly.

Finance, Ownership and Control.

For all forms of mass media there is a complex, but important, relationship between the sources of revenue which pay for it, who owns it, and who can control it. For example while WeChat is owned by Tencent and TikTok is owned by ByteDance, in both cases the Chinese government has (according to some people) a substantial degree of control over their activities.

The control of Federated Social Media systems is different, in that the name of the system, for example PeerTube – a Federated Video sharing system, and individual ‘instances’ have their own ownership and agendas. There are Peertube systems which host videos about technical subjects, such as https://peertube.debian.social/ and Framasoft, and big creator of Free Federated Software, both funded by donations.

You could think of, say Twitter, as an unfederated Social Media system, which provokes thought about the balance between who pays for it (a mixture of advertisers and investor(s)) and who controls it.

The Sender Policy Framework (SPF) is an important part of preventing email forgery on the Internet, preventing spammers from forging mail which pretends to be from you.

SPF records

It is common in modern mail systems for the email for an organisation to be delivered by some specialist third party, rather than the organisation itself. The SPF (Sender Policy Framework) record is a way of declaring to the world who you trust to be sending mail on your behalf.

Getting this right is important, as it is rather similar to a Power of Attorney, you are telling the world that they should trust the systems listed as if they were you when they receive an email. It is essentially an anti-forgery system.

If your SPF record is incorrect you are likely to encounter mail delivery problems. The impact will vary depending on the recipient, but the best way to fix them is to make sure your SPF record is correct using an online SPF checker.

SPF checkers

Several companies provide a web page which allow you to enter your domain name and they will tell you if there are problems with your SPF record – and offer to sell you as solution if a problem is detected. They all tend to tell you roughly the same if your SPF record is correct, but differ in how informative the message is if your SPF is invalid.

• https://mxtoolbox.com/spf.aspx
• https://dmarcly.com/tools/spf-record-checker
• https://www.proofpoint.com/us/cybersecurity-tools/dmarc-spf-creation-wizard#spf-check
• https://easydmarc.com/tools/spf-lookup

A search for ‘spf checker’ will turn up more. You can check any domain with them, not just your own, so you can see if a mail problem from some domain is SPF related.

SPF problem – too many DNS lookups

As the DNS is so important to the whole internet, RFC 7208 (one of the ‘rules of the road’ of the Internet) states that an SPF record MUST not require than 10 DNS lookups. Without this there would be a way for a bad person to attack people on the internet in a way which is difficult to trace – an example of this is explained at SPF Too Many DNS Lookups, in the section ‘Why is There an SPF Lookup Limit?’. That link also contains some general suggestions as to how to tidy the SPF record.

SPF and Surveymonkey

SurveyMonkey, a popular survey management company, can sometimes have its SPF record incorrectly added to that of customers, but they themselves state, in their help page, that ‘You do not need to add SPF or DKIM records to your domain when using SurveyMonkey.‘

SPF and bulk mail sending companies

You might have a contract with a bulk mail sending company, for example to send out a newsletter. Depending on the company you may be asked to add their SPF list to yours, and their SPF list may be quite large. Not all mailers require this, so it is worth checking.

Another possibility to consider is setting up a subdomain, such as newsletter.example.com, or a more generic emails.example.com, and asking the bulk mailing company to use that. That subdomain will then have its own SPF list, which will not normally need to include your own email provider.

SPF and changing email providers

It is quite natural, if you change email providers, to add the new one to your SPF list, but sometimes the step of removing the previous one to your SPF list can be forgotten.

But mail still goes to Gmail !

Just as the laws which most people follow do not apply to everyone, for example if they have diplomatic immunity, Google can choose which rules they follow. As in the case of Harry Dunn, just because they can does not make it correct.

If a wicked person was to divert a major road’s worth of traffic down your residential street it would cause chaos, but Google effectively owns multiple motorways, so they are immune to the problems which affect others.

This is a work in progress –

Many organisations are very concerned about their identity, hiring expensive consultants to redesign their logo, with the result being reviewed at the most senior level. Their identity on the Internet tends to receive much less attention, being delegated, without much thought to some ‘techie’ people, either inside or outside the organisation, without much guidance as to what the online identity should be.

At the highest level identities, or Names on the Internet are divided into a number of Domains in a structure a bit like a tree, although conventionally pictured with the root at the top! From the root come a number of major branches, the .com, .edu, .org, (etc) naming areas, originally for Commercial, Educational (American), other Organistations, and .uk, .de, .fr for United Kingdom, German (Deutschland), France and so on. Each of these branches is controlled by some naming authority, and some of those authorities sell, through brokers, names in that space on the open market. Others names are not for sale, for example you can not buy a name which ends .nhs.uk, which is reserved for the British National Health Service.

Most individuals, and some small organisations will not even start down this road and will exist purely as an identity within one of the Internet giants, for example Gmail, Facebook, Twitter etc – in which case their identity is whatever works within the rules of that system.

This article is for people and organisations who have, or are planning to have their own identity on the Internet. In examples I will assume you are using example.com as your name.

The Domain Name System (DNS)

Email

The mail system depends on the DNS, so if the DNS is set up incorrectly then there will be mail delivery problems.

MX records

(These are essential to mail delivery, and I will update this article to cover them)

SPF records

It is common in modern mail systems for the email for an organisation to be delivered by some specialist third party, rather than the organisation itself. The SPF (Sender Policy Framework) record is a way of declaring to the world who you trust to be sending mail on your behalf.

Problems with your SPF record will cause problems delivering your email, and have their own article on SPF Problems.