OpenSMTPD relay setup

These days, most of us don't run a mail server to send e-mail. Instead, we deliver to a provider's mail server via SMTP.

Setting this up in OpenSMTPD is quite simple. We will assume that all of the machine's e-mail is going to be relayed through a single GMail account. A more complex situation will require a more complex setup.

This is based on the EXAMPLES section in OpenSMTPD's manual.

In general (particularly if you're using multi-factor authentication), you need to set up an »app password« for your Google account. The 16 character password must be stored in an arbitrary file, we'll call it /etc/mail/secrets, giving it an arbitrary »label« for which we'll use gmail.

# touch /etc/mail/secrets
# chmod 640 /etc/mail/secrets
# chown root:_smtpd /etc/mail/secrets
# echo "gmail username:password" > /etc/mail/secrets

Next, the following line needs to be added to /etc/mail/smtpd.conf:

table secrets file:/etc/mail/secrets

Then, the relay action of /etc/mail/smtpd.conf needs to be adjusted as follows. We're using the same »label« gmail that we used in the secrets file.

action "relay" relay \
        host smtps://gmail@smtp.gmail.com \
        auth <secrets>

You should then restart smtpd.

# rcctl restart smtpd

Last, in order to provide a proper »From« line, you should add a line like the following to ~/.mailrc:

set from="User Name <username@gmail.com>"

It's now time to send your first e-mail, preferably to yourself:

$ echo hi | mail -s test0 username@gmail.com

If it gets stuck, look at /var/log/maillog for the error. While your e-mail is in the queue, OpenSMTPD will automatically attempt to deliver it after every change to the configuration. Use smtpctl show queue and smtpctl remove to control the mail queue.

Installing NetBSD on a DEC Multia by booting over the network

As it happens, I own a DEC Multia which has been sitting around for far too long and deserves some of my attention.

Unfortunately, when I boot from the built-in SCSI disc, the NetBSD bootstrap cannot find /boot and crashes. I then get back to the SRM console. I don't know what I did to the system years ago, but it's not booting.

The Multia has a built-in floppy drive, but it's my only floppy drive, so I can't create boot floppies for it (even though I still have an old pack of disks).

So I decided to try installation via bootp/tftp. On my OpenWRT router, I configured a static IP address for my Multia's MAC address since bootp doesn't support dynamic assignments. I then set the tftp path to /tmp/tftp since that partition has enough space on my router.

I'm following the instructions in NetBSD's diskless(8) manual page for setting up a network boot.

NetBSD's netboot secondary stage boot loader goes into the /tmp/tftp/ directory and I set the tftp file name to netboot.

Here the trouble starts: the boot loader needs to have the client's MAC address hardcoded into it because it can't find it out by itself. This is the job of setnetbootinfo(8), which of course needs another NetBSD system to run on.

Since I only have an OpenBSD system running on Intel handy, I compile their setnetbootinfo program:

$ cc -o setnetbootinfo \
    -I /usr/src/sys/arch/alpha/stand \
    /usr/src/sys/arch/alpha/stand/setnetbootinfo/setnetbootinfo.c

I use it like this:

$ wget https://ftp.openbsd.org/pub/OpenBSD/6.4/alpha/netboot
$ ./setnetbootinfo -a xx:xx:xx:xx:xx:xx netboot

It also works on NetBSD's netboot loader and it now advances one step further, trying to find out the kernel server and file name by using bootp.

The above mentioned manual page has the following sample configuration for NetBSD's dhcpd:

host myclient {
        hardware ethernet 8:0:20:7:c5:c7;
        fixed-address myclient;         # client's assigned IP address
        filename "myclient.netboot";    # secondary bootstrap
        next-server myserver;           # TFTP server for secondary bootstrap
        option swap-server myserver;    # NFS server for root
        option root-path "/export/myclient/root";
}

OpenWRT runs dnsmasq instead of dhcpd, so I need to translate the swap-server and root-path options to it.

Abridged for brevity, I get this on my OpenWRT box:

# dnsmasq --help dhcp
Known DHCP options:
[...]
 16 swap-server
[...]
 17 root-path
[...]

This suggests a dnsmasq(1) command line like this:

# dnsmasq \
        --dhcp-option=option:swap-server,0.0.0.0 \
        --dhcp-option=option:root-path,/data/netboot

This can be set up using the following uci commands:

# uci add_list 'dhcp.@dnsmasq[0].dhcp_option=option:swap-server,0.0.0.0'
# uci add_list 'dhcp.@dnsmasq[0].dhcp_option=option:root-path,/data/netboot'
# uci commit

This should add two lines in the config dnsmasq section of /etc/config/dhcp:

list dhcp_option 'option:server-swap,0.0.0.0'
list dhcp_option 'option:root-path,/data/netboot'

Afterwards, /etc/init.d/dnsmasq restart activates the changes.

I now get stuck at bootp: no reply exactly like Tobias, whose problem turned out to be his BNC connection while I'm using twisted pair. In my case, the problem turned out to be caused by a fix for »broken« firmware, as described in PR/6446. After finding another working Alpha box, I was able to patch netboot and get it working (see below).

Eventually I found a netboot image for Debian Lenny (the last one to support Alpha) in their archive. The linux boot works without a second-stage boot loader and loads the boot loader and kernel together using the SRM console's BOOTP/TFTP request. The boot loader asks for kernel arguments and then proceeds to load the kernel at the aboot> prompt. However, there is no further reaction after the message

aboot: starting kernel network with arguments

Back to the NetBSD netboot loader: it seems I need a little patch in if_prom.c to remove some code that looks like it was added to work around an unspecified bug in certain firmware versions:

Index: if_prom.c
===================================================================
RCS file: /pub/NetBSD-CVS/src/sys/arch/alpha/stand/netboot/if_prom.c,v
retrieving revision 1.19
diff -u -p -u -8 -r1.19 if_prom.c
--- if_prom.c   13 Mar 2003 14:15:58 -0000      1.19
+++ if_prom.c   9 Apr 2019 06:09:21 -0000
@@ -98,26 +98,24 @@ prom_get(struct iodesc *desc, void *pkt,
        prom_return_t ret;
        time_t t;
        int cc;
        char hate[2000];
 
        t = getsecs();
        cc = 0;
        while (((getsecs() - t) < timeout) && !cc) {
-               if (broken_firmware)
+               if (0 && broken_firmware)
                        ret.bits = prom_read(booted_dev_fd, 0, hate, 0);
                else
                        ret.bits = prom_read(booted_dev_fd, sizeof hate, hate, 0);
                if (ret.u.status == 0)
                        cc = ret.u.retval;
        }
-       if (broken_firmware)
-               cc = min(cc, len);
-       else
+       if (len < cc)
                cc = len;
        memcpy(pkt, hate, cc);
 
        return cc;
 }
 

What is strange here is that the routine always returns the requested number of bytes read even if timeout occurs unless broken_firmware is true, in which case it always passes 0 as the number of bytes to read to the PROM reading routine which doesn't work (on my machine). The broken_firmware flag is based on whether the boot device name contains the hardware ethernet address, which isn't the case on my system (see above the need to use setnetbootinfo). Still, I need to pass the correct amount of len in order to read successfully. For reference, the SRM console version is

Multia SRM Console  BL5 V3.8-1, built on Jun 22 1995 at 18:10:45

With this, I can successfully boot the NetBSD 8 installation kernel. Installing OpenBSD was then just a couple of commands away:

# mount_nfs jeopardy:/data/netboot /mnt
# dd if=/mnt/miniroot64.fs of=/dev/rsd0c
# reboot

Patching OpenBSD installation process

To successfully install OpenBSD 6.4 and 6.5 with 88MB of RAM and a 512MB HDD, I needed to disable kernel relinking (KARL) because the machine has no swap space and runs out of memory, resulting in randomly killed ksh processes. So before running the installation, I removed that section from the installation script install.sub.

# ed install.sub
/Relink
                echo -n "Relinking to create unique kernel..."
-
        if [[ -f $_kernel_dir.tgz ]]; then
c
        if false; then
.
w
q
# ./install

Note: It might be sufficient to skip installing the kernel tgz package, but I haven't tried that.

After installation and before the first boot, I also modified the on-disk rc script to avoid kernel relinking:

# ed /mnt/etc/rc
/reorder_kernel
/usr/libexec/reorder_kernel &
s/^/#
#/usr/libexec/reorder_kernel &
w
q

Lastly, I turned off library address space layout randomisation as it takes a very long time during boot on this machine.

# echo library_aslr=NO >> /mnt/etc/rc.conf.local

For geeks (and google), here is the NetBSD dmesg of the machine:

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
    2018 The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 8.0 (INSTALL) #0: Tue Jul 17 14:59:51 UTC 2018
 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/alpha/compile/INSTALL
(PCI ISA), 267MHz, s/n
8192 byte page size, 1 processor.
total memory = 90112 KB
(2368 KB reserved for PROM, 87744 KB used by NetBSD)
avail memory = 75640 KB
timecounter: Timecounters tick every 0.976 msec
Kernelized RAIDframe activated
mainbus0 (root)
cpu0 at mainbus0: ID 0 (primary), LCA-2 (21066)
lca0 at mainbus0
pci0 at lca0 bus 0
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
siop0 at pci0 dev 6 function 0: Symbios Logic 53c810 (fast scsi)
siop0: interrupting at isa irq 11
scsibus0 at siop0: 8 targets, 8 luns per target
sio0 at pci0 dev 7 function 0: vendor 8086 product 0484 (rev. 0x84)
tlp0 at pci0 dev 8 function 0: DECchip 21040 Ethernet, pass 2.3
tlp0: interrupting at isa irq 15
tlp0: Ethernet address 08:00:2b:e5:f0:1e
tlp0: 10baseT, 10baseT-FDX, 10base5, manual
vendor 0047 product 0280 (miscellaneous network, revision 0x47) at pci0 dev 9 function 0 not configured
tga0 at pci0 dev 11 function 0: DC21030 step B, board type T8-02
tga0: 1024 x 768, 8bpp, Bt485 RAMDAC
tga0: interrupting at isa irq 10
wsdisplay0 at tga0 (kbdmux ignored): console (std, vt100 emulation)
isa0 at sio0
lpt0 at isa0 port 0x3bc-0x3bf irq 7
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0 (mux ignored): console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 (mux ignored)
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
mcclock0 at isa0 port 0x70-0x71: mc146818 compatible time-of-day clock
stray isa irq 3
stray isa irq 4
timecounter: Timecounter "clockinterrupt" frequency 1024 Hz quality 0
timecounter: Timecounter "PCC" frequency 266631696 Hz quality 1000
scsibus0: waiting 2 seconds for devices to settle...
sd0 at scsibus0 target 0 lun 0:  disk fixed
sd0: 518 MB, 4212 cyl, 4 head, 63 sec, 512 bytes/sect x 1061712 sectors
sd0: sync (100.00ns offset 8), 8-bit (10.000MB/s) transfers
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
md0: internal 4650 KB image area
root on md0a dumps on md0b
root file system type: ffs
kern.module.path=/stand/alpha/8.0/modules

And here's OpenBSD's.

[ using 1143368 bytes of bsd ELF symbol table ]
consinit: not using prom console
Copyright (c) 1982, 1986, 1989, 1991, 1993
 The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2018 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.4 (GENERIC) #264: Thu Oct 11 23:15:35 MDT 2018
    deraadt@alpha.openbsd.org:/usr/src/sys/arch/alpha/compile/GENERIC
(PCI ISA), 267MHz
8192 byte page size, 1 processor.
real mem = 92274688 (88MB)
rsvd mem = 2424832 (2MB)
avail mem = 78299136 (74MB)
mainbus0 at root
cpu0 at mainbus0: ID 0 (primary), LCA-2 (21066 pass 2)
lca0 at mainbus0
pci0 at lca0 bus 0
siop0 at pci0 dev 6 function 0 "Symbios Logic 53c810" rev 0x02: isa irq 11
scsibus0 at siop0: 8 targets, initiator 7
sd0 at scsibus0 targ 0 lun 0:  SCSI2 0/direct fixed serial.TOSHIBA_MK1924FBV_75M20420W_M_/_
sd0: 518MB, 512 bytes/sector, 1061712 sectors
sio0 at pci0 dev 7 function 0 "Intel 82378IB ISA" rev 0x84
de0 at pci0 dev 8 function 0 "DEC 21040" rev 0x23, DEC 21040 pass 2.3: isa irq 15, address 08:00:2b:e5:f0:1e
tga0 at pci0 dev 11 function 0 "DEC 21030" rev 0x02: DC21030 step B, board type T8-02
tga0: 1024 x 768, 8bpp, Bt485 RAMDAC
tga0: interrupting at isa irq 10
wsdisplay0 at tga0 mux 1: console (std, vt100 emulation)
isa0 at sio0
isadma0 at isa0
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x3bc/4 irq 7
pcic0 at isa0 port 0x3e0/2 iomem 0xd0000/65536
pcic0 controller 0:  has sockets A and B
pcmcia0 at pcic0 controller 0 socket 0
pcmcia1 at pcic0 controller 0 socket 1
pcic0: irq 14, polling enabled
mcclock0 at isa0 port 0x70/2: mc146818 or compatible
stray isa irq 14
stray isa irq 7
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
siop0: target 0 now using 8 bit 10.0 MHz 8 REQ/ACK offset xfers
root on sd0a (942e3e0ab7215cc1.a) swap on sd0b dump on sd0b
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec

Denon RC-1146 remote with LibreELEC 9

In LibreELEC 9, the IR driver was changed from lircd to in-kernel decoding with ir-keytable.

I'm still using the GPIO IR receiver from this tutorial on my Raspberry Pi.

To activate it, I need to add a dtoverlay.

# mount -o remount,rw /flash
# echo dtoverlay=gpio-ir >> /flash/config.txt
# mount -o remount,ro /flash

I then proceeded along this tutorial.

# systemctl stop kodi
# systemctl stop eventlircd

I get from ir-keytable:

# ir-keytable
Found /sys/class/rc/rc0/ (/dev/input/event2) with:
 Name: gpio_ir_recv
 Driver: gpio_ir_recv, table: rc-rc6-mce
 lirc device: /dev/lirc0
 Supported protocols: other lirc rc-5 rc-5-sz jvc sony nec sanyo mce_kbd rc-6 sharp xmp
 Enabled protocols: lirc nec rc-6
 bus: 25, vendor/product: 0001:0001, version: 0x0100
 Repeat delay = 500 ms, repeat period = 125 ms

After some trial and error with different protocols, I finally found:

# ir-keytable --protocol sharp --test
Protocols changed to sharp
Testing events. Please, press CTRL-C to abort.
1009.423304: lirc protocol(sharp): scancode = 0x8ac
1009.423341: event type EV_MSC(0x04): scancode = 0x8ac
1009.423341: event type EV_SYN(0x00).

As with all things Linux, documentation is notoriously difficult to come by. According to ir-keytable --version, LibreELEC-9.0.1 uses v4l-utils-1.14.2. This is an important detail since v4l-utils have since changed their keytable format to toml!

I could not find any documentation for the old keytable format, so I had a look at the source code of keytable.c.

Apparently, a # in the first line can be followed by table and type keywords specifying the name of the table and what is otherwise known as the protocol, separated by various combinations of whitespace, equal signs, colons and commas. Otherwise, lines beginning with # are ignored. The remaining lines may start with an optional keyword "scancode", followed by whitespace, equals sign or colon. This is followed by the scancode (in a format understood by strtoul(3)) and terminated by whitespace, equals sign or colon. Finally, there should be a keycode terminated by whitespace, equals sign, colon or an open parenthesis(!). The keycodes can be ones listed by irrecord --list or, better yet, ones from the devinput section of /usr/share/kodi/system/Lircmap.xml to make sure they are assigned the desired function in Kodi, or in a format suitable for strtol(3). The rest of the line is ignored and can thus be used for comments.

Since Kodi is connected to the DVD input of my amp, I used the remote's default DVD profile. Eventually, I ended up with the following /storage/.config/rc_keymaps/denon1146dvd:

# table denon1146dvd, type: sharp
0x6c0 KEY_POWER
0x881 KEY_0
0x882 KEY_1
0x883 KEY_2
0x884 KEY_3
0x885 KEY_4
0x886 KEY_5
0x887 KEY_6
0x888 KEY_7
0x889 KEY_8
0x88a KEY_9
0x88c KEY_SUBTITLE 10+
0x891 KEY_INFO menu
0x892 KEY_BACK
0x898 KEY_NEXT
0x899 KEY_PREVIOUS
0x89a KEY_FASTFORWARD
0x89b KEY_REWIND
0x89d KEY_PAUSE
0x8a0 KEY_PLAY
0x8a1 KEY_STOP
0x8ac KEY_UP
0x8ad KEY_DOWN
0x8ae KEY_RIGHT
0x8af KEY_LEFT
0x8bb KEY_ENTER
0x8bc KEY_EPG search
0x8bd KEY_MENU top menu

The non-obvious assigments are: the key labelled »menu« on the remote sends a KEY_INFO code since that seems to be the most useful code in Kodi (file info or playing info, respectively). The »top menu« key sends KEY_MENU which takes you to Kodi's main menu and toggles the play menu. I assigned KEY_EPG to the »search« key since KEY_EPG invokes the context menu in the menu and the decoder info while playing. Finally, I cheekily remapped the »10+« key to KEY_SUBTITLE since the former seems to be unused and I use the latter a lot.

Last, I activated it like this:

# echo '* * denon1146dvd' >> /storage/.config/rc_maps.cfg

Setting $PAGER correctly

You would think that setting up a classical tool like more(1) nowadays should not be too difficult. Alas, the devil is in the details.

Up front, here are my requirements for a pager:

  • If I just scroll through the output, it should behave like cat(1). Consequently, it must not clear the screen and it must exit as soon as it reaches EOF.
  • It should display colour.
So far, so straightforward.

As much as I like to use POSIX utilities, more(1) doesn't fit the bill because it is specified to print control characters as implementation-defined multi-character sequences. Using non-standard switches like -R in order to modify its behaviour seems pointless, so I opted for less(1) instead.

Less has a couple of options to make it behave the way I want:

-E
Causes less to automatically exit the first time it reaches end-of-file.
-R
ANSI "color" escape sequences are output in "raw" form.
This sounds like it should do the right thing…

First of all, I tested my changes using the line

$ PAGER='less -ER' git log --all
Interestingly, it did not seem to make a difference whether I used the -R switch or not. As it turns out, this is because unless it is already set, git (and hg likewise) add the environment variable
LESS=FRX
when calling the pager. You can use PAGER=env to verify this. This means that for testing, it's actually better to use
$ git log --all --color | less -ER

As an aside, git and hg also set another variable which must be for a pager that I don't know:

LV=-c

Another stumbling block came when I actually tried this on one of my macOS systems. I had TERM set to xterm and output didn't appear at all if it was less than a screenful. This turned out to be because xterm's initialisation string switches to the alternate screen and clears it. Since my requirement is to work like cat(1), I don't need the alternate screen for this purpose. But I must admit that it's a nice feature when using vi(1), for example, so I don't want to change my TERM setting to ansi which doesn't have an alternate screen.

So the correct solution (which was already anticipated by git) is to also pass the -X option to less(1), causing it to ignore the terminal initialisation sequence.

And finally, unfortunately the meaning of the -c option is reversed on OpenBSD to what POSIX specifies for more(1), and clearing the screen is the default behaviour for less(1) on that platform. So in order to make my .profile work, I ended up using the code

if [ OpenBSD == "$(uname -s)" ]
then
    PAGER='less -ERXc'
else
    PAGER='less -ERX'
fi
export PAGER

Understanding and debugging fontconfig font fallback on macOS

Recently, I used an app on macOS that uses fontconfig. While this filled me with an appropriate sense of dread, I encountered an issue which forced me into debugging its font selection process.

Unfortunately, the app could not display certain symbols I was using (⚭, ⚯) even though my system had fonts providing those glyphs. Using FontBook.app, I found out that the two glyphs are contained in Apple Symbols.

In order to debug the issue, I used fc-match as follows.

$ fc-match -s :charset=26ad # ⚭
LastResort.otf: ".LastResort" "Regular"
$ fc-match -s :charset=26af # ⚯
LastResort.otf: ".LastResort" "Regular"

This result shows us that fontconfig doesn't find a font for the requested glyphs and returns the LastResort font which instead provides replacement glyphs with the Unicode character code inside them.

Knowing that a font was actually available for those glyphs, I started debugging the issue:

$ FC_DEBUG=4 fc-match -s :charset=26ad
[…]
FcConfigSubstitute donePattern has 5 elts (size 16)
        family: "Arial"(s) […]
        hintstyle: 1(i)(w)
        charset:
        0026: 00000000 00000000 00000000 00000000 00000000 00002000 00000000 00000000(s)
        lang: "en"(w)
        prgname: "fc-match"(s)

[…]

The FC_DEBUG variable selects »match/test/edit execution« debug output.

This shows the process by which fontconfig builds up the pattern that it uses to search for a font. The string donePattern indicates the final step which is used to find a font. The output shows the list of font families that are considered and the remaining criteria, specifically the hintstyle, the required charset and the language.

Digging deeper, I wanted to find out what information fontconfig has about the Apple Symbols font.

$ FC_DEBUG=2 fc-match -s :charset=26ad
[…]
Font 7 Pattern has 22 elts (size 22)
        family: "Apple Symbols"(w)
        familylang: "en"(w)
        […]
        charset:
        […]
        0026: ffffffff ffffffff ffffffff ffffffff 1fffffff 0007ffff 00000000 00000000
        […]
        lang: el|fj|ho|ia|ie|io|nr|om|sm|so|ss|st|sw|to|ts|uz|xh|zu|kj|kwm|ms|ng|rn|rw|sn|za(w)
[…]

Here, the FC_DEBUG setting requests »extensive font matching information«.

This snippet shows that the font does indeed contain the requested glyph. However, for some reason fontconfig has not assigned English (en) to the list of languages with which Apple Symbols can be used. I assume that this is the result of some algorithm based on the characters contained in the font. According to FontBook.app, the font itself definitely claims to support English, among many other languages:

Asu, Bemba(bem), Bena(bez), Chiga, Cornish(kw), English(en), Greek(el), Gusii(guz), Indonesian(id), Kalenjin(kln), Kinyarwanda(rw), Luo(luo), Luyia(luy), Machame, Makhuwa-Meetto(mgh), Makonde(kde), Malay(ms), Morisyen, North Ndebele(nd), Nyankole(nyn), Oromo(om), Rombo, Rundi(rn), Rwa, Samburu(saq), Sangu, Shambala(ksb), Shona(sn), Soga(xog), Somali(so), Swahili(sw), Taita(dav), Teso(teo), Uzbek(uz), Vunjo(vun), Zulu(zu)

In order to fix this problem, I added a section to my ~/.config/fontconfig/fonts.conf which assigns the correct language codes to the fontconfig cache for the Apple Symbols font.

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
    <!-- Apple Symbols loses some language tags and just ends up with
         el, fj, ho, ia, ie, io, nr, om, sm, so, ss, st, sw, to, ts, uz,
         xh, zu, kj, kwm, ms, ng, rn, rw, sn, za.

         Full list taken from FontBook.app:

         Asu, Bemba(bem), Bena(bez), Chiga, Cornish(kw), English(en),
         Greek(el), Gusii(guz), Indonesian(id), Kalenjin(kln),
         Kinyarwanda(rw), Luo(luo), Luyia(luy), Machame,
         Makhuwa-Meetto(mgh), Makonde(kde), Malay(ms), Morisyen, North
         Ndebele(nd), Nyankole(nyn), Oromo(om), Rombo, Rundi(rn), Rwa,
         Samburu(saq), Sangu, Shambala(ksb), Shona(sn), Soga(xog),
         Somali(so), Swahili(sw), Taita(dav), Teso(teo), Uzbek(uz),
         Vunjo(vun), Zulu(zu)
     -->
    <match target="scan">
        <test compare="eq" name="family">
            <string>Apple Symbols</string>
        </test>
        <edit binding="same" mode="assign_replace" name="lang">
            <langset>
                <string>bem</string>
                <string>bez</string>
                <string>dav</string>
                <string>el</string>
                <string>en</string>
                <string>fj</string>
                <string>guz</string>
                <string>ho</string>
                <string>ia</string>
                <string>id</string>
                <string>ie</string>
                <string>io</string>
                <string>kde</string>
                <string>kj</string>
                <string>kln</string>
                <string>ksb</string>
                <string>kw</string>
                <string>kwm</string>
                <string>luo</string>
                <string>luy</string>
                <string>mgh</string>
                <string>ms</string>
                <string>nd</string>
                <string>ng</string>
                <string>nr</string>
                <string>nyn</string>
                <string>om</string>
                <string>rn</string>
                <string>rw</string>
                <string>saq</string>
                <string>sm</string>
                <string>sn</string>
                <string>so</string>
                <string>ss</string>
                <string>st</string>
                <string>sw</string>
                <string>teo</string>
                <string>to</string>
                <string>ts</string>
                <string>uz</string>
                <string>vun</string>
                <string>xh</string>
                <string>xog</string>
                <string>za</string>
                <string>zu</string>
            </langset>
        </edit>
    </match>
</fontconfig>

Note that properties of type langset do not support appending, so the full resulting set needs to be specified. Also, the replacement needs to take place during »scanning« when the cache is built in order to affect the information stored for the font. As a consequence, the fontconfig cache must be force-rebuilt because fc-cache doesn't consider changes to the configuration when it decides whether to rebuild a cache.

$ fc-cache -f

Now, fc-match should give the following output:

$ fc-match -s :charset=26ad
Apple Symbols.ttf: "Apple Symbols" "Regular"
LastResort.otf: ".LastResort" "Regular"

Additionally, I also added the following section to my fonts.conf which adds Apple Symbols as a fallback font for the sans-serif family. This slightly increases the likelihood of it being picked. The fact that it was already picked up beforehand shows that when it is really desparate to find a glyph, fontconfig considers all the fonts it knows about and not just ones with a matching family.

    <!-- Consider glyphs from Apple Symbols, e.g. U+26ad, U+26af. -->
    <alias>
        <family>sans-serif</family>
        <default>
            <family>Apple Symbols</family>
        </default>
    </alias>

In the end, I discovered that my app still did not work because it turned out that it wasn't actually using Pango Cairo's fontconfig backend but the CoreText backend instead. After changing this using PANGOCAIRO_BACKEND=fontconfig I was now able to display the glyphs I needed.

Finding a build tool

As someone who actually thinks that make(1) is a wonderful tool if used correctly, that's what I've normally been using. I'm talking about BSD make, mind you, not the abomination that is GNU make.

For those who are unaware, a typical Makefile looks somewhat like this:

PROG_CXX=       hello

.include <bsd.prog.mk>

Since I also take note of my environment, it hasn't escaped me that that's not what most people seem to be doing nowadays. While I don't have much time for people whining about make(1) given its clear, straightforward and problem-oriented syntax, I do believe there is room for improvement and I'm curious what people have been coming up with in the decades since make's inception.

One of the most widely used tools nowadays appears to be CMake. However, looking at the syntax of CMake's language reminds me mostly of Tcl, and that's not a good thing. Passing sequences of strings as arguments where certain strings modify the interpretation of the strings that come after them is not just a glaring lack of design but also neither fault tolerant nor easy to read. It is also based on the idea of (badly) generating code which I find revolting. As if that wasn't bad enough, it also clutters the source tree with cache files and other clutter which is unacceptable for someone refusing to go down the .*ignore route. Next.

The next big candidate that comes up is Boost.Build. Interestingly, the Jam language suffers almost identical shortcomings as CMake's with lists of strings and certain keywords modifying their semantic, making me almost believe that this might be endemic to people designing build systems. Fail.

Another popular option that comes up is Gradle. It requires a JDK in order to run the build process. What is more, it has a feature matrix showing which platforms and compilers it supports – how is that even relevant!? No thanks.

The search continues…

To be evaluated (in no particular order apart from alphabetical):

IPsec VPN for macOS High Sierra and Android Nougat

After switching to OpenWRT from DD-WRT which doesn't support IPsec, this is post describes the settings I use.

Since Android (at least up to Nougat) does not support IKEv2, we will use a certificate-based IKEv1 setup (IPsec Xauth RSA in Android speak).

strongSwan configuration

The configuration is taken from strongSwan's usable examples.

# /etc/ipsec.conf - strongSwan IPsec configuration file

config setup
        #charondebug="asn 2, cfg 2, chd 2, dmn 2, enc 1, esp 2, ike 2, imc 2, imv 2, job 2, knl 2, lib 2, mgr 2, net 2, pts 2, tls 2, tnc 2"

conn %default
        fragmentation=yes
        dpdaction=clear
        dpddelay=30s
        dpdtimeout=90s

        reauth=no
        rekey=no

        leftauth=pubkey
        leftcert=serverCert.pem
        leftid=@vpn.example.com
        leftsubnet=0.0.0.0/0
        leftfirewall=yes

        rightauth=pubkey
        rightsourceip=10.0.2.0/24
        rightdns=10.0.1.1

        # Android Nougat native client.
        # IKE:AES_CBC_256/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024
        # macOS High Sierra Cisco IPsec.
        # IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
        ike=aes256-sha384-sha256-prfsha384-prfsha256-modp2048-modp1024!

        # For some reason, data cannot be transferred using Android's second suggestion.
        # ESP:AES_CBC_256/HMAC_SHA1_96/NO_EXT_SEQ
        # ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ
        esp=aes256-sha1!

conn pubkey
        auto=add

conn xauth-pubkey
        # Android's native client requires XAuth in addition to certificates.
        rightauth2=xauth
        auto=add

conn xauth-hybrid
        leftsendcert=always
        rightauth=xauth
        auto=add

conn ipsecuritas
        # IPSecuritas apparently needs a specific IP or the CHILD_SA cannot be found.
        rightsubnet=10.0.3.1/32
        auto=add

In order to prevent routing headaches, it's advisable to assign addresses to IPsec clients (rightsourceip) from a different subnet than the LAN (10.0.1.0/24 in this example).

I did not set up IKEv2 connections (for use on macOS, for example) because that would require installing additional modules for EAP, such as eap-md5 or eap-tls.

# /etc/ipsec.secrets - strongSwan IPsec secrets file

: RSA moonKey.pem
client : XAUTH "password"

I have chosen the legacy ipsec configuration in favour of swanctl because the latter is a much bigger packet to install.

Firewall configuration

Confusingly, OpenWRT /etc/config/firewall already has a couple of rules for IPsec (Allow-IPSec-ESP and Allow-ISAKMP), but another important one is missing. Moreover, as of this writing these rules handle IPsec forwarding to the LAN. In order for the router to be able to receive IPsec traffic, the option dest lan must be dropped. Here's the result:

config rule
        option name             Allow-IPSec-ESP
        option src              wan
        #option dest             lan
        option proto            esp
        option target           ACCEPT

config rule
        option name             Allow-ISAKMP
        option src              wan
        #option dest             lan
        option proto            udp
        option dest_port        500
        option target           ACCEPT

config rule
        option name             Allow-NATT
        option src              wan
        option proto            udp
        option dest_port        4500
        option target           ACCEPT

The WAN interface needs to accept packets unwrapped from IPsec tunnels which can be recognised by their »ipsec« policy using the »policy module«. In order for clients connected via IPsec to be reachable from the LAN, packets bound for an IPsec tunnel must not be subjected to NAT. This can be accomplished using the following rule in /etc/firewall.user, for example:

# Accept tunnelled traffic.
iptables \
        -t filter \
        -A input_wan_rule \
        -m policy \
        --dir in \
        --pol ipsec \
        --mode tunnel \
        -j ACCEPT \

# Prevent NATting traffic bound for IPsec tunnel.
iptables \
        -t nat \
        -A postrouting_wan_rule \
        -m policy \
        --dir out \
        --pol ipsec \
        --mode tunnel \
        -j ACCEPT \

Debugging

Activating the line charondebug in /etc/ipsec.conf (above) or adding the following section to /etc/strongswan.d/charon-logging.conf is helpful when debugging issues:

charon {
    syslog {
        daemon {
            default = 1
        }
    }
}
# logread -f

The following commands can help with routing and firewall debugging:

# ip -f inet route list table all
# iptables -t [filter|nat] -L -v
# iptables -t filter -I reject --log-prefix 'reject ' -j LOG
# /etc/init.d/firewall stop

Other settings

In order to provide DNS to VPN clients, the DNS server must not be configured as »local only« (localservice=0).

config dnsmasq
        option domainneeded      1
        option rebind_protection 1
        option rebind_localhost  1
        option local             /lan/
        option domain            lan
        option authoritative     1
        option readethers        1
        option leasefile         /tmp/dhcp.leases
        option resolvfile        /tmp/resolv.conf.auto
        option nonwildcard       1
        option localservice      0
        option add_local_fqdn    3

Key and certificate generation

Generating certificates requires strongswan-pki which is a separate install. First, generate the key for your certificate authority.

# ipsec pki --gen --outform pem >caKey.pem

The CA key is particularly sensitive and must never leak. Next, create a certificate for your certificate authority.

# cacert=/etc/ipsec.d/cacerts/caCert.pem
# ipsec pki --self \
        --in caKey.pem \
        --dn 'C=CH, O=rienajouter, CN=rienajouter CA' \
        --ca \
        --outform pem \
        >"$cacert"

The »distinguished name« for every certificate you create must be unique – otherwise the certificates will be considered interchangeable and authentication will fail. Next, create a key and certificate for the VPN server.

# key=/etc/ipsec.d/private/serverKey.pem
# cert=/etc/ipsec.d/certs/serverCert.pem
# ipsec pki --gen --outform pem >"$key"
# ipsec pki --pub --in "$key" \
| ipsec pki --issue \
        --cacert "$cacert" \
        --cakey caKey.pem \
        --dn 'C=CH, O=rienajouter, CN=vpn.example.com' \
        --san 'vpn.example.com' \
        --flag serverAuth \
        --flag ikeIntermediate \
        --outform pem \
        >"$cert"

For the VPN server certificate to work reliably with macOS, the CN of the distinguished name as well as the subject alt name (--san) must match the DNS name of the VPN gateway and the ikeIntermediate flag must be present. Windows on the other hand requires the serverAuth flag.

Last, for each client create a certificate to be shared with the client.

# key=/etc/ipsec.d/private/clientKey.pem
# cert=/etc/ipsec.d/certs/clientCert.pem
# ipsec pki --gen --outform pem >"$key"
# ipsec pki --pub --in "$key" \
| ipsec pki --issue \
        --cacert "$cacert" \
        --cakey caKey.pem \
        --dn 'C=CH, O=rienajouter, CN=client' \
        --outform pem \
        >"$cert"

Generating keys and certificates in PEM format helps when exporting them to the clients.

Doing the same with openssl is a nightmare because openssl was created in order to prevent the widespread use of cryptography, and quite successfully so far.

$ openssl req \
        -new \
        -newkey rsa:2048 \
        -subj '/C=CH/O=rienajouter/CN=client' \
        -keyout "$key" \
        -nodes \
| openssl x509 \
        -req \
        -CA "$cacert"  \
        -CAkey caKey.pem \
        -sha256 \
        -days $((3 * 365)) \
        >"$cert"

In case you forget, here's how to find out what's inside one of the key or certificate files:

$ ipsec pki --print --in "$cert"
$ openssl rsa -in "$key" -text -noout
$ openssl x509 -in "$cert" -text -noout

Certificate distribution

In order to distribute the certificates generated in the previous step, they must be packaged together into PKCS#12 files. The command to do so is:

$ openssl pkcs12 \
        -certfile "$cacert" \
        -inkey "$key" \
        -in "$cert" \
        -export \
        -out client.p12 \

When exporting for macOS, make sure to use a non-empty password when prompted by the above command. After importing the CA and client certificates into the system keychain on macOS, set the CA certificate's »trust« for »IPsec« to »always trust«. You also need to grant access to the private key inside the client certificate to /usr/sbin/racoon. The »access control« tab is only shown for keys, which in turn only appear when selecting »keys« or »certificates« in the left-hand pane of »Keychain Access«.