JHHM

site

my personal website
git clone https://jhhm.no/repos/site.git

site/content/writings/writing-an-openbsd-daemon/index.md

get raw file: index.md

1; Writing an OpenBSD Daemon
2; openbsd go tech *nix
3; Mon Nov 16 10:02:57 AM CET 2026
4
5# Writing an OpenBSD Daemon
6
7I recently{^At least it was recently when I started writing this!} migrated my server to OpenBSD, and decided to put some polish on my Twitch RSS generator [tweed](/repos/tweed/). It's a fairly simple Go program that makes some HTTP requests and serves RSS files via HTTP. The old version ran out of its working directory, reading its config and storing its data there. With the new update my goal was to have the program follow a more standard Unix daemon layout, keep its files in the conventional places, and run as a daemon user. Lastly, I wanted to do some OpenBSD-specific hardening using pledge and unveil.
8
9## Logging
10
11Traditionally, Unix daemons log using syslog. Syslog has been the standard on Unixes for over 40 years, and got standardised in 2009 by [RFC 5424](https://www.rfc-editor.org/rfc/rfc5424.txt). Using it in Go is simple: just import '''log/syslog''', call '''syslog.New''' to create your syslog writer, and start logging.
12
13In my program, I open syslog with the daemon facility. Log calls go through a helper function that falls back to stderr when syslog isn't opened (more on that later).
14
15'''
16func initSyslog() error {
17	w, err := syslog.New(syslog.LOG_DAEMON|syslog.LOG_INFO, "tweed")
18	if err != nil {
19		return err
20	}
21	sysLog = w
22	return nil
23}
24
25func logAt(pri syslog.Priority, format string, args ...any) {
26	msg := fmt.Sprintf(format, args...)
27	if sysLog == nil {
28		fmt.Fprintln(os.Stderr, msg)
29		return
30	}
31	switch pri {
32	case syslog.LOG_ERR:
33		sysLog.Err(msg)
34	case syslog.LOG_WARNING:
35		sysLog.Warning(msg)
36	default:
37		sysLog.Info(msg)
38	}
39}
40'''
41
42What happens to these messages depends on how your syslog daemon is configured. On OpenBSD they mostly end up in '''/var/log/''', and are rotated using [newsyslog](https://man.openbsd.org/newsyslog.8). On systemd systems they are stored as binary files, and viewed using journalctl.
43
44Lastly, I added a flag to log directly to the terminal instead. This is done by simply not calling '''initSyslog''', making it fall back to stderr. It's a useful debugging feature I have seen on several OpenBSD daemons like [httpd](https://man.openbsd.org/httpd.8) and [sshd](https://man.openbsd.org/sshd.8), and it made setting up the daemon a nicer experience.
45
46'''
47if !*debug {
48	if err := initSyslog(); err != nil {
49		fatal("syslog: %v", err)
50	}
51}
52'''
53
54## Config
55
56The standard place for configuration files on most Unix-like systems is '''/etc/'''. NetBSD and FreeBSD both store non-base configuration files in the '''/usr/''' directory (in different places, of course), so I made the program search for its configuration files in multiple places.{^I only look for '''/etc/tweed/''' and '''/usr/local/etc/tweed/''', but might add more if I ever feel like it, or if someone else feels like it!} Importantly, I decided to not have my daemon create any directories on its own. I see this step as a part of the install process, which the user should do deliberately.
57
58'''
59var defaultConfigDirs = []string{
60	"/etc/tweed",
61	"/usr/local/etc/tweed",
62}
63
64func findConfigDir(given string) (string, error) {
65	if given != "" {
66		return given, nil
67	}
68	for _, d := range defaultConfigDirs {
69		if _, err := os.Stat(filepath.Join(d, "config.ini")); !errors.Is(err, os.ErrNotExist) {
70			return d, nil
71		}
72	}
73	return "", fmt.Errorf("no config.ini found (looked in %s)",
74		strings.Join(defaultConfigDirs, ", "))
75}
76'''
77
78The config file is a plain key-value file with INI syntax.
79
80'''
81# /etc/tweed/config.ini
82appId = ...
83appSecret = ...
84channels = foo, bar
85port = 8081
86dataDir = /var/www/tweed/
87'''
88
89I also decided to add a config flag to the program, allowing the user to set a custom config directory at runtime (this is the '''given''' argument to '''findConfigDir'''). This gives at least basic support for systems that keep configuration somewhere else.
90
91## Data
92
93The other place our program touches the filesystem is its data directory. It creates a feed directory inside of the data directory, and writes and serves RSS files in it. Again, where data belongs depends on the OS and what type of data it is. Persistent state, like our RSS files, is usually stored somewhere in '''/var/'''. On OpenBSD I believe '''/var/db/''' is preferred, while on Linux '''/var/lib/''' seems to be a common choice. I also made a configuration option to set a custom data directory (see above example config).
94
95The code is quite similar to the config handling.
96
97'''
98var defaultDataDirs = []string{
99	"/var/db/tweed",
100	"/var/lib/tweed",
101}
102
103func findDataDir() (string, error) {
104	for _, d := range defaultDataDirs {
105		fi, err := os.Stat(d)
106		if errors.Is(err, fs.ErrNotExist) {
107			continue
108		}
109		if err != nil {
110			return "", fmt.Errorf("data dir %s: %w", d, err)
111		}
112		if !fi.IsDir() {
113			return "", fmt.Errorf("data dir %s: not a directory", d)
114		}
115		return d, nil
116	}
117	return "", fmt.Errorf("no data directory found (looked for %s)",
118		strings.Join(defaultDataDirs, ", "))
119}
120'''
121
122## Protection Against Multiple Instances
123
124I wanted to make sure the program never leaves the data directory in an invalid state, since the RSS feeds are naïvely served directly from the filesystem. My first mitigation was to write each feed to a temporary file, then rename it into its final position. Since a rename within the same filesystem is atomic, anyone reading the feed sees either the complete old file or the complete new one, never a half-written file.
125
126That doesn’t help, however, if a second instance is writing to the same data directory. The two could end up writing to the same temporary file at once, leaving a corrupted feed behind. The flock syscall solves exactly this issue. Calling a non-blocking exclusive flock on a file will fail if another process has already called flock on it, and that process is still running.
127
128'''
129if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil {
130	f.Close()
131	if errors.Is(err, syscall.EWOULDBLOCK) {
132		return nil, fmt.Errorf("another tweed instance is using data dir %s", dir)
133	}
134	return nil, fmt.Errorf("data dir %s: locking %s: %w", dir, path, err)
135}
136'''
137
138We do this check at the start of the program, and exit the program with an error if another instance is using the same data directory.
139
140## Proper Signal Handling
141
142In order for a daemon to be easily managed by a service control utility, like [rcctl](https://man.openbsd.org/rcctl.8) on OpenBSD or systemctl under systemd, it should shut down gracefully on SIGTERM. For tweed, this means letting in-flight HTTP requests finish and leaving all RSS files in a valid state.
143
144For the implementation, SIGINT / SIGTERM cancel '''ctx''', and the goroutine that monitors Twitch channels checks '''ctx''' between making calls and while sleeping. '''main''' blocks until the context is cancelled, gives in-flight requests up to 5 seconds to finish, then waits for the monitor to return.
145
146'''
147ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
148defer stop()
149
150go func() {
151	defer close(monitorDone)
152	monitor(ctx, &cfg)
153}()
154
155<-ctx.Done()
156
157shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
158defer cancel()
159srv.Shutdown(shutdownCtx)
160<-monitorDone
161'''
162
163Inside the monitor loop the sleep is a select, so a signal cuts it short.
164
165'''
166select {
167case <-ctx.Done():
168	return
169case <-time.After(time.Duration(cfg.RefreshMinutes) * time.Minute):
170}
171'''
172
173I decided against handling SIGHUP (reload), since restarting the program has the same effect, and the startup time is instantaneous. This means that a SIGHUP will terminate the program though, so don't send it SIGHUP expecting a reload.
174
175## Pledge and Unveil
176
177I first encountered OpenBSD's security function [pledge](https://man.openbsd.org/pledge.2) while hacking on my dwm build. I was rewriting the core event loop to use poll, but my changes made the program brutally crash on my OpenBSD laptop. I remember being quite puzzled at the time, but after looking at the stack trace and some man pages I eventually learned all about how pledge (and [unveil](https://man.openbsd.org/unveil.2)) worked.
178
179Pledge lets a program limit itself to named groups of system calls (like '''stdio''' or '''inet'''), making anything it hasn't promised forbidden. Unveil works much the same way, just for the filesystem. Basically, once the program has read its configuration file and is ready to enter its main lifecycle loop, we restrict its capabilities to the bare minimum required for operating. If any of our promises are broken, like accessing a file outside of those unveiled, or making a call to a non-pledged subsystem, the program is immediately killed. This is a simple yet effective security measure to prevent programs from going haywire if something unexpected occurs, or they become compromised.
180
181Calling pledge and unveil from Go is fairly simple with the [sys/unix](https://pkg.go.dev/golang.org/x/sys/unix) package. Since we only want to include these calls when compiling for OpenBSD, we have to use Go's build constraints. We start by creating a file '''sandbox_openbsd.go''', which Go only compiles when targeting OpenBSD (because of its '''_openbsd''' suffix).
182
183'''
184package main
185
186import "golang.org/x/sys/unix"
187
188func unveil(path, perms string) error { return unix.Unveil(path, perms) }
189func unveilBlock() error { return unix.UnveilBlock() }
190func pledge(promises string) error { return unix.PledgePromises(promises) }
191'''
192
193Next comes '''sandbox_other.go''', whose '''//go:build !openbsd''' line includes it on every other target. The functions are just defined as stubs in this case.{^I have to say that I find all of this less elegant than a simple '''#ifdef __OpenBSD__''', but I can see the pros of not having macros, even if I prefer having them.}
194
195'''
196//go:build !openbsd
197
198package main
199
200func unveil(string, string) error { return nil }
201func unveilBlock() error { return nil }
202func pledge(string) error { return nil }
203'''
204
205With our functions defined for every target, we can call them from our sandbox function.
206
207'''
208func sandbox(cfg *Config) error {
209	_ = time.Local.String()
210
211	paths := []struct{ path, perms string }{
212		{filepath.Join(cfg.DataDir, "feeds"), "rwc"},
213		{"/etc/ssl/cert.pem", "r"},
214		{"/etc/resolv.conf", "r"},
215		{"/etc/hosts", "r"},
216		{"/dev/log", "rw"},
217	}
218	for _, p := range paths {
219		if err := unveil(p.path, p.perms); err != nil {
220			return fmt.Errorf("unveil %s: %w", p.path, err)
221		}
222	}
223	if err := unveilBlock(); err != nil {
224		return fmt.Errorf("unveil block: %w", err)
225	}
226	if err := pledge("stdio rpath wpath cpath fattr inet dns unix"); err != nil {
227		return fmt.Errorf("pledge: %w", err)
228	}
229	return nil
230}
231'''
232
233We start by unveiling the paths that we need in order to run, with the needed file permissions. We need to be able to read, write and create files in our feeds directory, read the given files in '''/etc/''' for networking purposes, and read and write to '''/dev/log''' for syslog. Note that since we call '''sandbox''' after reading our configuration files, we don't need to access them anymore. Then, we call '''unveilBlock''', blocking all further attempts to unveil any paths.
234
235Our pledge call includes all of the needed subsystems that we need to access. '''stdio''' for stdio (I know, shocking), '''rpath''', '''wpath''' and '''cpath''' for reading, writing, creating and removing files. '''fattr''' for chmod, '''inet''' and '''dns''' for network stuff, and lastly '''unix''' for syslog's socket. That's a fairly broad set of promises, so here unveil probably does most of the protection.
236
237## To Double Fork or not to Double Fork
238
239Traditionally, Unix daemons do a thing called double forking. They do this to escape the launching shell's session and controlling terminal, and to avoid being a session leader. I won't go into why,{^I would probably explain it worse than any other online resource, since I have literally never done it!} since the advantages of doing it on modern systems are slim to none. It's also my understanding that most modern service managers advise against doing it, since they can / want to handle detaching themselves.
240
241Unsurprisingly, given its heritage, the practice is closely tied to C programming and doesn't easily translate to Go. Go's runtime is multithreaded from startup, so in order to reproduce the same result you would have to re-execute the binary as a detached child, I believe. This is not very pretty, and since double forking in the year of our lord 2026 is dubious to begin with, I decided to not double fork.
242
243## Setting Up the Daemon
244
245The last thing to do is to actually run the thing. I built the program for OpenBSD by
246specifying the target OS (and architecture for my server).
247
248'''
249GOOS=openbsd GOARCH=amd64 go build -o tweed
250'''
251
252Then I copied it to the server with [scp](https://man.openbsd.org/scp.1), and installed it to its final location.
253
254'''
255install -o root -g bin -m 755 tweed /usr/local/sbin/tweed
256'''
257
258It's good practice to run a daemon, especially a network-facing one, as its own user.
259
260'''
261useradd -L daemon -g =uid -d /var/empty -s /sbin/nologin -c "tweed daemon" _tweed
262'''
263
264This creates a user with login class '''daemon''', a group of the same name and id, an empty home directory and no login shell.
265
266Next, I create the configuration directory and the config file. The permissions let only root edit the files, while members of the '''_tweed''' group can read them. Other users should not be allowed to read the files, since the config contains secrets.
267
268'''
269install -d -o root -g _tweed -m 750 /etc/tweed
270install -o root -g _tweed -m 640 config.ini /etc/tweed/
271'''
272
273While setting up the daemon I had a realisation that I probably should have had a long time ago: httpd can serve the RSS files. On my old server I had tweed serve them on its own, but that's a whole other thing to set up and have fail. With this realisation I made a configuration option for serving feeds, set the data dir to be in '''/var/www/''', and made httpd serve the files.
274
275'''
276appId = ...
277appSecret = ...
278
279refreshMinutes = 5
280ttlMinutes = 5
281
282serveFeeds = false
283dataDir = /var/www/tweed/
284...
285'''
286
287Finally, I wrote an rcctl service file for the daemon. It sets the daemon user, runs tweed in the background, and disables reload (since we don't handle SIGHUP).
288
289'''
290#!/bin/ksh
291
292daemon="/usr/local/sbin/tweed"
293daemon_user="_tweed"
294
295. /etc/rc.d/rc.subr
296
297rc_bg=YES
298rc_reload=NO
299
300rc_cmd $1
301'''
302
303## Takeaways
304
305As always, I greatly underestimated how much work the polishing would take. I fell right into the [ninety-ninety rule](https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule): the first 90% of the code takes the first 90% of the time, and the last 10% takes the other 90%. This sort of made me question whether it was even worth it. After all, the program had been running on my old server for about a year without any problems. I definitely learned a lot about writing daemons through this process though, and hopefully I can take that with me to future projects.