site
my personal website
git clone https://jhhm.no/repos/site.git
site/content/writings/writing-an-openbsd-daemon/index.md
get raw file: index.md
1; Writing an OpenBSD Daemon 2; openbsd go tech *nix 3; Mon Nov 16 10:02:57 AM CET 2026 4 5# Writing an OpenBSD Daemon 6 7I recently{^At least it was recently when I started writing this!} migrated my server to OpenBSD, and decided to put some polish on my Twitch RSS generator [tweed](/repos/tweed/). It's a fairly simple Go program that makes some HTTP requests and serves RSS files via HTTP. The old version ran out of its working directory, reading its config and storing its data there. With the new update my goal was to have the program follow a more standard Unix daemon layout, keep its files in the conventional places, and run as a daemon user. Lastly, I wanted to do some OpenBSD-specific hardening using pledge and unveil. 8 9## Logging 10 11Traditionally, Unix daemons log using syslog. Syslog has been the standard on Unixes for over 40 years, and got standardised in 2009 by [RFC 5424](https://www.rfc-editor.org/rfc/rfc5424.txt). Using it in Go is simple: just import '''log/syslog''', call '''syslog.New''' to create your syslog writer, and start logging. 12 13In my program, I open syslog with the daemon facility. Log calls go through a helper function that falls back to stderr when syslog isn't opened (more on that later). 14 15''' 16func initSyslog() error { 17 w, err := syslog.New(syslog.LOG_DAEMON|syslog.LOG_INFO, "tweed") 18 if err != nil { 19 return err 20 } 21 sysLog = w 22 return nil 23} 24 25func logAt(pri syslog.Priority, format string, args ...any) { 26 msg := fmt.Sprintf(format, args...) 27 if sysLog == nil { 28 fmt.Fprintln(os.Stderr, msg) 29 return 30 } 31 switch pri { 32 case syslog.LOG_ERR: 33 sysLog.Err(msg) 34 case syslog.LOG_WARNING: 35 sysLog.Warning(msg) 36 default: 37 sysLog.Info(msg) 38 } 39} 40''' 41 42What happens to these messages depends on how your syslog daemon is configured. On OpenBSD they mostly end up in '''/var/log/''', and are rotated using [newsyslog](https://man.openbsd.org/newsyslog.8). On systemd systems they are stored as binary files, and viewed using journalctl. 43 44Lastly, I added a flag to log directly to the terminal instead. This is done by simply not calling '''initSyslog''', making it fall back to stderr. It's a useful debugging feature I have seen on several OpenBSD daemons like [httpd](https://man.openbsd.org/httpd.8) and [sshd](https://man.openbsd.org/sshd.8), and it made setting up the daemon a nicer experience. 45 46''' 47if !*debug { 48 if err := initSyslog(); err != nil { 49 fatal("syslog: %v", err) 50 } 51} 52''' 53 54## Config 55 56The standard place for configuration files on most Unix-like systems is '''/etc/'''. NetBSD and FreeBSD both store non-base configuration files in the '''/usr/''' directory (in different places, of course), so I made the program search for its configuration files in multiple places.{^I only look for '''/etc/tweed/''' and '''/usr/local/etc/tweed/''', but might add more if I ever feel like it, or if someone else feels like it!} Importantly, I decided to not have my daemon create any directories on its own. I see this step as a part of the install process, which the user should do deliberately. 57 58''' 59var defaultConfigDirs = []string{ 60 "/etc/tweed", 61 "/usr/local/etc/tweed", 62} 63 64func findConfigDir(given string) (string, error) { 65 if given != "" { 66 return given, nil 67 } 68 for _, d := range defaultConfigDirs { 69 if _, err := os.Stat(filepath.Join(d, "config.ini")); !errors.Is(err, os.ErrNotExist) { 70 return d, nil 71 } 72 } 73 return "", fmt.Errorf("no config.ini found (looked in %s)", 74 strings.Join(defaultConfigDirs, ", ")) 75} 76''' 77 78The config file is a plain key-value file with INI syntax. 79 80''' 81# /etc/tweed/config.ini 82appId = ... 83appSecret = ... 84channels = foo, bar 85port = 8081 86dataDir = /var/www/tweed/ 87''' 88 89I also decided to add a config flag to the program, allowing the user to set a custom config directory at runtime (this is the '''given''' argument to '''findConfigDir'''). This gives at least basic support for systems that keep configuration somewhere else. 90 91## Data 92 93The other place our program touches the filesystem is its data directory. It creates a feed directory inside of the data directory, and writes and serves RSS files in it. Again, where data belongs depends on the OS and what type of data it is. Persistent state, like our RSS files, is usually stored somewhere in '''/var/'''. On OpenBSD I believe '''/var/db/''' is preferred, while on Linux '''/var/lib/''' seems to be a common choice. I also made a configuration option to set a custom data directory (see above example config). 94 95The code is quite similar to the config handling. 96 97''' 98var defaultDataDirs = []string{ 99 "/var/db/tweed", 100 "/var/lib/tweed", 101} 102 103func findDataDir() (string, error) { 104 for _, d := range defaultDataDirs { 105 fi, err := os.Stat(d) 106 if errors.Is(err, fs.ErrNotExist) { 107 continue 108 } 109 if err != nil { 110 return "", fmt.Errorf("data dir %s: %w", d, err) 111 } 112 if !fi.IsDir() { 113 return "", fmt.Errorf("data dir %s: not a directory", d) 114 } 115 return d, nil 116 } 117 return "", fmt.Errorf("no data directory found (looked for %s)", 118 strings.Join(defaultDataDirs, ", ")) 119} 120''' 121 122## Protection Against Multiple Instances 123 124I wanted to make sure the program never leaves the data directory in an invalid state, since the RSS feeds are naïvely served directly from the filesystem. My first mitigation was to write each feed to a temporary file, then rename it into its final position. Since a rename within the same filesystem is atomic, anyone reading the feed sees either the complete old file or the complete new one, never a half-written file. 125 126That doesn’t help, however, if a second instance is writing to the same data directory. The two could end up writing to the same temporary file at once, leaving a corrupted feed behind. The flock syscall solves exactly this issue. Calling a non-blocking exclusive flock on a file will fail if another process has already called flock on it, and that process is still running. 127 128''' 129if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { 130 f.Close() 131 if errors.Is(err, syscall.EWOULDBLOCK) { 132 return nil, fmt.Errorf("another tweed instance is using data dir %s", dir) 133 } 134 return nil, fmt.Errorf("data dir %s: locking %s: %w", dir, path, err) 135} 136''' 137 138We do this check at the start of the program, and exit the program with an error if another instance is using the same data directory. 139 140## Proper Signal Handling 141 142In order for a daemon to be easily managed by a service control utility, like [rcctl](https://man.openbsd.org/rcctl.8) on OpenBSD or systemctl under systemd, it should shut down gracefully on SIGTERM. For tweed, this means letting in-flight HTTP requests finish and leaving all RSS files in a valid state. 143 144For the implementation, SIGINT / SIGTERM cancel '''ctx''', and the goroutine that monitors Twitch channels checks '''ctx''' between making calls and while sleeping. '''main''' blocks until the context is cancelled, gives in-flight requests up to 5 seconds to finish, then waits for the monitor to return. 145 146''' 147ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) 148defer stop() 149 150go func() { 151 defer close(monitorDone) 152 monitor(ctx, &cfg) 153}() 154 155<-ctx.Done() 156 157shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) 158defer cancel() 159srv.Shutdown(shutdownCtx) 160<-monitorDone 161''' 162 163Inside the monitor loop the sleep is a select, so a signal cuts it short. 164 165''' 166select { 167case <-ctx.Done(): 168 return 169case <-time.After(time.Duration(cfg.RefreshMinutes) * time.Minute): 170} 171''' 172 173I decided against handling SIGHUP (reload), since restarting the program has the same effect, and the startup time is instantaneous. This means that a SIGHUP will terminate the program though, so don't send it SIGHUP expecting a reload. 174 175## Pledge and Unveil 176 177I first encountered OpenBSD's security function [pledge](https://man.openbsd.org/pledge.2) while hacking on my dwm build. I was rewriting the core event loop to use poll, but my changes made the program brutally crash on my OpenBSD laptop. I remember being quite puzzled at the time, but after looking at the stack trace and some man pages I eventually learned all about how pledge (and [unveil](https://man.openbsd.org/unveil.2)) worked. 178 179Pledge lets a program limit itself to named groups of system calls (like '''stdio''' or '''inet'''), making anything it hasn't promised forbidden. Unveil works much the same way, just for the filesystem. Basically, once the program has read its configuration file and is ready to enter its main lifecycle loop, we restrict its capabilities to the bare minimum required for operating. If any of our promises are broken, like accessing a file outside of those unveiled, or making a call to a non-pledged subsystem, the program is immediately killed. This is a simple yet effective security measure to prevent programs from going haywire if something unexpected occurs, or they become compromised. 180 181Calling pledge and unveil from Go is fairly simple with the [sys/unix](https://pkg.go.dev/golang.org/x/sys/unix) package. Since we only want to include these calls when compiling for OpenBSD, we have to use Go's build constraints. We start by creating a file '''sandbox_openbsd.go''', which Go only compiles when targeting OpenBSD (because of its '''_openbsd''' suffix). 182 183''' 184package main 185 186import "golang.org/x/sys/unix" 187 188func unveil(path, perms string) error { return unix.Unveil(path, perms) } 189func unveilBlock() error { return unix.UnveilBlock() } 190func pledge(promises string) error { return unix.PledgePromises(promises) } 191''' 192 193Next comes '''sandbox_other.go''', whose '''//go:build !openbsd''' line includes it on every other target. The functions are just defined as stubs in this case.{^I have to say that I find all of this less elegant than a simple '''#ifdef __OpenBSD__''', but I can see the pros of not having macros, even if I prefer having them.} 194 195''' 196//go:build !openbsd 197 198package main 199 200func unveil(string, string) error { return nil } 201func unveilBlock() error { return nil } 202func pledge(string) error { return nil } 203''' 204 205With our functions defined for every target, we can call them from our sandbox function. 206 207''' 208func sandbox(cfg *Config) error { 209 _ = time.Local.String() 210 211 paths := []struct{ path, perms string }{ 212 {filepath.Join(cfg.DataDir, "feeds"), "rwc"}, 213 {"/etc/ssl/cert.pem", "r"}, 214 {"/etc/resolv.conf", "r"}, 215 {"/etc/hosts", "r"}, 216 {"/dev/log", "rw"}, 217 } 218 for _, p := range paths { 219 if err := unveil(p.path, p.perms); err != nil { 220 return fmt.Errorf("unveil %s: %w", p.path, err) 221 } 222 } 223 if err := unveilBlock(); err != nil { 224 return fmt.Errorf("unveil block: %w", err) 225 } 226 if err := pledge("stdio rpath wpath cpath fattr inet dns unix"); err != nil { 227 return fmt.Errorf("pledge: %w", err) 228 } 229 return nil 230} 231''' 232 233We start by unveiling the paths that we need in order to run, with the needed file permissions. We need to be able to read, write and create files in our feeds directory, read the given files in '''/etc/''' for networking purposes, and read and write to '''/dev/log''' for syslog. Note that since we call '''sandbox''' after reading our configuration files, we don't need to access them anymore. Then, we call '''unveilBlock''', blocking all further attempts to unveil any paths. 234 235Our pledge call includes all of the needed subsystems that we need to access. '''stdio''' for stdio (I know, shocking), '''rpath''', '''wpath''' and '''cpath''' for reading, writing, creating and removing files. '''fattr''' for chmod, '''inet''' and '''dns''' for network stuff, and lastly '''unix''' for syslog's socket. That's a fairly broad set of promises, so here unveil probably does most of the protection. 236 237## To Double Fork or not to Double Fork 238 239Traditionally, Unix daemons do a thing called double forking. They do this to escape the launching shell's session and controlling terminal, and to avoid being a session leader. I won't go into why,{^I would probably explain it worse than any other online resource, since I have literally never done it!} since the advantages of doing it on modern systems are slim to none. It's also my understanding that most modern service managers advise against doing it, since they can / want to handle detaching themselves. 240 241Unsurprisingly, given its heritage, the practice is closely tied to C programming and doesn't easily translate to Go. Go's runtime is multithreaded from startup, so in order to reproduce the same result you would have to re-execute the binary as a detached child, I believe. This is not very pretty, and since double forking in the year of our lord 2026 is dubious to begin with, I decided to not double fork. 242 243## Setting Up the Daemon 244 245The last thing to do is to actually run the thing. I built the program for OpenBSD by 246specifying the target OS (and architecture for my server). 247 248''' 249GOOS=openbsd GOARCH=amd64 go build -o tweed 250''' 251 252Then I copied it to the server with [scp](https://man.openbsd.org/scp.1), and installed it to its final location. 253 254''' 255install -o root -g bin -m 755 tweed /usr/local/sbin/tweed 256''' 257 258It's good practice to run a daemon, especially a network-facing one, as its own user. 259 260''' 261useradd -L daemon -g =uid -d /var/empty -s /sbin/nologin -c "tweed daemon" _tweed 262''' 263 264This creates a user with login class '''daemon''', a group of the same name and id, an empty home directory and no login shell. 265 266Next, I create the configuration directory and the config file. The permissions let only root edit the files, while members of the '''_tweed''' group can read them. Other users should not be allowed to read the files, since the config contains secrets. 267 268''' 269install -d -o root -g _tweed -m 750 /etc/tweed 270install -o root -g _tweed -m 640 config.ini /etc/tweed/ 271''' 272 273While setting up the daemon I had a realisation that I probably should have had a long time ago: httpd can serve the RSS files. On my old server I had tweed serve them on its own, but that's a whole other thing to set up and have fail. With this realisation I made a configuration option for serving feeds, set the data dir to be in '''/var/www/''', and made httpd serve the files. 274 275''' 276appId = ... 277appSecret = ... 278 279refreshMinutes = 5 280ttlMinutes = 5 281 282serveFeeds = false 283dataDir = /var/www/tweed/ 284... 285''' 286 287Finally, I wrote an rcctl service file for the daemon. It sets the daemon user, runs tweed in the background, and disables reload (since we don't handle SIGHUP). 288 289''' 290#!/bin/ksh 291 292daemon="/usr/local/sbin/tweed" 293daemon_user="_tweed" 294 295. /etc/rc.d/rc.subr 296 297rc_bg=YES 298rc_reload=NO 299 300rc_cmd $1 301''' 302 303## Takeaways 304 305As always, I greatly underestimated how much work the polishing would take. I fell right into the [ninety-ninety rule](https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule): the first 90% of the code takes the first 90% of the time, and the last 10% takes the other 90%. This sort of made me question whether it was even worth it. After all, the program had been running on my old server for about a year without any problems. I definitely learned a lot about writing daemons through this process though, and hopefully I can take that with me to future projects.