Hunchentoot, Common LISP and NixOS

Posted on January 2, 2024

Synopsis

Common LISP is a really smart language, and it stimulates the brain of its users, especially with Emacs and Sly. However, I’ve encountered with many types of trivial problems while just setting up its working environment, due to its relatively small userbase.

Along the way, I found out that Nix is a great tool for building a reproducible development/build environment. While it may be tricky in the first place (nix expressions etc), once you figure out the correct expression for your build environment, it describes the build environment better than any kind of documentation written in our natural language.

Hence, I started to describe my setup only with Nix expressions, Elisp, and just one .sbclrc file.

Additionally, I have to mention that building a small web server is the best way for many languages to start its introduction. So, I’m dropping one here a small prescription with Hunchentoot (aka tbnl, it’s old name), one of the best web server framework for CL.

Shared Library Object

In nix, it is not recommended (more like, not allowed) to access the shared library in a “shared” way. Therefore, in case of Hunchentoot, which requires libcrypto.so, will not work with quicklisp, even if you specify the openssl package in your home manager.

Each project’s shell.nix specifies the package which can provide the shared object file. That’s the recommended, flawless way to do it.

SBCL Initialization

To make this work, first put the following .sbclrc in $HOME.

;; https://discourse.nixos.org/t/tip-for-sbcl-quicklisp-and-library-dependencies-with-pkg-config/12065
#+asdf (require :asdf)
(ql:quickload :cffi :silent 'nil)
(pushnew (merge-pathnames ".nix-profile/lib/" (user-homedir-pathname))
         cffi:*foreign-library-directories*)
(pushnew (merge-pathnames ".nix-profile/lib64/" (user-homedir-pathname))
         cffi:*foreign-library-directories*)

Nix will place the libraries provided by packages specified in shell.nix appropriately when it is required (to be more specific, if you’re using direnv with shell.nix, all the libraries will stay in ~/.nix-profile while you’re in the project directory).

To locate and use search and destroy them, .sbclrc specifies the nix-specific library locations. I believe sometimes it just works even without this rc file, but having this file makes sure your sbcl knows everything it needs, regardless of its build version.

Roswell

Roswell is a handy tool to manage various versions of sbcl. Of course, mere Nix can handle this sort of job as well. However, I made a choice due to Roswell providing an easier way of setting up quicklisp. If you install a Roswell helper, you will load the script ~/.roswell/helper.el~. Following is the script I modified. Since I’m not using slime but sly, and I need to load .sbclrc, I commented out like half of the helper script ;)

(defun roswell-configdir ()
  (substring (shell-command-to-string "ros roswell-internal-use version confdir") 0 -1))

(defun roswell-load (system)
  (let ((result (substring (shell-command-to-string
                            (concat "ros -L sbcl-bin -e \"(format t \\\"~A~%\\\" (uiop:native-namestring (ql:where-is-system \\\""
                                    system
                                    "\\\")))\"")) 0 -1)))
    (unless (equal "NIL" result)
      (load (concat result "roswell/elisp/init.el")))))

(defun roswell-opt (var)
  (with-temp-buffer
    (insert-file-contents (concat (roswell-configdir) "config"))
    (goto-char (point-min))
    (re-search-forward (concat "^" var "\t[^\t]+\t\\(.*\\)$"))
    (match-string 1)))

(defun roswell-directory (type)
  (concat
   (roswell-configdir)
   "lisp/"
   type
   "/"
   (roswell-opt (concat type ".version"))
   "/"))

(defvar roswell-slime-contribs '(slime-fancy))


;; Following is simply for adding sly/slime hook. It's not working well. Also since this made sly conflict with slime (which I don't want), so I commented this out.
;; (let ((type (or (ignore-errors (roswell-opt "emacs.type")) "slime")))
;;   (cond ((equal type "slime")
;;          (let ((slime-directory (roswell-directory type)))
;;            (add-to-list 'load-path slime-directory)
;;            (require 'slime-autoloads)
;;            (setq slime-backend (expand-file-name "swank-loader.lisp"
;;                                                  slime-directory))
;;            (setq slime-path slime-directory)
;;            (slime-setup roswell-slime-contribs)))
;;         ((equal type "sly")
;;          (add-to-list 'load-path (roswell-directory type))
;;          (require 'sly-autoloads))))

;; This is also problematic. "ros run" is not loading .sbclrc of course.
;; (setq inferior-lisp-program "ros run")

Emacs Config

Then, place the following line in your emacs initialization file to use the sbcl installed by Roswell, initializing by with our .sbclrc. Needless to say, you need pkgs.roswell in your home manager configuration.nix to use roswell later.

(setq inferior-lisp-program "ros -L sbcl -Q -l ~/.sbclrc run")

;; Also these are well-known
(load (expand-file-name "~/.roswell/helper.el"))
(use-package sly :ensure t)

Nix Expression

Now the most important shell.nix.

let
  sources = import ./nix/sources.nix;
  pkgs = import sources.nixpkgs {};
in pkgs.mkShell {
  nativeBuildInputs = [
    pkgs.niv
    pkgs.zlib
    pkgs.roswell
  ];

  shellHook = '' 
    export LD_LIBRARY_PATH=${pkgs.lib.makeLibraryPath [
                 pkgs.openssl
               ]}
  '';
}

Here, sources is referring to my pinned version of nixpkgs, so that’s trivial. What makes the file important is shellHook and export.LD_LIBRARY_PATH part. This is where nix exposes the shared object files provided openssl, thus making libcrypto.so available.

READYSETGO

Now what do you do? I assume you have direnv and nix ready in your system. Then you simply echo "use nix" > .envrc and direnv allow in your project directory to automatically trigger nix-shell whenever you cd into dir. After they’re complete, you can safely launch Emacs inside the project (or, like me, you can use direnv-mode), and M-x sly to make your REPL work. Like Sly says, “Take this REPL, brother, and may it serve you well.”

Good Luck.