Stuff

A few people have asked me about how I manage this site. It changes frequently, as I will often throw stuff up very quickly, move it around, and take it down again.

The whole system is managed by editing local files in Emacs and then a couple of quick keystrokes push out any changes to free web-hosting on GitHub. I'll explain each of the pieces in turn.

Org-mode is a part of Emacs, and is a whole ecosystem for, well, organizing information. Org files are just plain text with some formatting conventions that superficially are not unlike Markdown. Here's an example:

* How does this site work?

A few people have asked me about how I manage this site. It changes
frequently, as I will often throw stuff up very quickly, move it
around, and take it down again.

The whole system is managed by editing local files in Emacs and then a
couple of quick keystrokes push out any changes to free web-hosting on
GitHub. I'll explain each of the pieces in turn.


* It Starts with Org Mode

Org-mode is a part of Emacs, and is a whole ecosystem for, well,
organizing information. [[https://orgmode.org/][Org]] files are
just plain text with some formatting conventions that superficially
are not unlike [[https://en.wikipedia.org/wiki/Markdown][Markdown]].
Here's an example:

That is the exact content that this page starts with. Because I spend most of my day in Emacs already (see email), it is super easy to just pop into an org file, make some edits (or create a new file) and automatically push the changes out to "production".

You can specify a lot of different formats in Org-mode very easily. For example:

*bold*, _underline_, +strikethrough+, /italics/

| n | n^2 | n^3 |
|---+-----+-----|
| 1 |   1 |   1 |
| 2 |   4 |   8 |
| 3 |   9 |  27 |

$y=x^2$

produces:

bold, underline, strikethrough, italics

\(y=x^2\)

This is just scratching the surface or Org-mode. To quote from the Org FAQ:

In its default setup, Org-mode offers a simple outlining and task management environment. But it also has a huge number of features and settings "under the hood." Advanced users have configured Org-mode for almost every conceivable use scenario.

Emacs has a notion of packages, and there a few available for automating the translation of org files into html. In fact, Org-mode has built-in exporting to a number of different formats, including \(\LaTeX\), PDF, ODT/Word, and html. While the default html output from Org is serviceable, there is a more powerful package available that is more flexible, produces nicer html output, and helps keep various ancillary files organized.

The main package for this is ox-publish, which is really part of Org-mode itself. You can create an elisp file (a bit of interpretable code written in Emacs's own variant of the Lisp programming language) that:

1. Makes sure ox-publish is in place
2. Describes where the source (.org) files for the site are located (relative to the directory where the elisp script is).
3. Describes the "publishing" directory where html output files should end up.
4. Sets various other optional parameters (e.g. css style sheets to use, etc.)
5. Runs the actual page creation process.

Then, any time you make any edits or structural changes to the underlying org document, you just invoke this script to automagically process all the changes.

There are many other things you can do within ox-publish. The exact script I use for this site is in the code block below, and I include resources at the end of this document where you can find more details on all the configuration options available.

;;
;; Bring in the package system (elpa/melpa), make sure that is not
;; messing with our normal (user) package dir but is doing so in the
;; local .packages directory, and then bring in htmlize (mostly for
;; syntax highlighting in code blocks).
;;

(require 'package)
(setq package-user-dir (expand-file-name "./.packages"))
(setq package-archives '(("melpa" . "https://melpa.org/packages/"   )
                         ("elpa"  . "https://elpa.gnu.org/packages" )))

;; Init the package system
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))

;; Install dependencies
(package-install 'htmlize)

;;
;; Load the publishing system
;;
(require 'ox-publish)

(setq org-html-validation-link             nil ;; don't show a "Validate" link at the bottom of the page
      org-html-head-include-scripts        nil ;; don't put default org-html export scripts in place
      org-html-doctype                     "html5"
      org-html-html5-fancy                 t
      org-html-head-include-default-style  nil ;; skip the default style sheet
      org-html-viewport                    '((width "device-width")
                                             (initial-scale "0.5")
                                             (minimum-scale "")
                                             (maximum-scale "")
                                             (user-scalable ""))

      org-html-head                        "<link rel=\"stylesheet\" href=\"https://cdn.simplecss.org/simple.min.css\"/>
                                            <link rel=\"stylesheet\" type=\"text/css\" href=\"/static/css/customizations.css\" />
                                            <link rel=\"stylesheet\" type=\"text/css\" href=\"/static/css/org-html-styling.css\" />"
                                            ;; ^ This last file is generated by running org-html-htmlize-generate-css
      ;;org-html-head                       "<link rel=\"stylesheet\" href=\"https://latex.vercel.app/style.css\" />"
      ;;org-html-head                       "<link rel=\"stylesheet\" href=\"/css/latex.css\" />"
      ;;org-html-htmlize-output-type        'inline-css
      org-html-htmlize-output-type          'css

  )

(setq org-publish-project-alist
  (list
   (list "stuff"
     :recursive t
     :base-directory "./content"
     :base-extension "org"
     :publishing-directory "./public"
     :publishing-function 'org-html-publish-to-html
     :with-author nil
     :with-title nil
     :with-creator t
     :with-toc nil
     :section-numbers nil
     :time-stamp-file t)
   (list "static"
     :recursive t
     :base-directory "./static"
     :base-extension "css\\|js\\|pdf\\|png"
     :publishing-directory "./public/static"
     :publishing-function 'org-publish-attachment
     :with-author nil
     :with-title nil
     :with-creator t
     :with-toc nil
     :section-numbers nil
     :time-stamp-file t)))

(org-publish-all t)


(message "Build complete!")

The htmlize package is included in there so that if I have a code block in a given file, like this python file:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
async def root():
    return {"message": "Hello, World!"}

I get syntax highlighting in the exported html automatically without having to do anything further.

Everything up to this point has been about creating content in org format and then automatically converting it into nice looking html. To host it at github, we need a couple of more pieces in place.

The first step is to get a github account. Even if you already have an account for other purposes, it might make sense to have a separate one just for hosting a site (it's somewhat more complicated to juggle projects, repositories, and github pages when you only want to host a website for one repository from a given account, but it can be done).

Once you've got the github account setup, you should create a repository in that account with a name that consists of your username plus "github.io", like this:

     username.github.io

Then use git (or the Emacs module magit) to sync all the org files, directory structure, build script, etc. to the newly created repository.

Now you can take advantage of github's free web-hosting service (called Github Pages) to serve the files via a "Continuous Integration" (CI) process. CI is often spoken of in hushed tones that suggest complex processes involving infrastructure and platform systems to automatically deploy very sophisticated systems. All it means in this context is that github will automatically run a script every time you push new changes to the repository. In particular, if you make a directory inside .github called workflows, any yaml file placed in there will automatically be processed by github after a push. So if we have a file like this called ./github/workflows/publish.yml:

name: Publish to GitHub Pages

on: [push]

permissions:
  contents: write

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - name: Check out
        uses: actions/checkout@v1

      - name: Install Emacs
        run: sudo apt install emacs-nox --yes

      - name: Build the site
        run: ./build.sh

      - name: Publish generated content to GitHub Pages
        uses: JamesIves/github-pages-deploy-action@v4
        with:
          folder: public

Processing that yaml will invoke a virtual machine at github running ubuntu ("runs-on: ubuntu-latest"). On that vm, we will actually quickly install emacs ("run: sudo apt install emacs-nox –yes") and then run a script ("run ./build.sh:) which is just a shell wrapper around the ox-publish script from above. It looks like this:

#!/bin/sh
emacs -Q --script build-site.el

This will then regenerate all the site content in the ./public directory of our repository. The yaml file then says to use a very popular github action (defined in JamesIves/github-pages-deploy-action@v4) to push the content in the /public directory to the GitHub Pages site for this repository. The site should then be available at username.github.io.

All of that may sound a little complicated, with CI yaml invocations of virtual machines, build scripts and whatnot. But once you've wrapped your head around it once and got it set up, it's all fully automated from them on.

The last little detail on this is how to have a custom domain name point at the github hosted content. This is actually well documented on github at:

• Managing a custom domain for your GitHub Pages site

You can use those instructions to set up either an APEX name (e.g. example.com) or a subdomain (e.g. foo.example.com). The main caveat to be aware of is that when you first configure the domain name on github's Settings page, you should wait 24 hours or so before turning on https enforcing. Even in this day and age, DNS propagation can take a while, and that all has to be working correctly and distributed out before the https setting will work.

If you need somewhere to register a domain, I've had good luck with namecheap and porkbun.

If you've never used Emacs and/or org-mode, it might seem like a lot of upfront work to get over the mountain of Emacs just so you can easily manage the molehill of pushing out website updates.

I actually came to Emacs after roughly 20 years of using joe (Joe's Own Editor), and some of my muscle memory for text editing was so deeply ingrained that I had to customize emacs to mimic many of joe's keystrokes.

But whatever path you take to get to Emacs, it will almost certainly be worth it. If you are an engineer, developer, or other technical user, Emacs (plus Org-mode, dired, mu4e, git/magit, etc., etc.) is just such a powerful tool that is worth learning to use well. If the path happens to start with wanting to manage a website in Org-mode, there are worse ways to begin the journey.

And once it is all in place, you can push changes or new content to a site so quickly that it becomes almost second nature.

The amazing System Crafters website has a whole series of videos related to Emacs, and a section devoted to website publishing with Emacs:

• Build Your Website with Org Mode
• Automated Org Mode Website Publishing with GitHub or SourceHut

The above link covers using ox-publish quite extensively, and if you examine the setup they walk through, you'll see it was the source for a lot of what I implemented.

There's also another guide to using ox-publish on the Org-mode website here:

• Publishing Org-mode files to HTML