org-publish

publish.el
- Dependencies
- Configuration
Styling
- pre/postambles
- CSS
  - theme
  - main.css
Building

Last year I decided i wanted to rewrite my Emacs config, and wanted to make it public through some kind of website/blog. Being an Emacs and org-mode enthousiast I naturally wanted to work on it in a workflow that feels close to my daily habits. So i went with Hugo, a static site generiator that appearantly supports org-mode out of the box. Guess what,.. I spent so much time trying to get things the way I wanted them to be that I kept postponing creating actual content up until the point that i forgot about that site all together, So now there is a single post on some obscure Jira package that i wrote, and ahalf baked Emacs config that i couldn't get to both export nicely to the site and be useful as a config at the same time. I guess the only piece of software that supports org-mode is org-mode itself.

So with the new year (yay, new years resolutions) coming up, and a bunch of actual potential content in december (advent of code) I thought it might be a good idea to give this whole website/blog a new try. This time going org-mode only. And to keep it interesting, I'll use this very post as the literate configuration file for publishing the actual site itself.

So wit no further ado, here we go.

Over the weekend I managed to get a few moments to sqeeze in some time to work on an initial config for this site. Having only touched this part of Emacs a couple of years back I naturally spent quite some time in the manuals and online spitting through other people's configrations. Not all examples and documentation is as complete as we can hope for, Which emphasizes the need to publish my own configuration along with some context as for why I did it this way.

publish.el

Dependencies

Publishing with a pure vanilla Emacs sounds pretty awesome, but that appears to be an impossible route. I think i was about half an hour in when I decided I wanted syntax highlighting on the sourceblocks, which requires an addidiotnal package called htmlize. This means pulling in a package, which in itself is not that hard, except when your publishing from a clean environment, i.e. no packages from my daily development setup will be available.

So to be able to use external packages in our publishing environment, we'll first need to setup our dependencies. This code is pulled from one of David Wilson's excellent System Crafters articles.

;; Set the package installation directory so that packages aren't stored in the
;; ~/.emacs.d/elpa path.
(require 'package)
(setq package-user-dir (expand-file-name "./.packages"))
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))

;; Initialize the package system
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))

;; Install dependencies
(package-install 'htmlize)

;; require some additional packages
(require 'ox-publish)

We'll also need some extra supporting functions as the default is rarely good enough for me. The next method is used to generate the titles for the sitemap. Borrowed from here.

(add-to-list 'org-export-options-alist '(:excerpt "EXCERPT" nil nil parse))

;; https://www.danliden.com/posts/20211203-this-site.html
(defun my/org-publish-find-excerpt (entry project)
  "try to exctract excerpt from document"
  (with-temp-buffer
    (insert-file-contents (concat (org-publish-property :base-directory project) entry))
    (goto-char (point-min))
    (cond
     ((re-search-forward "^\s*#\\+BEGIN_PREVIEW$" nil 1)
      (progn
        (goto-char (point-min))
        (let ((beg (+ 1 (re-search-forward "^\s*#\\+BEGIN_PREVIEW$" nil 1)))
              (end (progn (re-search-forward "^\s*#\\+END_PREVIEW$" nil 1)
                          (match-beginning 0))))
          (buffer-substring beg end))))
     (nil "-"))))



(defun my/org-sitemap-date-entry-format (entry style project)
  "Format ENTRY in org-publish PROJECT Sitemap format ENTRY ENTRY STYLE format that includes date."
  (let ((filename (org-publish-find-title entry project)))
    (if (= (length filename) 0)
        (format "*%s*" entry)

      (format "[[file:%s][%s]] {{{timestamp(%s)}}}\n%s"
              entry
              filename
              (format-time-string "%Y-%m-%d"
                                  (org-publish-find-date entry project))
              (my/org-publish-find-excerpt entry project)
))))

(setq org-export-global-macros
      '(("timestamp" . "@@html:<span class=\"timestamp\">$1</span>@@")
        ("excerpt" . "@@html:<div class=\"excerpt\">$1</span>@@")))

And finally, I'll need to be able to read in some files to inject them as header and footer for every exported page. As this is not a simple built-in function, here is a simple function that does just that.

(defun pub/read-file (filename)
  "Return the contents of FILENAME."
  (with-temp-buffer
    (insert-file-contents filename)
    (buffer-string)))

Configuration

(setq org-html-htmlize-output-type 'css)

;; pretty source code
(setq org-src-fontify-natively t)

;; we'll use our own styling
(setq org-html-head-include-default-style nil)

;; include pre and postambles on every page
(setq pub/preamble (pub/read-file "build/includes/preamble.html"))
(setq pub/postamble (pub/read-file "build/includes/postamble.html"))

;; this should be extracted to literate block with read-file to load
(setq org-html-head-extra "
<link rel=\"stylesheet\" type=\"text/css\" href=\"/assets/css/main.css\" />
<link rel=\"stylesheet\" type=\"text/css\" href=\"/assets/css/htmlize.css\" />
<meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">
")

;; define all various parts of the export
(setq org-publish-project-alist
      `(
        ("posts"
         :base-directory "./src/posts/"
         :base-extension "org"
         :publishing-directory "./build/public_html/"
         :recursive t
         :publishing-function org-html-publish-to-html
         :headline-levels 4
         :auto-sitemap t
         :html-preamble ,pub/preamble
         :html-postamble ,pub/postamble
         :sitemap-title "faijdherbe.net - posts"
         :html-html5-fancy t
         :sitemap-filename "index.html"
         :with-tags nil
         :html-link-org-files-as-html t
         :section-numbers nil
         :sitemap-sort-files anti-chronologically
         :sitemap-format-entry my/org-sitemap-date-entry-format)

       ("advent-of-code"
         :base-directory "./src/advent-of-code/"
         :base-extension "org"
         :publishing-directory "./build/public_html/advent-of-code/"
         :recursive t
         :publishing-function org-html-publish-to-html
         :headline-levels 4
         :auto-sitemap t
         :html-preamble ,pub/preamble
         :html-postamble ,pub/postamble
         :sitemap-title "Advent of Code"
         :html-html5-fancy t
         :sitemap-filename "index.html"
         :with-tags nil
         :with-toc t
         :html-link-org-files-as-html t
         :section-numbers nil
         :sitemap-sort-files anti-chronologically
         :sitemap-format-entry my/org-sitemap-date-entry-format)

        ("emacs"
         :base-directory "./src/emacs/"
         :base-extension "org"
         :publishing-directory "./build/public_html/emacs/"
         :recursive t
         :publishing-function org-html-publish-to-html
         :headline-levels 4
         :html-preamble ,pub/preamble
         :html-postamble ,pub/postamble
         :html-html5-fancy t
         :with-tags nil
         :html-link-org-files-as-html t
        )


        ("statics"
         :base-directory "./assets/"
         :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg"
         :publishing-directory "./build/public_html/assets/"
         :recursive t
         :publishing-function org-publish-attachment
         )

        ("tangled-statics"
         :base-directory "./build/assets/"
         :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg"
         :publishing-directory "./build/public_html/assets/"
         :recursive t
         :publishing-function org-publish-attachment
         )

        ("org" :components ("posts"
                            "emacs"
                            "advent-of-code"
                            "statics"
                            "tangled-statics"))
        ;; ... add all the components here (see below)...

        )
)

And last but not least, the command to start the actual publication. It's currently preceded by two commands that reset caches and timestamps as I'm still in the process of fixing CSS and stuff. Without those commands Org would skip publication of files that "have not changed", which might be invalid due to undetected pre-/postamble fixes.

(org-publish-reset-cache)
(org-publish-remove-all-timestamps)

;; disable confirmation on export
(setq org-confirm-babel-evaluate nil)

(org-publish-project "org")
(message "copy favicon")
(copy-file "./src/favicon.ico" "./build/public_html/favicon.ico" t)

Styling

As the default published site looks and feels a bit too basic, we'll introduce a little styling ourselves. Starting off with a custom header (preamble) and footer (postamble)

pre/postambles

These pre- and postambles are both written in HTML, as I did not succeed in exporting these simple datastructures from org-mode. Maybe later I'll be able to change is part. The content's of this src_block wont be tangled, but injected into other blocks instead.

<nav>
    <span><a href="/">faijdherbe.net</span></a>
    <ul>
        <li><a href="/">posts</a></li>
        <li><a href="/advent-of-code/">advent of code</a></li>
        <li><a href="/emacs/">emacs</a></li>
    </ul>
</nav>

<footer>
  <span>Generated at %T</span>
  <ul>
    <li><a target="blank" href="https://github.com/faijdherbe">github</a></li>
    <li><a target="blank" href="https://phpc.social/@jlfaijdherbe">mastodon</a></li>
    <li><a target="blank" href="https://boardgamegeek.com/user/faijdherbe">bgg</a></li>
  </ul>
</footer>

CSS

To make this thing look a little bit better we'll need some CSS as well. The css will consist of two files. One will contain the syntax highlighting, and the other the styling of the pages. We'll start of with the main styling.

theme

key	name	light	dark
bg	background	#EEE	#333
bgt	background tinted	#DDD	#444
fg	foreground	#333	#EEE
fgt	foreground tinted	#444	#DDD
p	primary	#70F	#0FF
s	secondary	#F0F	#F0F

This code block is exuted while tangling the css file. The css blocks in the rendered version of this document show the resolved colors based on the table above, but when looking at the source directly it looks something like a { color: <<color(key="p", mode="dark")>> }.

(let ((result (assoc key theme))
      (err "#F00"))
  (format "%s" (if (string= mode "dark") (cadddr result)
   (caddr result))))

main.css

The overall settings of the styling. Set default margins, colors etc.

html, body {
    background-color: #EEE;
    color: #333;
    margin: 0px;
    height: 100%;
    line-height: 1.5em;
}

.src {
    padding: 1em;
    background-color: #DDD;
    border-style: solid;
    border-color: #444;
    border-width: 0px 0px 1px 1px;
    overflow: auto;           
}

.outline-2 {
    padding-bottom: 3em;
}

.content {
    max-width: 20cm;
    text-align: justify;
    margin: auto;
}

a {
    color: #70F;
    text-decoration: underline;
    font-weight: bold;
}

a:hover {
    color: #F0F;
}


blockquote{
    font-style: italic;
    margin: 1em 0em;
}

sitemap

The sitemap is where the listing of the blogposts is presented. It is (currently) a simple list of document titles and their publish date.

.org-ul {
  list-style-type: none;
  margin: 2em 0em 2em 0em;
  padding: 0;
}

.org-ul li {
    padding: 1em;
    margin: 1em 0em 1em 0em;              
}

.org-ul li .timestamp {
    display: block;
    font-size: .75em;
    font-style: italic;
    margin-bottom: 1em;
}

table of ontents

#table-of-contents {
    padding: .5em 0em;
    background-color: #DDD;
    color: #444;
    border-style: solid;
    border-width: 0px 0px 0px 2px;
    border-color: #444;

}
#table-of-contents div {
    padding: 0;
}
#table-of-contents h2 {
    display: none;
}

#table-of-contents ul {
    list-style-type: none;
}

Preamble

#preamble {
    border-style: dotted;
    border-color: #444;
    border-width: 0px 0px 1px 0px;
}

#preamble span {
    float: right;
    display: block;
    margin: 0px 2px;
    text-align: center;
}

#preamble ul {
  list-style-type: none;
  margin: 0px;
  padding: 0;
  overflow: hidden;
}
#preamble li {
    display: inline;
    float: left;
    margin: 0px 2px;
}
#preamble a {
  display: block;
  font-weight: bolder;
  text-align: center;
  padding: 14px 16px;
}

postamble

#postamble {
    font-style: italic;
    font-size: .75em;    
    padding: 1em;
    border-style: dotted;
    border-width: 1px 0px 0px 0px;
    border-color: #444
}

#postamble span {
    float: left;
}

#postamble ul {
    list-style-type: none;
    margin: 0px;
    padding: 0;
    overflow: hidden;
}
#postamble li {
    display: inline;
    float: right;
    margin: 0px 0px;
}
#postamble a {
    display: block;
    text-align: center;
    padding: 0px 1em;
}

media queries

We've got a few media queries to support. The first one will be for smaller screens (phones) as we want these to be able to read the site as well, and the second is about respecting the browser's / system's theme settings (dark vs. light).

@media only screen and (max-width: 20cm) {
    body {
        margin: 1em;
    }
    #postamble ul {
        text-align: center;
    }
    #postamble li {
        float: none;
        display: inline-block;
    }
    #postamble span {
        float: none;
        width: 100%;
        text-align: center;
        display: block;
    }
}

For dark themes, well switch the foreground and background colors, as well as the dark / light versions. unfortunately i know no better way that to manually override every element that has some kind of color.

@media (prefers-color-scheme: dark) {
    html, body {
        background-color: #333;
        color: #EEE;
    }
    .src {
        background-color: #444;
        border-color: #EEE;
    }
    a {
        color: #0FF;
    }
    a:hover {
        color: #F0F;
    }
    #table-of-contents {
        background-color: #444;
        color: #EEE;
        border-color: #DDD;
    }
    #preamble {
        border-color: #DDD;
    }
    #postamble {
        border-color: #DDD;
    }
}

Building

Makefile

Whoah, recursion… You'll need this file to be tangled in order to tangle this file :head-exploding:. So yes, you might want to do a manual tangle on this file in order to get the latest makefile published. I could have also skipped this chapter and just link to the Makefile instead, but I think that this document would be incomplete if I did and… i like recursions. If your viewing this documentation from the Github repository, you'll notice I've got this file checked in anyway, which is solely because I want to be able to publish directly from emacs using projectile, instead of having to manually tangle first. (yeah I know, C-c C-v t is not that hard to press, it's just not in my main flow and hopefully I there will be a point in time that I did'nt open this file for ages. Having to look it up when migrating to a new laptop will be a real pain, probably comparable to the man-flu.

clean:
        echo "cleaning"
        rm -rf build/public_html/*

tangle:
        echo "tangling"
        emacs --batch -l org --eval "(setq org-confirm-babel-evaluate nil)" --eval "(org-babel-tangle-file \"src/posts/org-publish.org\")"

build:
        echo "building"
        emacs --batch -l org --script "scripts/build.el"

publish: 
        echo "publishing"
        bash ./private-sync-script.sh

all: tangle build publish       
        echo "all done"

sync to server

To sync this to the server i've included a small bash script that I didn't publish in this document for security reasons. But the contents look roughly like this:

#!/bin/bash
scp -i <identityfile> \ 
    -r \
    build/public_html/* \
    <username>@<hostname>:<remote-path>

When running this script above on Termux (yes, i want to be able to write and publish from my phone, or my uConsole if it ever arrives) the file permissions on the server got messed up. This rsync script seems to do a bit better, but is probably still not the best way to go.

#!/bin/bash
rsync -r \
      --perms \
      --chmod=u+rwx,g+rx,o+rx \
      ./build/public_html/* \
      <username>@<hostname>:<remote-path>

Test server

I saw something about an Elisp webserver while writing this documentation, but its getting late so I'm going to go the easy way and just publish the Docker Compose file instead. It uses Apache2 and does not like deleting the build/ directory while it's running. Hence the removal of the contents of the public folder in the makefile above.

web:
  hostname: faijdherbe.docker
  image: ubuntu/apache2
  volumes:
    - ./build/public_html:/var/www/html
  ports:
    - "8000:80"