mirror of
https://github.com/wezm/wezm.net.git
synced 2024-11-18 04:42:47 +00:00
Add linked-list post
This commit is contained in:
parent
acea9bd1a3
commit
43667e68b9
1 changed files with 191 additions and 0 deletions
191
v2/content/posts/2024/linked-list.md
Normal file
191
v2/content/posts/2024/linked-list.md
Normal file
|
|
@ -0,0 +1,191 @@
|
||||||
|
+++
|
||||||
|
title = "Building and Launching My New Link Blog, linkedlist.org (Twice)"
|
||||||
|
date = 2024-10-31T13:56:30+10:00
|
||||||
|
|
||||||
|
#[extra]
|
||||||
|
#updated = 2024-06-06T08:24:45+10:00
|
||||||
|
+++
|
||||||
|
|
||||||
|
I've started a new tech focused link blog over at
|
||||||
|
[linkedlist.org](https://linkedlist.org). "Not another tech blog", I hear you
|
||||||
|
groan, and rightly so. However my intention is **not** to cover topics that are
|
||||||
|
already well reported upon like Apple, Google, Microsoft, the latest drama at
|
||||||
|
OpenAI, and other stuff like that. Instead, I plan to focus more open-source,
|
||||||
|
programming, hardware, software, Linux, Rust, retro computing etc. There's some
|
||||||
|
more details in [the welcome post][welcome].
|
||||||
|
|
||||||
|
In this post I'm going to cover the process I took to the build the site
|
||||||
|
(twice) and some of the considerations that went into it—for a site with only a
|
||||||
|
handful of pages there was a surprising amount of them.
|
||||||
|
|
||||||
|
<!-- more -->
|
||||||
|
|
||||||
|
When building Linked List my reference was [Daring Fireball] by John Gruber,
|
||||||
|
which I've enjoyed reading for close to two decades now (aside from some of the
|
||||||
|
recent takes on the EU DMA). The site is simple on the surface but it's clear
|
||||||
|
that John has put a bunch of thought into a number of different aspects of the
|
||||||
|
site.
|
||||||
|
|
||||||
|
The high-level things I wanted were:
|
||||||
|
|
||||||
|
- The distinction of a link post (to an external site) versus a first party post[^1].
|
||||||
|
- An RSS feed that has special accommodations for link posts where the item
|
||||||
|
links to the external site.
|
||||||
|
- Automated publishing of new posts to Mastodon.
|
||||||
|
- A clean design.
|
||||||
|
- Responsive (mobile friendly).
|
||||||
|
|
||||||
|
### The First Build
|
||||||
|
|
||||||
|
I initially toyed with the idea of using [micro.blog] to host the site as it
|
||||||
|
can host blogs, micro or not, and has a lot of built-in support for
|
||||||
|
cross-posting to services like Mastodon. Ultimately I decided it wasn't quite
|
||||||
|
the right fit for what I was aiming for.
|
||||||
|
|
||||||
|
Concluding I'd need to build it myself I reached for my go-to
|
||||||
|
static-site-compiler: [Zola]. As part of the micro.blog experiment I took a
|
||||||
|
liking to their [Alpine theme], which as it was open-source I used as the base
|
||||||
|
style for the Zola site.
|
||||||
|
|
||||||
|
My first hurdle was that I wanted a particular URL hierarchy:
|
||||||
|
|
||||||
|
`linkedlist.org/YYYY/MM/DD/post-slug`
|
||||||
|
|
||||||
|
This proved a challenge to achieve with Zola. There was an [open issue]
|
||||||
|
about this sort of structure, which led me
|
||||||
|
<https://github.com/scouten/ericscouten.travel>. I was able to replicate the
|
||||||
|
approach used there to achieve what I wanted.
|
||||||
|
|
||||||
|
Creating a new post was a bit involved though as it required creating each of
|
||||||
|
the intermediate directories if they didn't exist, as well as an `_index.md`
|
||||||
|
file with particular content in each newly created directory. I automated this
|
||||||
|
with a Ruby script, which was also able to pre-populate the front matter for a
|
||||||
|
new link post.
|
||||||
|
|
||||||
|
### Cross-posting to Mastodon
|
||||||
|
|
||||||
|
For cross-posting I wanted to use the site's feed as the source and have the
|
||||||
|
tool post newly published items. There are dozens of these RSS to Mastodon
|
||||||
|
tools on GitHub. I evaluated 16 of them against a handful of requirements:
|
||||||
|
|
||||||
|
**No runtime[^2]**
|
||||||
|
|
||||||
|
This ruled out the vast majority, as many were written in Python or
|
||||||
|
JavaScript/TypeScript. I don't want to have to deal with operating tools using
|
||||||
|
these languages as I find handling their dependencies and breakage due to
|
||||||
|
upgrades annoying.
|
||||||
|
|
||||||
|
**Conditional Requests**
|
||||||
|
|
||||||
|
It feels like table stakes that a tool that is polling a HTTP resource will
|
||||||
|
make conditional requests using headers like `If-Modified-Since` or
|
||||||
|
`If-None-Match` so that it will only be fetched and processed if modified. The
|
||||||
|
vast majority of the tools I evaluated just fetched the feed every time they
|
||||||
|
polled it though.
|
||||||
|
|
||||||
|
**Robust Against Duplicate Posts**
|
||||||
|
|
||||||
|
I want to reduce the possibility of something causing a flood of posts or
|
||||||
|
duplicates. Some of the tools I evaluated did alright here, but many did not.
|
||||||
|
|
||||||
|
----
|
||||||
|
|
||||||
|
None felt like they covered all these, so I took [the code I had written for
|
||||||
|
the Read Rust tooter][tooter] and reworked it to use a feed as the data source
|
||||||
|
instead of a database. I spent a fair bit of time making it support
|
||||||
|
multiple feeds and feed formats, as well multiple posting targets. I plan to
|
||||||
|
open-source it in the future.
|
||||||
|
|
||||||
|
I protect against duplicate posts and post floods by:
|
||||||
|
|
||||||
|
- Marking all existing items of a feed as seen before on first run.
|
||||||
|
- Tracking the guid of each published item, and only publishing new ones.
|
||||||
|
- Tracking the content of each Mastodon post published so that if an item slips
|
||||||
|
through the other guards it will hopefully be stopped at this point.
|
||||||
|
- Use idempotency keys when publishing to Mastodon so that if a post fails
|
||||||
|
on the client but is actually successfully processed by the server,
|
||||||
|
it will be rejected on a subsequent attempt to post it.
|
||||||
|
|
||||||
|
Before the scope creep in the cross-posting tool occurred I added a [JSON Feed]
|
||||||
|
to Linked List as these was a bit easier to consume than the Atom feed. Only
|
||||||
|
problem was [Zola didn't support JSON Feed][zola-json-feed]. I solved this with
|
||||||
|
a `jaq` script described in [my previous post](@/posts/2024/json-feed-zola.md).
|
||||||
|
|
||||||
|
With the cross-posting tool mostly feature complete I deployed it to a
|
||||||
|
Raspberry Pi Zero W, running on my desk. Every 15 mins `cron` fires it up to
|
||||||
|
check for, and post new items in the feed.
|
||||||
|
|
||||||
|
### Polish
|
||||||
|
|
||||||
|
There was a long tail of smaller things that I implemented before the initial
|
||||||
|
launch, such as:
|
||||||
|
|
||||||
|
- Dark mode
|
||||||
|
- [OpenGraph] metadata
|
||||||
|
- Mastodon account and verification metadata
|
||||||
|
- Archive pages such as `/2024` and `/2024/10`
|
||||||
|
- The myriad of favicon images
|
||||||
|
([realfavicongenerator.net](https://realfavicongenerator.net/) helped a lot
|
||||||
|
here)
|
||||||
|
|
||||||
|
### Launch & Rewrite
|
||||||
|
|
||||||
|
Finally, on 11 October I soft-launched the site with a post on Mastodon. Over
|
||||||
|
the next couple of weeks I published posts to the site, eventually realising
|
||||||
|
I'd never added pagination to the home page. I went to add it that weekend and
|
||||||
|
discovered that it was going to be very difficult owing to how I'd achieved
|
||||||
|
the URL hierarchy with Zola.
|
||||||
|
|
||||||
|
Zola had got the site up, but I took this (as well as some of the earlier
|
||||||
|
friction) as a sign it was time for a custom approach. Over the next few
|
||||||
|
evenings and some weekend time I rewrote it in Rust using Axum as the HTTP
|
||||||
|
server layer.
|
||||||
|
|
||||||
|
Since things were more under my control now I took the opportunity to:
|
||||||
|
|
||||||
|
- Drop trailing slashes from URLs.
|
||||||
|
- Set long-lasting `Cache-Control` values for static files and include them
|
||||||
|
with a cache busting hash.
|
||||||
|
- Bundle all static files like CSS and fonts into the binary in release builds.
|
||||||
|
- Add pagination to the home page.
|
||||||
|
|
||||||
|
I also had to do a bit of extra work to support things you get for free in a
|
||||||
|
Zola/static site:
|
||||||
|
|
||||||
|
- `Etag` headers
|
||||||
|
- Conditional request support
|
||||||
|
- `sitemap.xml`
|
||||||
|
|
||||||
|
On 30 October I deployed the new version of the site. It renders the same
|
||||||
|
Markdown files as the Zola site, so publishing new posts is still just rsyncing
|
||||||
|
the files.
|
||||||
|
|
||||||
|
The Markdown content is loaded from disk at start up and rendered out of RAM.
|
||||||
|
It doesn't currently cache the rendered result, but most responses are generated
|
||||||
|
in 1ms or less anyway. A filesystem watcher is used to notice when the files
|
||||||
|
are changed and automatically reload them.
|
||||||
|
|
||||||
|
With the rewrite out of the way I now have more time for more regular posting
|
||||||
|
to the site, I hope you'll [follow along].
|
||||||
|
|
||||||
|
[^1]: Gruber actually calls the collection of link posts on Daring Fireball
|
||||||
|
[the Linked List][df-linked], although I was only tangentially aware of
|
||||||
|
this when I embarked on this project. My main motivation was that I liked
|
||||||
|
the cross-over with [the data structure][data-structure] and that I already
|
||||||
|
had the `linkedlist.org` domain, originally registering it in 2011.
|
||||||
|
[^2]: I.e. a native binary that can be run without having to install an interpreter
|
||||||
|
or similar first.
|
||||||
|
|
||||||
|
[Alpine theme]: https://github.com/microdotblog/theme-alpine
|
||||||
|
[Daring Fireball]: https://daringfireball.net/
|
||||||
|
[data-structure]: https://en.wikipedia.org/wiki/Linked_list
|
||||||
|
[df-linked]: https://daringfireball.net/linked/
|
||||||
|
[follow along]: https://linkedlist.org/follow
|
||||||
|
[JSON Feed]: https://www.jsonfeed.org/
|
||||||
|
[micro.blog]: https://micro.blog/
|
||||||
|
[open issue]: https://github.com/getzola/zola/issues/2275
|
||||||
|
[OpenGraph]: https://ogp.me/
|
||||||
|
[tooter]: https://github.com/wezm/read-rust/tree/master/rust/src
|
||||||
|
[welcome]: https://linkedlist.org/2024/09/14/welcome-to-linkedlist
|
||||||
|
[zola-json-feed]: https://github.com/getzola/zola/issues/311
|
||||||
|
[Zola]: https://www.getzola.org/
|
||||||
Loading…
Reference in a new issue