I've been trying to make a cool static site generator numerous times. Well, yes, it actually is a quite fun thing to tinker with, but I've found myself struggling to decide on what do I want to see in the final app and what I do not; what features might be needed by other users and what features might be an overkill. So I ultimately went with Zola[1] and encourage you to try it too! Aaaand it is written in Rust, too. It's available in Alpine Linux's repos so it's trivial to set up a CI/CD to compile your blog.
I plan on writing a static site generator soon myself. And I briefly wondered the same thing and realized that I could very easily answer that question: ruthlessly build whatever I personally need and not what others need. I don't usually follow that line of thinking, but the static site generator space is so full of so many great tools that I don't think there's really anything for me to contribute to it.
Instead, I think the killer feature of my static site generator is going to be exactly what makes it difficult for others to use: its tight coupling with my specific use case. I'll be able to hard-code configuration knobs. Avoid dealing with edge cases. Bake strange non-generic and single-blog-post-specific logic right into the tool.
(The reason why I want to build my own is because I've tried using someone else's. And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it, which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.)
I went this route. At first I tried writing a bunch of Python scripts reading a YAML manifest. However, after ~2 weeks I already got warnings and errors about things which got deprecated/removed/changed by a new Python version and the YAML library I was using. So I dumped it and rewrote the thing in XML and XSLT. It's been working for a couple years with no breakages. Essentially 0 maintenance after the initial work. This design was about generating a tree of directories with Markdown files turned into HTML, PDF files turned into pages linking to them, and some directory renames, GUIDs and dates (for Atom feeds, which I generate for each directory) specified in a manifest file.
I've since taken up photography, so now I need to automatically read metadata from jpegs, add some tags and organise it on the site in albums. It's a bit outside the reach of XSLT, so first I've tried using Python with Jinja2 and PIL. Same problem as previously. So I've rewritten it in Go with the help of encoding/json (for the manifest), html/template (for the template), image/jpeg, golang.org/x/image/draw (for generating thumbnails of different sizes for different screen sizes) packages and exiftool (reading and removing EXIF tags). Pandoc has been present throughout all the rewrites, because I like its flavour of Markdown and options to shift heading levels by a specified value, so that they fit in with the surrounding template. I'm putting some finishing touches and am going to upload it in the coming days.
I've done exactly what you describe and I highly recommend it.
My personal website is served by a couple hundred lines of Rust. There is no database or ORM -- I just defined structs like `BlogPost` or whatever and stuck a `#[derive(Serialize, Deserialize)]` on it and use the `serde` library to JSONify it, store it flat in a hard-coded folder.
I even wrote my own dead-simple Markdown-ish flavor that is tailored exactly to my writing style. I use a lot of em-dashes, for example, so it translates `--` into `—` (rather, &mdash). Or, since a lot of my posts link to other posts, I can do `[post title_of_other_post]` and it'll generate the appropriate link.
Whenever I want a new feature while writing a particular blog post, e.g. pretty inline SVG, I just go add it and then keep writing.
We think alike. I started out with Hugo because it's very popular and relatively easy to use. It has a bunch of features that I had to read about but didn't have a use for -- overhead.
One of the most annoying aspects of working with hugo was the friction from idea to publishing it.
I decided to go a completely different route and built https://prose.sh
With prose all I have to do is to create a markdown file and then `scp` it to prose.sh. The platform takes care of the rest and I can still enjoy my own editor and terminal tooling. Even better, authentication happens via SSH so the friction is extremely low.
It's a nice balance between writing a blog on a platform that does everything for you and being able to use my own tooling to write content.
Another big issue with Hugo is when I want to make edits to the content. My brain is weird and I only scrutinize my content after it has been published. So I end up making a lot of edits after an article has been live for a few minutes. It's a terrible habit and Hugo required deployment for every little change.
I've thought about converting prose to be SSG -- we could generate the html when a user uploads the file -- but honestly the site is fast as it is so I don't see the value in that added complexity.
Nice! I have a Hugo site I’ve tweaked over the years, and my workflow is similar to yours. I create/edit an MD file, then run a shell command which pushes changes to Github, which triggers a redeploy.
How do you handle images? That’s my main annoyance / biggest hurdle when publishing. I have to take each image, resize it for web, put it in static folder, and reference that pathname in the MD file. Not a big deal, but would love an easier way. I suppose I could write a simple shell script that does all of the resizing and prints out a path back for me to use.
Can you use `hugo server` (much like `jekyll/zola serve`) and a local Git repo to polish up the content locally, explore various permutations, before pushing to a cloud-based host?
> And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, []something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it[], which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.
This is true, as a general principle, not just of static site generators, but of all software, and is one of the major Reasons Why We Can't Have Nice Things, and why automatic updates are evil and poisonous, and manual updates dangerous and often-avoided. See eg https://jacquesmattheij.com/why-johnny-wont-upgrade/.
I don't think so. There's lots of software that I've used and upgraded for decades and rarely if never run into any issues.
I don't think it's a general principle. I don't like generalizing to "all" unless I have a really good reason to do so. Instead, I stuck to my specific experience with static site generators.
You should check out what fasterthanlime does for their blog. Theirs uses sqlite for storing all state and the write plugins into the Rust Liquid template implementation to directly query sqlite so they could do whatever they wanted without recompiling. It was pretty neat.
I have been looking to break out parts of static-site-generators to make it easier to bring one up. So far I only have two primitive building blocks
My needs are far simpler than that. I don't need state. I don't need dynamic content. I don't need ads. I don't need tracking. I don't need search. I just need a way to publish content and maybe an RSS feed. It doesn't even need to serve files. Everything is static. (I'm not saying fasterthanlime has all those things. Just saying what I don't need.)
I can't even imagine I need a template system beyond something I can write in a few dozen lines with some regexes.
I did the SQLite thing more than a decade ago. I ain't going back to it unless I have a really really good reason for it.
I really just need a purely static web site where I can write Markdown posts. There's some other stuff I want in terms of producing the static content, but in terms of web site functionality itself, it should just be HTML, CSS and some images.
I also don't mind re-compiling. For my needs, "compiling" my web site is likely to take less than 1 second.
I plan on trying https://luapress.org/ one of the next weeks. Looks good at a first glance -- give it a quick check if you get a free hour, would be interested what you think of it.
I don't have a free hour unfortunately. And I know I'm not going to use that because I don't want to setup a Lua environment. Indeed, I have enough Lua experience to know that they break the language in big ways for what seems like every release. That's a liability. What happens when luapress stops getting regular maintenance and the version of Lua it uses is no longer packaged in my Linux distro? Oops. Ticking time bomb.
I meant it. I'm done with static site generators built by others. They have caused me too much pain.
Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated. That's my experience. The only remedy I see is to build my own and simplify ruthlessly through tight coupling.
"You either live long enough to see yourself become the villain, or you die a hero."
> Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated.
Sadly you're right. I've been monitoring in the trenches for a while and what you said is covering it perfectly.
RE: maintenance, I was aware but was wondering whether Luapress is mature and complete enough to not need maintenance. I'll give it a go.
And I truly get your point about point releases of languages breaking stuff. I am severely burned out on that front myself that's why my next game plan is to buy and setup a Linux workstation and just isolate various software pieces that I [might] need in Docker containers and periodically backup those for extra good measure.
I am done with all that constantly moving and breaking crap as much as you are. Just looking to find software for all my needs and nail its versions and installs and just allow myself to not constantly keep up with the youngsters' play toys.
Appreciate all your software contributions and a good part of them are my favorite in their area.
I also am quite neurotic about version control, updates etc and if you're prepared to climb up a fairly steep learning curve, I'd recommend Nix/NixOS. It's a much more robust and comprehensive solution than Docker. Docker is not as good a technology as it should be in this use case, and it does stuff like cacheing CMD instructions in the Dockerfile based on their value as a string (rather than the result of the command). This can and does lead to Docker images changing between builds even when the Dockerfile itself has not changed.
Yep, thanks for reminding me about Docker -- I just haven't figured out something much better yet. Maybe mini-VMs like Firecracker or just straight up qemu will be my final stops, who knows (as I plan to buy a workstation with hefty RAM amounts; will likely start at 256GB). I still have a lot of time to think about how to approach the whole thing.
--
For Nix, I think you know what I'll tell you: you don't recommend a war veteran with PTSD "just one last war". I get what they're trying to do but their discourse on various platforms has left much to be desired and they seem opposed to offer more ergonomic CLIs and/or UIs. I hear that's changing as well so I'll be checking them out a few times a year. As it is right now, I wouldn't even mind the steep learning curve -- I am not completely burned out and I still punch quite hard in my work -- but mysterious error messages and maintainers ending discussions with "well then it's not for you" et. al. are just not appealing and feel like I'm taking a risk that will not pay off. Nix still feels like somebody's experimentation project.
I truly hope they take off as their idea deserves but that must come with simplicity on the level of your average Joe and Jane pasting 2-3 commands in a terminal (sort of how you install Rust; it's literally two pasted commands). Before something similar happens, Nix is doomed to remain a niche curiosity.
I just wrote like a Perl script that generates my site. It's just printing stuff and converting markdown to HTML, right? Never breaks, never changes, does exactly what I need it to.
Presumably there are reasons people want to use more specialised tools, but I don't have any of those.
I've just started to take that approach (exactly what I need, no more, no less) with personal process automation, and it's really working for me.
I made a few attempts in the past at building more general-purpose tools that would cover my use cases, but I've never followed through on them. Too much of the work was in the bits that don't actually benifit me right now.
My new tool is simply called "Other Jeff". I write onto it what I need right now. Do I have a series of pull requests that need babysitting to get them over the line? Other Jeff learns to poll their status and notify me of important events along the way. If I want to store some state about what I'm working on right now, I need a database of some kind, right? Nope. Not any more. I hard-code it into `db.rs`, because it turns out there's almost nothing I need to persist that's too burdensome to update by hand.
So what I have in the end is a single Rust program that grows and shrinks depending on my needs at the time, accumulates reusable helpers if and when they actually make sense for _me_, and slowly learns to automate more of my daily distractions.
This. All the big static generators in the market are kinda of generic enough that they demand upfront investment from you. Whether its Jekyll, Hugo or any big shot you have to fit a lot of things according to them. My personal website is built by Jekyll, but I finding a lot harder to make changes because I am not comfortable with a Ruby tooling( Once, I had to debug an obscure Ruby related bug in M1 Mac). I am comfortable with Python though( I have considerable proficiency and good documentation) which then I will use it for my own static generator.
I guess writing your own non-generic own-static generator should be relatively easy and worth the effort as you wouldn't have to read some obscure documentation (like Jekyll).
Sounds cool, I hope one day I stumble upon your article that utilizes those very custom hard-coded knobs. Yet I wouldn't make a tool like that since tools should be KISS.
If I were to make my own generator from scratch, I'd go with python. It's the language I work with the most. A lot of my blog content will be generated from python (figures etc) or will show python code. So having a site generator in the same language can help me add features by hacking on the generator code in a way that a single binary cannot.
I migrated from Pelican (Python) to Zola (Rust) because of the ease of replication of the environment. After a few years it was a pain to recreate my pelican environment and even the tools I'd used to try make that situation easier (pipenv) were undergoing their own bitrot.
I went with Hugo too. I used Gatsby in the past, but it was slower and it felt a bit odd using GraphQL in the way it does. Maybe that'd make more sense if I was using an API to deliver content, but for simple markdown pages, it seemed unnecessary.
On the other hand, having single binary is good because you don't have to install the runtime to tinker with the tool. Yet I do agree that tinkering options are very limited in comparison.
> I've found myself struggling to decide on what do I want to see in the final app and what I do not; what features might be needed by other users and what features might be an overkill.
It's tricky. You can't accept all the features otherwise you just have a mess. At the same time you need to also add/supports features that a lot of people want but _you_ personally don't need if you want the tool to be used by more than 1 person. For example with SSG I don't use themes, I just write my own templates for every site. Despite that I still added it to Zola because most people want themes. Some people will not be happy when you say no to some features they want but hey, it's open-source they can always fork it.
I'm considering using Pandoc with Soupault to my website markup agnostic by being dependent on Pandoc. Soupault can act as a HTML processor although I'm not sure if that's enough to not need a template langauge. Or maybe I'm mistaken about Soupault.
I moved from Jekyll to Zola and it's been great until I started tinkering with templates and stuff recently. Single binary and simple themes were main reasons to choose this. Wanted a feature that wasn't present in the version I started with (about 2 years ago). So, I got the newest version and a few things broke subtly (ex: syntax highlighting). Which led me to tinkering (I don't know much about HTML/CSS/etc).
It was actually good to try a few things almost blindly and then slowly narrowing with a bit more thought process. Been a long time since I had such fun investigating unknown codebase. This even helped me move my post processing scripts to templates. And then I went a step further by reading about meta tags (things like image, description, etc) and adding them via an extra variable in the post frontmatter.
That's exactly why I never used Jekyll and don't really understand people that use it nowadays. It is AWFUL. Not only it uses greatly outdated Ruby, you also HAVE to install outdated Ruby on your machine to play with themes, serve, compile and are forced to use RVM. I know GitHub Pages can compile the page for you, but when you want to customize a theme you have to go through all that hell. It's just a terrible experience. And installing plugins is another hell.
I use Jekyll for a few sites and it works perfectly. Design is bootstrap and so nothing to do with Jekyll and it always compiles the site, takes a few seconds. Works fine on my mac and linux.
Would I have picked Jekyll today? no. Is it worth the cost of switching? no.
I'm literally in the process of porting my blog to Zola right now (from Octopress).
One issue I hit is that taxonomies aren't available at the section-level (nor are they section-specific, in case you want to treat sections as sub-sites), this makes it seemingly impossible to have a section taxonomy view.
I might try to patch it, but it's tricky as it's like merging the taxonomy code with the page and section rendering.
[1] https://www.getzola.org/