N79FT A Skybolt Story

B-Log, The Nerd's Builder's Log

On: May 20, 2014
In: blog
Tags: tools, software, website

The Problem

... was that I needed something to organize my builder's log. Being a software engineer by trade, I didn't want to go old school with paper logs and pictures, and whatever-random-collection-of-stuff-on-a-computer I could do. I wanted something that's easy to add entries to, calculates everything automatically, and allows me to publish it on the Web. Plus, I find it pleasant to view HTML as opposed to .doc, .pdf, or whatever.

Being a nerd, I just had to make it as nice for myself as I could...

The KitLog Pro

Many-a-builder is using this apparently successful piece of work. Apparently, as of writing this, they have over 1100 logs published on their mykitlog.com.

I have played around with it quite a bit. The program is trivially simple; but I can't stand the way it's architected.

  • Windows only
  • Uses Access database for storage (sic!!)
  • Only three pictures per entry (really???)

Oh, the Access databases. Basically, what this means to me as a user is if it ever .. erm.. decides to stop working, I will loose all my data. "Revert to a backup", you say.. Yes, but if the problem is that I can't add my N+1st record because the DB just corrupted itself, no backup will help -- Im stuck.

On top of that if, god forbid, KitLog ever goes out of business, I am stuck with the data that I can't export w/o some serious reverse engineering.

No thank you. :)

The Early Attempts

So, I was toying around with other stuff.


Really cool (probably, second best to WordPress) blogging platform. Plus, links very nicely with Picasa, making image inserts a breeze.

The problem? How do I track time spent?

Also, the "going-out-of-business" problem was still there...

Moving on...

A CMS (Drupal or Joomla or whatever) + custom plugins

Now, that was better. Those are open source, good content entry and organization tools, and I can export it all out of a MySQL DB should I need to.

But, they require a server (which means either Im running a server, or require hosting). And frankly, I believed that there must be a better way...


Close, but no cigar. Sphinx is the documentation framework used by Python and many others (including the company I work for).

It takes a bunch of ReST files as input (more on that later), and runs them through a converter, producing whatever you want (there are writers for html, txt, doc, pdf, and so on). The parser and converter is called docutils.

The problem with Sphinx is that it's geared towards writing documentation, and frankly, felt a bit too heavy to me for writing a builder's log.

While googling about yet-another-thingie-I-wanted-to-figure-out-about-Sphinx, I stumbled upon an article that talked about static site generators. There lied the Holy Grail....

Mutilation of the Pelican

The Pelican. I fell in love with this thing almost immediately. Here's what it does:

  • Takes a directory with a bunch of ReST, Markdown, or ASCIIDOC
  • Parses all those files into internal state, obtaining metadata from both files and paths (ie, if you organize your files in category/<date>/name way, you can tell it to properly parse that path and populate the metadata)
  • Uses Jinja templates for generating a bunch of HTML output
  • Allows plugins and custom templates
  • Is all Python!

Ive spent about a week tearing it to pieces and figuring out how it works, and adding a few plugins to add things that I needed for the B-Log that it didn't have. You be the judge of results :).

Here's what Ive done:

  • Im writing text in ReST, so Ive added the custom directive to it allowing for 'figures'. Problem is, ReST image directive produces HTML that wasn't much useful to present images the way I wanted them. Instead of modifying core Pelican code, I elected to add a custom 'Figure' directive that automatically gets a thumbnail (resized by the plugin processing the directive).
  • Ive written the plugin to allow sane handling of images placed next to the entries (since by default Pelican wants you to separate images and text) (and it's internal mechanism of not separating them is broken as of 3.3.0 that Im using (and Im too lazy to fix and commit it back to Pelican ;) ).
  • Ive added the custom optional metadata tag allowing to specify time logged per entry; and the plugin that calculates totals.
  • Then came theming. Pelican is geared towards bloggers, so I had to tweak a lot of things to be better suited for B-Logging (build logging, remember? :) ). In the process, Ive switched to Bootstrap since it makes it very easy to organize content on the webpage. I added theme templates to produce the build log with totals, etc, etc. Also integrated FancyBox - a really cool JavaScript image overlay.

I like the end-result. I enjoyed hacking at it, and this was the first time I touched Python (so had to learn that on the way). Pretty neat (though I still prefer Perl (now, why do you need lists and tuples and sets again? :) ).

The only draw back is that it's regenerating everything every time. You can easily see how that might become a problem when you get a lot of entries, but Im feeling like Im ways away. 30 pages take about a second. I think I will need a lot of records before this becomes a problem; and at that point I will rewrite the whole damn thing to cache stuff.. probably :)

You can check it out here. The repo includes my theme, plugins, some tools, and sample content. It requires Python (I used 3.4 and did not test anything else), Pelican (pip install pelican) and Pillow (pip install pillow). Oh year, and you will need Visual Studio if you're using pip on Windows..

Read the config file; and read thru the first two pages of Pelican docs to get the idea. sample-content/ contains my sample generated content that I was testing with. content-gen/ contains the simple Perl script that generates as much sample content as you want.

How it Works

To me as an end-user, it looks like this.

My source files are organized in directories by category and date:


and so on. There are a few special directories I use, __static for static pages (ie, About section of this site is a static page), and blog, for blog entries (they don't show up under a Category in the Project Log). This page for example is a Blog entry.

If I want a new entry, I just add a directory with the date, and start writing it in my favorite text editor (VIM!!). I drop images. After Im done, I run Pelican, which generates the full website (html, images, everything!) into an output directory. Voila! I can stop right there, and I will have a fully working local version of my log, since it's all static HTML, but we want this online, don't we? So, the only thing left is...


This was another revelation. Apparently, GitHub hosts static pages! 'Uploading a site' is a git push, and it transfers only deltas, so no fiddling with FTP, rsync, or whatever! They allow custom domain names instead of username.github.io, all you need is to set your A records in your DNS to point to their servers. It's awesome!

And So,

I am very happy with this so far. I have all my logs in practically plain text format, parser for which is open source and available. I have it nicely organized, and I can back it up as I please (to GitHub, anyone? :) ). I can re-write the programs that parse and produce final .html. I can extend the generator and change the theme to produce a printable version of the log, styled for paper rather than web. The list goes on!


Up ↑

Powered by B-Log, which is based on Pelican, heavily plugged and themed.

© Copyright 2014-2015 "N79FT". All rights reserved.

This construction log only shows how I did things during the construction of my Skybolt. These pages are for information and personal entertainment only and not to be construed as the only way, or even the perceived correct way of doing things. You are responsible for your own construction techniques.