[79FT]: Building Things
How I went about b-logging
|On:||May 20, 2014|
|In:||[Misc] B Log|
|Tags:||tools, software, website|
Note: this article was updated in 2018 to reflect changes I did in version 2 of B-Log. I want to have a single article to link to in case people are interested...
Notes on the update are here.
... was that I needed something to organize my builder's log. Being a software engineer by trade, I didn't want to go old school with paper logs and pictures, and whatever-random-collection-of-stuff-on-a-computer I could do. I wanted something that's easy to add entries to, calculates everything automatically, and allows me to publish it on the Web. Plus, I find it pleasant to view HTML as opposed to .doc, .pdf, or whatever.
Being a nerd, I just had to make it as nice for myself as I could...
Many-a-builder is using this apparently successful piece of work. Apparently, as of writing this, they have over 1100 logs published on their mykitlog.com.
I have played around with it quite a bit. The program is trivially simple; but I can't stand the way it's architected.
Oh, the Access databases. Basically, what this means to me as a user is if it ever .. erm.. decides to stop working, I will loose all my data. "Revert to a backup", you say.. Yes, but if the problem is that I can't add my N+1st record because the DB just corrupted itself, no backup will help -- I'm stuck.
On top of that if, god forbid, KitLog ever goes out of business, I am stuck with the data that I can't export w/o some serious reverse engineering.
No thank you. :)
So, I was toying around with other stuff.
Really cool (probably, second best to WordPress) blogging platform. Plus, links very nicely with Picasa, making image inserts a breeze.
The problem? How do I track time spent?
Also, the "going-out-of-business" problem was still there...
Now, that was better. Those are open source, good content entry and organization tools, and I can export it all out of a MySQL DB should I need to.
But, they require a server (which means either I'm running a server, or require hosting). And frankly, I believed that there must be a better way...
Close, but no cigar. Sphinx is the documentation framework used by Python and many others (including the company I work for).
It takes a bunch of ReST files as input (more on that later), and runs them through a converter, producing whatever you want (there are writers for html, txt, doc, pdf, and so on). The parser and converter is called docutils.
The problem with Sphinx is that it's geared towards writing documentation, and frankly, felt a bit too heavy to me for writing a builder's log.
While googling about yet-another-thingie-I-wanted-to-figure-out-about-Sphinx, I stumbled upon an article that talked about static site generators. There lied the Holy Grail....
The Pelican. I fell in love with this thing almost immediately. Here's what it does:
I did the initial version in 2014 (the date of this article). Then, I've spent about a week tearing it to pieces and figuring out how it works, and adding a few plugins to add things that I needed for the B-Log that it didn't have. I added time tracking, changed layout, got a Bootstrap-based template and modified it, etc.
In 2018, I fixed some suff and added a few more things.
You be the judge of results :).
Here's what I've done, in the end:
I like the end-result. I enjoyed hacking at it, and this was the first time I touched Python (so had to learn that on the way). Pretty neat (though I still prefer Perl (now, why do you need lists and tuples and sets again? :) ).
The only drawback is that it's regenerating everything every time. You can easily see how that might become a problem when you get a lot of entries, but I'm feeling like I'm ways away. 30 pages take about a second. I think I will need a lot of records before this becomes a problem; and at that point I will rewrite the whole damn thing to cache stuff.. probably :)
The code is here. The repo includes my theme, plugins, some tools, and sample content. It requires Python (I used 3.6 and did not test anything else), Pelican (which is checked into the repo in pelican/, along with it's dependencies), and Pillow (pip install pillow). Oh yeah, and pelicanconf.py's PATH_METADATA will likely require tweaking of slashes on Windows.
Read the config file (pelicanconf.py); and read thru Pelican docs to get the idea. sample-content/ contains my sample generated content that I was testing with. content-gen/ contains the simple Perl script that generates as much sample content as you want.
To me as an end-user, it looks like this.
My source files are organized in directories like so:
/project-one/section-one/ 2001-01-01-entry-one/ entry-one.rst picture.jpg section-one-log.log /project-one/section-two/ 2001-01-02-entry-two/ entry-two.rst picture.jpg section-two-log.log /project-two/section-one/ 2002-01-01-entry-one/ entry-one.rst picture.jpg /project-two/section-two/ 2002-01-02-entry-two/ entry-two.rst picture.jpg
and so on. Note the text *.log files - those are just log entries without the articles. They are optional, as you can see in Project Two.
Text log files look like this:
2011-01-01 | Some Summary Here | 1.02 2011-01-02 | Some Other Summary Here | 2.02
Note the separator - that's configurable :).
There can be as many log files in a "Project/Section" as one wishes, they will all be parsed, and all entries will be properly sorted by date.
There can be entire sections without articles - but a project must have at least one article to be properly displayed. Yeah, you got to at least declare your intentions :).
This directory structure will get parsed into two Projects (Project One and Project Two) with their separate logs. Each project will get it's own menu on the left. To display the menus, B_LOG_PROJECTS_MENU config var should list them.
If there was any time logged in a project (either in article entries, log files, or both), that project's menu will get an extra "Log" entry, which will neatly display all log entries, and some totals.
There are a few special directories I use, pages (configurable in PAGE_PATHS) for static pages (ie, About section of this site is a static page), and blog, for blog entries (they don't show up under any projects).
If I want a new article-type entry, I just add a directory in the project/section path, and start writing the article in my favorite text editor (VIM!!). If I decide to log time, I just use my special logged metadata entry at the top of the article. I drop images into article's directory, and use my figure ReST directive to reference them.
If I just need to log some time without polluting the Internet too much with useless text, I add it to an appropriate .log file, or create a new one.
If I don't want to log time, I don't have to - I can just post an entry into the project w/o time logged.
After I'm done, I run Pelican, which generates the full website (html, images, everything!) into an output directory. Voila! I can stop right there, and I will have a fully working local version of my log, since it's all static HTML, but we want this online, don't we? So, the only thing left is...
This was another revelation. Apparently, GitHub hosts static pages! 'Uploading a site' is a git push, and it transfers only deltas, so no fiddling with FTP, rsync, or whatever! They allow custom domain names instead of username.github.io, all you need is to set your A records in your DNS to point to their servers. It's awesome!
I am very happy with this so far. I have all my logs in practically plain text format, parser for which is open source and available. I have it nicely organized, and I can back it up as I please (to GitHub, anyone? :) ). I can re-write the programs that parse and produce final .html. I can extend the generator and change the theme to produce a printable version of the log, styled for paper rather than web. The list goes on!
You start with a bag full of luck and an empty bag of experience. The trick is to fill the bag of experience before you empty the bag of luck.
© Copyright "79FT". All rights reserved. Feel free to cite, but link back to the pages cited.
This website only shows how I did things in my various projects. These pages are for information and personal entertainment only and not to be construed as the only way, or even the perceived correct way of doing things. You are responsible for your own safety and techniques.