My publishing tool chain

Describes my current publishing tool chain, from latex to pandoc.

Introduction

The following paragraphs describe my publishing tool chain. I am working on my doctorate right now, therefore, my needs are pretty clear: (a) I need to maintain a bibliography, (b) I need to publish to various target formats from one source file, and (c) References and quoting needs to follow my faculty’s standards.

Bibliography

My bibliography is very important for me, it contains an almost complete history of readings and research. Complying with referencing requirements is mandatory. As a result, I maintain my bibliography with almost pedantic precision. The file format of choice is bibtex. The official bibtex homepage provides a good introduction to bibtex. Bibtex stems from the latex tool chain, which I used a lot until I encountered Pandoc. Bibtex is also the name of the bibtex tool, but it is irrelevant in my tool chain, as I just use the bibtex format to store references. The upside of using the bibtex filesystem is that there’s a whole ecosystem around it, such as Zotero which can import and export bibtex files.

Writing

I used to write in a little niche tool called LyX. Lyx boasts excellent latex support and a highly reduced user interface that stimulates writing and thinking instead of focusing on formatting - unlike MS Word. As mentioned before, I encountered pandoc a couple of weeks ago and ditched latex in favor of pandoc. You can find more information about pandoc below. Pandoc’s file format of choice is markdown. Markdown is a simple, text based formatting approach for text that is easy to read and write, even in plaintext. As a result, I can edit my articles in literally any text editor. It helps to change the font of my text editor to Times New Roman in order to get into a better writing flow.

My current editor of choice is SubLime. Noteworthy plugins are the WordCounter-st3 package and MarkDownEditing package, which both help to stick to a word count and improve the markdown editing experience. I also started using the GhostWriter editor, which gives me a very hassle free editing experience. I am not yet decided if I prefer Sublime or GhostWriter, time will tell.

The next step is to compile my markdown file into some target format. My publishing target requires HTML files and PDF files. As a result, I ended up using pandoc. References need to follow the Harvard standard, and in particular the CiteThemRight standard. One decision factor has been the large amount of reference formatting styles contained in the Zotero Style Repository and the Citation Styles database. These styles follow the Citation Style Language. Readers that have spent time to create own formatting classes for bibtex know how cryptic bibtex’ formatting rules can become (detail: some strange Reverse Polish Notation in Computer Science). Compared to latex/bibtex style files, CSL files are much easier to modify. I ended up using Cite Them Right 10th edition - Harvard. To compile, I use the following bash file …

#!/bin/bash
export PATH=/home/ustaudinger/.local/bin:$PATH
f=$1
echo $f

echo "Precompiling references ..."

# precompile to a file with references ....
pandoc $f ../../bibliography/default.yaml -F pandoc-crossref  -f markdown-tex_math_dollars ---bibliography ../../bibliography/bibliography.bib --csl ../../bibliography/harvard-cite-them-right.csl t markdown-citations -s -o $f.tmp


echo "Compiling to HTML" 
#pandoc ../../bibliography/default.yaml $f.ref.md -o $f.html
pandoc  ../../bibliography/default.yaml -F pandoc-crossref  --number-sections    -s --bibliography ../../bibliography/bibliography.bib --csl ../../bibliography/harvard-cite-them-right.csl -f  markdown-tex_math_dollars $f -o $f.html

echo "Compiling to PDF"
pandoc ../../bibliography/default.yaml --template=../../bibliography/template.tex -s --bibliography ../../bibliography/bibliography.bib --csl ../../bibliography/harvard-cite-them-right.csl  -f  markdown-tex_math_dollars --filter pandoc-crossref --filter pandoc-citeproc --filter pandoc-citeproc-preamble -M citeproc-preamble=../../bibliography/preamble.tex $f -o $f.pdf

echo "Compiling to TEX"
pandoc ../../bibliography/default.yaml --biblatex --template=../../bibliography/template.tex  -s --bibliography ../../bibliography/bibliography.bib --csl ../../bibliography/harvard-cite-them-right.csl  -f  markdown-tex_math_dollars  --filter pandoc-crossref --filter pandoc-citeproc --filter pandoc-citeproc-preamble -M citeproc-preamble=../../bibliography/preamble.tex $f -o $f.tex

echo "Compiling to WORD"
pandoc ../../bibliography/default.yaml   -s --bibliography ../../bibliography/bibliography.bib --csl ../../bibliography/harvard-cite-them-right.csl -f markdown-tex_math_dollars  --filter pandoc-crossref $f -o $f.docx

echo "Done. "

… as you can see, I reference my bibliography, a default yaml file, a custom tex template and several filters.

Publishing

As a final step, I upload my PDFs to my target platform and copy-paste my HTML into some discussion forum. Pandoc’s capabilities to publish to many different target formats are one of its biggest advantages over latex. While some use cases might require sophisticated latex setups, pandoc does a good job at abstracting and hiding the more complicated details of latex behind an easy to use command line interface.

Now, how do I get readers!?

Updates

(1. Jan 2018) I stumbled upon @Smith2016, who describes an alternative approach, but highly similar to mine to academic publishing with pandoc. His article is more comprehensive, and I like the idea of makefiles, too.