A little markdown goes a long way

Write down what you did. Documenting your work in science is an essential skill. UNIX is no different. Documenting your actions is a good habit, and a necessary step for reproducing your work and sharing it with others. Write down the objectives of your project and the questions you are trying to answer (what and why). Record all the actions you took to analyze the data (how). Write down your observations and your conclusions from each figure or step in the analysis. If the project looks like a failure, importantly, don't just let it drop - write down why you think it failed. Write down your ideas about the experiment or the data every chance you get.

A simple text file can often work as a universal medium for capturing a workflow and if you develop a good habit of writing stuff down - your life will be much easier (as well as those who continue or interpret your work).

Markdown is a simple way of marking text to add formatting that makes the information more structured and readable as is, yet can also be rendered for html or pdf.

This looks like an important head line
======================================

By underlining text we can transform it 
to a Level 1 header.


This looks like a less important headline
-----------------------------------------

Using a more subtle underline results in 
a smaller header (Level 2).

There is an alternate way to make section 
headers. Simply use the pound sign.

# This is a level 1 header

## This is less important (level2)

### Level 3 header

And so on to level 6, the standard stuff
available in html documents.

If you use some conventions like this when you write your text file of notes, you can easily render it in pdf or html using pandoc. See the result.

Here is an example showcasing many of the different markdown features that can be used to give structure and form to a document.

Experiment Protocol by Markdown Example
=======================================

For today's experiment we will be analyzing the following
blockquote:

> ...the **twist** transcription factor binds broadly
> across the genome and regulates important developmental
> events.

### What we need

To accomplish this we need three things from the following list:

- an installed aligner
- a working directory
- a set of [fastq files](http://compgenomics/Data/ChIP-Seq/)

The mutants are summarized in the following Table:

Mutation | Dorsal | Result
-------- | ------ | ------
gd7      | none   | gd7_1.fastq.gz
Tol10b   | high   | tol10b_1.fastq.gz

The steps we will take are indicated in this ordered list:

1. Create a working directory
2. write a simple alignment script
    * see the class examples
3. check the server *before* we run the script

#### We also need to write some shell code 

our script might look something like:

```shell
    for f in FILES
    do
      tophat f
    done
```

We ran it on maple, which has ~~12 cores~~ 24 cores.

---

The example above is readable as is. But if we want to convert it to pdf or html, we can call the pandoc program to convert it. For instance, if the document above is found in the file: experiment.txt, we would convert it to html as follows. We use the -o option to tell pandoc where to put the output, and it guesses if we want html or pdf (or others) based on the file extension we give it.

pandoc experiment.txt -o experiment.html

If we want to add custom stylings to the html elements, we can use css, and add a reference to it at the top of the document. See the results:

experiment.txt (original) | experiment.html (html) | experiment_css.html (custom styles) | experiment_css.pdf

Pandoc is a flexible document conversion tool. For instance if you want to read a Microsoft Word document from the command line you can convert it to plain text as follows:

pandoc mydocument.docx -t plain

and you'll be able to read the results like a regular text document.

There are other ways of following the "markdown" concept (knitr, emacs org mode, etc.). You'll be working with RMarkdown documents later in the course.

Caveat: The section below ONLY APPLIES if you are on the Stowers Intranet, or VPN.
Since we are not on the internal network, the mechanism below WILL NOT WORK.

What's the point of converting your work to html anyway? One reason is so that you can easily share it with others (and keep things tidy for yourself). We have a very easy and flexible way to share documents with each other here at the institute called the bioinfo server. Documents and directories can be referenced via URL on our intranet. Almost regardless of what filesystem your work is in, you can still point to it or access it as follows:

https://webfs is the base URL, and then depending on what file system you're working in, you can form the rest of the URL. For instance:

Since franklin.stowers.org has a web server built in, we can serve documents through our public_html directory. There are two ways to do this.

1. Place any documents you want to serve in your public_html directory, and make sure they have the correct permissions set.
2. If you have a directory of results or documents you'd like to share to the world, you can create a link to this directory in your public_html directory, and it will be available for sharing.

Since we usually have working project directories in our home directory, I think it makes more sense to create our analysis documents in our working directory and link them to public_html. For instance, if you had a working directory for a project called project1, with subdirectories for the analysis, and BAM files and MACS results like so:

project1
|
├── analysis
│   ├── analysis1.html
│   ├── analysis1.Rmd
│   └── tracks
├── BAM
└── MACS

You could make the analysis documents and track data available to the world using a symbolic link into public_html like so:

ln -s /home/username/project1/analysis /home/username/public_html/project1

In this way, just using my own username as an example, when someone visited: http://franklin.stowers.org/~cws/project1, they would see the contents of: /home/cws/project1/analysis

To style the page elements using CSS (Cascading Style Sheets), you can insert a line like the following at the top of your markdown document, containing a local or remote reference to your styles.css file (the example below calls a remote reference).

<link href="https://research.stowers.org/cws/css/cws_md.css" rel="stylesheet"></link>

• Markdown cheatsheet
• A Guide to Organizing Computational Biology Projects