Detecting mutations from RNA-Seq (with a demo)

Detecting mutations from RNA-Seq is not a typical approach to detect mutations, but has drawn much attention in recent days. There are several reasons for that:

First, many RNA-Seq experiments are done mainly to detect expression changes of genes, transcripts, alternative splicings, and it has no extra costs to use computational approach to identify mutations. Second, several inverstigations show that mutations detectable from RNA-Seq accounts for up to 40% of those detected by DNA sequencing, and for some highly expression genes, this percentage may reach 80% (for example, see this paper). Third, it is much easier to do the RNA-Seq than whole genome sequencing or whole exome sequencing in terms of the sequencing cost, the amount of samples required, etc.

There are currently many different tools to detect mutations from high-throughput sequencing, for example, GATK. However, these algorithms are specifically designed for mutation detection in DNA sequencing, not RNA sequencing. The statistical evaluation of mutations in these softwares may not be suitable for RNA-Seq.

Here we present rnaseqmut, an easy-to-use program to detection variants in RNA-Seq read alignments. rnaseqmut is a light-weight C++ program to detect variants (or mutations, including SNPs, indels, RNA editing events) from a single or multiple RNA-Seq BAM files. It offers the following features:

 

• Perform de-novo mutation discovery from a given BAM file;
• For a user-defined mutation list, calculate the read coverage, including reads that support reference allele (reference reads) and alternative allele (alternative reads), from a given BAM file; 
• For a series of RNA-Seq samples, filter interesting mutations based on user-defined criteria. 

This software package includes a "core" C++ program, rnaseqmut, to call variants from a single BAM file, and a series of optional scripts, written in Python3 and bash, to identify putative interesting mutations in a group of RNA-Seq samples. Besides mutation detection from RNA-Seq, the "core" program (rnaseqmut) can also be used to call mutations from other high-throughput sequencing platforms, including ChIP-seq, DNA-Seq, etc.

For more information, check out in GitHub:

https://github.com/davidliwei/rnaseqmut

Demo: detecting mutations from a series of RNA-Seq BAM files

A demo script is provided in the demo directory, together with 4 sample RNA-Seq BAM files. In this demo, we provide two normal samples and two tumor samples, and would like to search for mutations that only occurs in one or more tumor samples, but not (or have low frequency) in any of the normal samples. We split this job into five steps:

• Step 1, scan the samples individually and get the mutation list for each sample. In this demo we output all possible mutations (those even occur in only 1 RNA-Seq read), but in reality you may just need those with enough read support (controlled by -i/--min_read option).

• Step 2, merge the mutation lists from individual samples. This will be the potentially interesting mutations we would like to investigate.

• Step 3, scan the samples again, using the provided mutation list in Step 2.

• Step 4, merge the mutations in step 3 into a big table. In this table, each row represents a mutation, and columns record the number of supporting reads (and the number of reference reads that do not support this mutation) in all samples. Based on this table, you can use your own criteria to search interesting mutations.

• Step 5 illustrates an example of filtering interesting mutations using a python3 script "filtermut.py". In this demo, we define "control" samples as two normal samples, and would like to look for mutations that satisfy the ALL of the following criteria:

1. occur in at least 1 non-control sample (controlled by -t/--min-recurrent option in filtermut.py) with at least 20% frequency (controlled by -f/--min-recfrac option) and 10 supporting reads (-d/--min-recread);
2. do not have any supporting reads in control samples (-b/--max-alt); 
3. have at least 4 non-supporting reads (reference reads) in control samples (-a/--min-ref). This requirement will exclude mutations with 0 read coverage in control samples thus we have no idea whether these mutations occur in control samples.

The final output is a VCF file recording mutations and read coverages in all 4 samples. You can also output tab-delimited file instead of VCF file for further downstream analysis (use -z/--no-vcf option in Step 5). For interpreting results, see the next section.

For more details, refer to the demo script and the usage of each programs/scripts.

Which tool is used for RNA-seq analysis?

Which tool is popular to perform RNA-seq analysis, including read mapping, gene/transcript expression level estimation and differential analysis? Here is the survey of which tool to used for RNA-seq analysis.

In summary, the majority will:

• aligns the reads to the genome (or genome+transcriptome) using Tophat;
• count reads using Cufflinks (the second choice is HTSeq-count, which is becoming popular);
• perform differential expression using DESeq/DEXSeq (followed by CuffDiff);
• use Ensemble (followed by Refseq/UCSC) as the annotation resource;
• use GOSeq (followed by IPA and Genego Metacore) for downstream analysis.

XML_C14N error when compiling libxmlsec

OK, as an bioinformatician I always work with compiling and running with many different tools. Here comes with the problem: I need to install the genefilter package in R, which is a great package for performing microarray analysis. In our institute cluster, it works well in the head node but not in other nodes. Further inspection shows that a static library, libxmlsec, is missing in these nodes. Compiling this library is painful as you have to deal with many different errors...

After a few days of struggle, I successfully compiled libxmlsec and now genefilter works fine for all nodes in the cluster. Here I just write down some error messages and my solutions, in cases others have the same problem.

XMLSEC compiling errors: XML_C14N_1_0 undeclared

The error message is something like:

c14n.c: In function 'xmlSecTransformC14NExecute':
c14n.c:423: error: 'XML_C14N_1_0' undeclared (first use in this function)
c14n.c:423: error: (Each undeclared identifier is reported only once
c14n.c:423: error: for each function it appears in.)
c14n.c:431: error: 'XML_C14N_1_1' undeclared (first use in this function)
c14n.c:439: error: 'XML_C14N_EXCLUSIVE_1_0' undeclared (first use in this function)

Solution:

XML_C14N_1_0 is defined in libxml2 header files, so check whether your libxml2 has some problems. More specifically, XML_C14N_1_0 is defined in "c14n.h", and older libxml2 headers this is not defined. For me this error occurs since I have an older version of libxml2 in the system, which the compiler overrides a newer version in my home directory. Here is the following command the MAKE used when error occurs:

gcc -DHAVE_CONFIG_H -I. -I.. -DPACKAGE=\"xmlsec1\" -I../include -I../include -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_DL_LIBLTDL=1, -I/usr/include/libxml2 -I/home/include/libxml2 -g -O2 -MT c14n.lo -MD -MP -MF .deps/c14n.Tpo -c c14n.c  -fPIC -DPIC -o .libs/c14n.o

Look at the include option (-I) in the above example: the system libxml2 headers are in front of my own libxml2 headers, so the system libxml2 headers (which are the older ones) are used instead of the newer libxml2 headers. My solution is to add a symbolic link in the "include" directory of libxml2 pointing to the new libxml2 headers in my home directory:

wl948@header:~/prog/libxmlsec/xmlsec1-1.2.18/include$ ln -s $HOME/include/libxml2/libxml libxml

which solve this issue because right now the newer libxml2 headers are in front of the system libxml2 files.

I'm sure there should be alternative solutions in the configure program by providing a proper option (probably by setting some environment variables) but I didn't check. Anyone has any ideas?

Estimating paired-end read insert length from SAM/BAM files

I wrote a single Python script to estimate the paired-end read insert length (or fragment length) from read mapping information (i.e., SAM/BAM files). The algorithm is simple: check the TLEN field in the SAM format, throw out pair-end reads whose pairs are too far away, and use them to estimate the mean and variance of the insert length.

This script is also able to provide a detailed distribution of read length and read span for your convenience. Please refer to the detailed usage below.

This script is distributed in GitHub now.

Usage:

getinsertsize.py [ SAM file | -]

or

samtools view [ BAM file ] | getinsertsize.py - 


Detailed Usage:


usage: getinsertsize.py [-h] [--span-distribution-file SPAN_DISTRIBUTION_FILE]
                        [--read-distribution-file READ_DISTRIBUTION_FILE]
                        SAMFILE

Automatically estimate the insert size of the paired-end reads for a given
SAM/BAM file.

positional arguments:
  SAMFILE               Input SAM file (use - from standard input)

optional arguments:
  -h, --help            show this help message and exit
  --span-distribution-file SPAN_DISTRIBUTION_FILE, -s SPAN_DISTRIBUTION_FILE
                        Write the distribution of the paired-end read span
                        into a text file with name SPAN_DISTRIBUTION_FILE.
                        This text file is tab-delimited, each line containing
                        two numbers: the span and the number of such paired-
                        end reads.
  --read-distribution-file READ_DISTRIBUTION_FILE, -r READ_DISTRIBUTION_FILE
                        Write the distribution of the paired-end read length
                        into a text file with name READ_DISTRIBUTION_FILE.
                        This text file is tab-delimited, each line containing
                        two numbers: the read length and the number of such
                        paired-end reads.

Sample output:

Read length: mean 90.6697303194, STD=15.9446036414

Possible read length and their counts:

{108: 43070882, 76: 50882326}

Read span: mean 165.217445903, STD=32.8914834802

Note: If the SAM/BAM file size is too large, it is accurate enough to estimate based on a few reads (like 1 millioin). In this case, you can run the script as follows:

head -n 1000000 [ SAM file ] |  getinsertsize.py -

or

samtools view [ BAM file ] | head -n 1000000 | getinsertsize.py -

Note: According to the SAM definition, the read span "equals the number of bases from the leftmost mapped base to the rightmost mapped base". This span is the distance between two reads in a paired-end read PLUS 2 times read length. Read span is different from the "mate-inner-distance" in Tophat (-r option), which measures only the distance between two reads in a paired-end read.

RNA-Seq Read Simulator

RNA-Seq is now a common protocol to study the expression of genes or transcripts. For research purposes, there are many simulators to simulate RNA-Seq reads, like Flux Simulator. But many times I found it hard to use because: 1) there are complicated parameters, 2) it requires large memory and 3) it crushes frequently. So I wrote a few scripts to generate simulated RNA-Seq reads, and publish them in a package "RNASeqReadSimulator".

 RNASeqReadSimulator is a set of scripts generating simulated RNA-Seq reads. It provides users a simple tool to generate RNA-Seq reads for research purposes, and a framework to allow experienced users to expand functions. RNASeqReadSimulator offers the following features:

1. It allows users to randomly assign expression levels of transcripts and generate simulated single-end or paired-end RNA-Seq reads. 
2. It is able to generate RNA-Seq reads that have a specified positional bias profile. 
3. It is able to simulate random read errors from sequencing platforms. 
4. The simulator consists of a few simple Python scripts. All scripts are command line driven, allowing users to invoke and design more functions.

The webpage of RNASeqReadSimulator is here. You can find the source code in GitHub.

Extract insert size of paired-end reads from Cufflinks output

The new version of Cufflinks tries to estimate the mean and std value of paired-end read insert size automatically. Sometimes we need to extract such information to check our sequencing data. Here I wrote a simple script to search Cufflinks outputs.

The script is written in Python. The implementation is easy: search for key words (like "Estimated Mean", etc) in log files. The usage is as follows:


This script extracts insert size information from Cufflinks logs.
Usage: getinsertsize [cufflinks log file]
Note: you may specify different log files using filename wildcards.

Sample output:

File    MapMass ReadLength      Mean    Std
cufflinksinvoke.sh.e2148442     34108402.45     85      150.35  19.18
cufflinksinvoke.sh.e2148443     31007287.59     85      155.19  17.45
cufflinksinvoke.sh.e2148444     37286038.02     85      123.87  25.60

Source code:

Qseq and export file format of Illumina

Most Illumina NGS data files we face are FASTQ/FASTA formats, which include the read sequence and (possible) quality scores. If reads are mapped to the chromosome or reference sequence, SAM/BAM file formats are common. However, sometimes we see other formats instead of these common file formats, for example .qseq or _export files. They are generated by earlier Illumina machines and it's best to convert them to commonly used FASTQ/SAM formats.

QSEQ file is the raw read file; export file format records the mapping location of the read, but the read sequence and quality is retained (like SAM/BAM format).


QSEQ File Formats

A sample qseq line is as follows:

SOLEXA5 1       4       1       1137    6698    0       1       TAAATCAAAAGCACAATGAGATATCAATTTTCACCCACTGGAATGGCTATA     aa]a]WY]F]aWaZWa]a][a]^]aaaaa_]Y``QaaaUa\aa]]YU`^]P     1

According to the Illumina Documentation (Pipeline CASAVA), here are the meanings of each field:

1. Machine name: (hopefully) unique identifier of the sequencer.
2. Run number: (hopefully) unique number to identify the run on the sequencer.
3. Lane number: positive integer (currently 1-8).
4. Tile number: positive integer.
5. X: x coordinate of the spot. Integer (can be negative).
6. Y: y coordinate of the spot. Integer (can be negative).
7. Index: positive integer. No indexing should have a value of 1. 
8. Read Number: 1 for single reads; 1 or 2 for paired ends.
9. Sequence 
10. Quality: the calibrated quality string.
11. Filter: Did the read pass filtering? 0 - No, 1 - Yes

EXPORT File Format

For export files, there are even more fields. Here is two sample lines of _export.txt files (notice that "|" is inserted between fields for better visualization; they are not included in original fields):

SOLEXA7_25_12N32AACC    |       |5      |1      |459    |1646   |       |1  |TGGGCCNACAACCCCGCACAGTCCCCNCCGCAACCCCCAGCGCTTGCCNC     |ENXXXUCXXXEEXXDXEXXXXPCXEXCLEGSNASSNJNKNHKAEHA?N?A     |NM     |       |       |       |       |       |       |       |       |       |       |N
SOLEXA7_25_12N32AACC    |       |5      |1      |560    |1611   |       |1      |GGCTTGGGAGCTGGTGCTTTCTTTTTTTCTTTTCTTTCTTTTTTTTTTTT     |YYYYYYXYYYYYYXYYYYYYYYYYYYYYYYSSSSSOOOOOOOOOOOOOON     |chr19  |       | 2649567        | R      |48C1   |46     |0      |       |       |0      |N      |Y

The fields are (also from Illumina Documentation):

1. Machine (Parsed from Run Folder name)
2. Run Number (Parsed from Run Folder name)
3. Lane
4. Tile
5. X Coordinate of cluster
6. Y Coordinate of cluster
7. Index string (Blank for a non-indexed run)
8. Read number (1 or 2 for paired-read analysis, blank for a single-read analysis)
9. Read
10. Quality string—In symbolic ASCII format (ASCII character code = quality value + 64)
11. Match chromosome—Name of chromosome match OR code indicating why no match resulted
12. Match Contig—Gives the contig name if there is a match and the match chromosome is split into contigs (Blank if no match found)
13. Match Position—Always with respect to forward strand, numbering starts at 1 (Blank if no match found)
14. Match Strand—“F” for forward, “R” for reverse (Blank if no match found)
15. Match Descriptor—Concise description of alignment (Blank if no match found)
• A numeral denotes a run of matching bases
• A letter denotes substitution of a nucleotide: For a 35 base read, “35” denotes an exact match and “32C2” denotes substitution of a “C” at the 33rd position
16. Single-Read Alignment Score—Alignment score of a single-read match, or for a paired read, alignment score of a read if it were treated as a single read. Blank if no match found; any scores less than 4 should be considered as aligned to a repeat
17. Paired-Read Alignment Score—Alignment score of a paired read and its partner, taken as a pair. Blank if no match found; any scores less than 4 should be considered as aligned to a repeat
18. Partner Chromosome—Name of the chromosome if the read is paired and its partner aligns to another chromosome (Blank for single-read analysis)
19. Partner Contig—Not blank if read is paired and its partner aligns to another chromosome and that partner is split into contigs (Blank for single-read analysis)
20. Partner Offset—If a partner of a paired read aligns to the same chromosome and contig, this number, added to the Match Position, gives the alignment position of the partner (Blank for single-read analysis)
21. Partner Strand—To which strand did the partner of the paired read align? “F” for forward, “R” for reverse (Blank if no match found, blank for single-read analysis)
22. Filtering—Did the read pass quality filtering? “Y” for yes, “N” for no

Conversion 

Here I provide some useful tools and scripts for conversion of QSEQ and EXPORT files.

Samtools provides a script "export2sam.pl" to convert export format to SAM format.

For qseq to FASTQ conversion, Xi Wang (from SeqAnswer.com) provides a very simple Perl script. I modified it a little bit to filter quality control reads (those with the 11th field is 0). If you want to keep these reads (which are not recommended), remove the if condition below.

#!/usr/bin/perl

use warnings;
use strict;

while (<>) {
	chomp;
	my @parts = split /\t/;
	if($parts[10] == 1){ # remove this if you want to keep quality control reads
		print "@","$parts[0]:$parts[2]:$parts[3]:$parts[4]:$parts[5]#$parts[6]/$parts[7]\n";
		print "$parts[8]\n";
		print "+\n";
		print "$parts[9]\n";
	}
}

It works pretty well for export file formats.

For export to FASTQ convert, the latestst MAQ package (>0.6.6) provides a script fq_all2std.pl to do this:

Usage: fq_all2std.pl
Command:
scarf2std Convert SCARF format to the standard/Sanger FASTQ
fqint2std Convert FASTQ-int format to the standard/Sanger FASTQ
sol2std Convert Solexa/Illumina FASTQ to the standard FASTQ
fa2std Convert FASTA to the standard FASTQ
seqprb2std Convert .seq and .prb files to the standard FASTQ
fq2fa Convert various FASTQ-like format to FASTA
export2sol Convert Solexa export format to Solexa FASTQ
export2std Convert Solexa export format to Sanger FASTQ
csfa2std Convert AB SOLiD read format to Sanger FASTQ
instruction Explanation to different format
example Show examples of various formats