Demo: phyloseq – A Bioconductor package for handling and analysis of high-throughput phylogenetic sequence data
Paul J. McMurdie and Susan Holmes
Statistics Department, Stanford University,
Stanford, CA 94305, USA
E-mail
mcmurdie@stanford.edu
susan@stat.stanford.edu
Websites
joey711.github.com/phyloseq
http://www-stat.stanford.edu/~susan/
Summary and Other Documentation Resources
This document supports a live demonstration of tools in the phyloseq package, and supplements other documentation resources available for the phyloseq package (e.g. wiki, vignettes, publications, function-level documentation, etc.). It is built automatically from its R-markdown source file with example code-chunks that can be reused if you have phyloseq installed. The R-markdown file and other related materials are publicly available, and hosted on GitHub with an explicit open-access license/ copyright statement:
There is also a GitHub-hosted issue-tracker for phyloseq, currently describing over 100 feature requests, bug reports, documentation revisions, help requests, and other suggestions. Only a small fraction of these issues are still outstanding, but the descriptions remain available for anyone to view and add comments/code.
The phyloseq package is under active development. Users are encouraged to consistently update their version from the phyloseq development website on GitHub. The most stable releases and development versions of phyloseq are hosted by Bioconductor. For installing from Bioconductor, and alternatives to "the bleeding edge", as well as the most updated installation news and instructions, please see the installation wiki page.
Example: Manually re-import the "esophagus dataset", which is already inclduded in the phyloseq package. Note that the esophagus dataset is a simple dataset consisting of just 3 samples and a relatively small richness, described with just a tree and OTU table. It is most useful for showing quick examples, such as computing the UniFrac distance, but for not much else. If interested, further details can be found by entering ?esophagus.
The biom-format is intended to be a complete representation of the OTU-clustering results, and so import can be performed with just one connection/file path.
The biom-format definition allows for both sparse and dense representations of the abundance data, and is also flexible enough to allow a "minimal" (abundance table onle) and "rich" forms (includes sample and taxonomy data). All of these forms are supported and automatically recognized/interpreted in phyloseq through the import_biom function.
# Define file path to all four format combinations
rich_dense_biom <- system.file("extdata", "rich_dense_otu_table.biom", package ="phyloseq")
rich_sparse_biom <- system.file("extdata", "rich_sparse_otu_table.biom", package ="phyloseq")
min_dense_biom <- system.file("extdata", "min_dense_otu_table.biom", package ="phyloseq")
min_sparse_biom <- system.file("extdata", "min_sparse_otu_table.biom", package ="phyloseq")
# Each import is only one line, we will import four different example# 'datasets'
rich_dense <- import_biom(rich_dense_biom, taxaPrefix ="greengenes")
rich_sparse <- import_biom(rich_sparse_biom, taxaPrefix ="greengenes")
min_dense <- import_biom(min_dense_biom, taxaPrefix ="greengenes")
min_sparse <- import_biom(min_sparse_biom, taxaPrefix ="greengenes")
# print summary if phyloseq, the component class otherwise.
biom_ex_print <- function(i) {
if (class(i) != "phyloseq") {
class(i)
} else {
i
}
}
sapply(list(rich_dense, rich_sparse, min_dense, min_sparse), biom_ex_print)
## [[1]]
## phyloseq-class experiment-level object
## OTU Table: [5 taxa and 6 samples]
## taxa are rows
## Sample Data: [6 samples by 4 sample variables]:
## Taxonomy Table: [5 taxa by 7 taxonomic ranks]:
##
## [[2]]
## phyloseq-class experiment-level object
## OTU Table: [5 taxa and 6 samples]
## taxa are rows
## Sample Data: [6 samples by 4 sample variables]:
## Taxonomy Table: [5 taxa by 7 taxonomic ranks]:
##
## [[3]]
## [1] "otu_table"
##
## [[4]]
## [1] "otu_table"
##
For convenience and stability, these "Bushman" data files were saved locally and imported from their local system location. An even more complicated "direct import" example is provided in the last section ("Extra Example") below, but produces the same result and is not run by the embedded code.
Import Bushman data, already downloaded.
uzdir <- "/Volumes/media/Research/study_1011_split_library_seqs_and_mapping/"
biom_file <- paste(uzdir, "study_1011_closed_reference_otu_table.biom", sep ="")
map_file <- paste(uzdir, "study_1011_mapping_file.txt", sep ="")
# Now import the .biom-formatted otu_table-tax_table file.
biom_otu_tax <- import_biom(biom_file, "greengenes")
# Add sample data to the dataset using merge
bmsd <- import_qiime_sample_data(map_file)
class(bmsd)
## phyloseq-class experiment-level object
## OTU Table: [1873 taxa and 100 samples]
## taxa are rows
## Taxonomy Table: [1873 taxa by 7 taxonomic ranks]:
Merging datasets or components
We need to merge these two separate Bushman dataset objects into one "phyloseq" object. Presently, the two data objects contain the otu_table, taxonomyTable, and sample_data components, respectively. If we had three objects that were all components (think single tables, or a tree), then we would use the constructor function, phyloseq. However, because the .biom file contained two tables (including an otu_table), the import_biom function returned a valid "phyloseq-class" instance instead that contained both components. Whenever you need to add or merge data componentes from one (or more) phyloseq-class objects, the merging function, merge_phyloseq, is recommended, rather than the constructor (phyloseq).
Extra Example: Direct ftp Download, Unzip, and Import
The .biom and sample data files are also provided online (ftp), and a useful way to download and import into phyloseq directly from the ftp address in the following example code. This is an example in which we download a zip file with both biom- and qiime-formatted data, unzip it in a temporary directory from with in R, import the relavant files using phyloseq importers, and then delete the temporary files. This code should be platform independent, but occasionally there are finicky Windows issues that arise.
(Note: this is not actually run in this demo. Would be redundant, and occasionally Windows issues might crash it, based on experience.)
# zipftp <-# 'ftp://thebeast.colorado.edu/pub/QIIME_DB_Public_Studies/study_1011_split_library_seqs_and_mapping.zip'# # First create a temporary directory in which to store the unpacked# file(s) from the .zip tmpdir <- tempdir() # Second create a temp file# where you will put the .zip-file itself temp <- tempfile() # Now# download the file and unzip to tmpdir directory download.file(zipftp,# temp) unzip(temp, exdir = tmpdir ) # Define the biom file-path biom_file# <- file.path(tmpdir, list.files(tmpdir, pattern='.biom')) # Define the# mapping file-path map_file <- file.path(tmpdir, list.files(tmpdir,# pattern='mapping')) # Now import the .biom-formatted# otu_table/taxonomyTable file. biom_otu_tax <- import_biom(biom_file,# 'greengenes') # Add sample data to the dataset using merge bmsd <-# import_qiime_sample_data(map_file) # Remove the temperorary file and# directory where you unpacked the zip files unlink(temp) unlink(tmpdir)
Basic Interaction with phyloseq Data
Let's look at some basic print and accessor functions/methods provided in phyloseq.
Print method
Bushman
## phyloseq-class experiment-level object
## OTU Table: [1873 taxa and 100 samples]
## taxa are rows
## Sample Data: [100 samples by 225 sample variables]:
## Taxonomy Table: [1873 taxa by 7 taxonomic ranks]:
What if we wanted to change the rank_names returned in the previous chunk? Since we know (in this case) that the "greengenes" taxonomy tools were used for the assignment we actually then know the names of the seven taxonomic ranks used. Just assign these to the colnames of the Bushman taxonomy table.
colnames(tax_table(Bushman)) <- c(k ="Kingdom", p ="Phylum", c ="Class",
o ="Order", f ="Family", g ="Genus", s ="Species")
Abundance Accessors
The purposes of the sample_sums and taxa_sums function are pretty straightforward, but the get_sample and get_taxa functions can bet a bit confusing.
Note how a sample name is required by get_taxa, and vice versa. This might seem confusing at first, but get_taxa is returning all the OTU abundances from one sample, while get_sample is returning the abundances from all samples for one OTU.
Some examples for plotting richness estimates from un-trimmed data.
plot_richness_estimates(Bushman)
plot of chunk plot-richness-1
(p <- plot_richness_estimates(Bushman, x ="SEX"))
plot of chunk plot-richness-1
p + geom_boxplot(data = p$data, aes(x = SEX, y = value, color =NULL), alpha =0.1)
plot of chunk plot-richness-1
Some others you might try (not run in demo)
plot_richness_estimates(Bushman, x ="INSOLUBLE_DIETARY_FIBER_G_AVE")
plot_richness_estimates(Bushman, x ="AGE_IN_YEARS")
plot_richness_estimates(Bushman, x ="VEGETABLE_PROTEIN_G_AVE")
Plotting an Annotated Phylogenetic Tree
A useful display on a phylogenetic tree is to add points next to tips/leaves/OTUs to represent samples in which the OTU was observed. This is facilitated in the phyloseq package through the plot_tree function, which produces a ggplot-based phylogenetic tree, and also allows several options for mapping color, shape, and size of these sample points to variables in the dataset. These point aesthetics can be mapped to sample data or to taxonomic data, depending on needs and which information needs to be reinforced in your graphic.
Caution: Trying to plot too many taxa (tree tips) at once obscures meaning. Let's look at just the Chlamydiae phylum in the incldued GlobalPatterns dataset. Note that this also requires subsetting the GlobalPatterns dataset using the subset_taxa function, part of the "preprocessing" tools described in the following section.
data(GlobalPatterns)
GlobalPatterns
## phyloseq-class experiment-level object
## OTU Table: [19216 taxa and 26 samples]
## taxa are rows
## Sample Data: [26 samples by 7 sample variables]:
## Taxonomy Table: [19216 taxa by 7 taxonomic ranks]:
## Phylogenetic Tree: [19216 tips and 19215 internal nodes]
## rooted
Map the sample environment ("SampleType") to point color, and taxonomic family to point shape. Additionally, label the tips with the Genus name and scale the point size by abundance
plot_tree(GP.chl, color ="SampleType", shape ="Family", label.tips ="Genus",
size ="abundance")
plot of chunk plot_tree-chlam
This figure contains the graphic produced by the plot_tree function in phyloseq. In this case the data is a subset of the GlobalPatterns dataset in which only OTUs from the phylum Chlamydiae are included. Additionally, the tree has been annotated with genus labels at each tree tip. The points next to tips represent samples in which the OTU was observed, and are shaped according to taxonomic rank of Family, and shaded according to the sample type (sample source environment).
plot_tree(GP.chl, "treeonly")
plot of chunk unnamed-chunk-11
This figure is the result of plotting the "bare" tree with no options directing the mapping of a variable to the tree, and "treeonly" as the argument to method. Not as informative as the previous tree.
Abundance bar plots
For direct quantitative observation/comparison of abundances.
data(enterotype)
TopNOTUs <- names(sort(taxa_sums(enterotype), TRUE)[1:10])
ent10 <- prune_taxa(TopNOTUs, enterotype)
plot_taxa_bar(ent10, "Genus", x ="SeqTech", fill ="TaxaGroup")
plot of chunk plot_taxa_bar-ex1
plot_taxa_bar(ent10, "Genus", x ="SeqTech", fill ="TaxaGroup") + facet_wrap(~Enterotype)
plot of chunk unnamed-chunk-12
This one takes a little while to calculate (not run).
This section includes examples preprocessing (filtering, trimming, subsetting, etc) phyloseq data. Let's start by resetting the GlobalPatterns example data and adding a human category.
data(GlobalPatterns)
GlobalPatterns
## phyloseq-class experiment-level object
## OTU Table: [19216 taxa and 26 samples]
## taxa are rows
## Sample Data: [26 samples by 7 sample variables]:
## Taxonomy Table: [19216 taxa by 7 taxonomic ranks]:
## Phylogenetic Tree: [19216 tips and 19215 internal nodes]
## rooted
# prune OTUs that are not present in at least one sample
GP <- prune_taxa(taxa_sums(GlobalPatterns) > 0, GlobalPatterns)
# Define a human-associated versus non-human categorical variable:sample_data(GP)$human <- factor(get_variable(GP, "SampleType") %in% c("Feces",
"Mock", "Skin", "Tongue"))
"Rarefy" Abundances to Even Depth
Although of perhaps dubious necessity, it is common for OTU abundance tables to be randomly subsampled to even sequencing depth prior various analyses, especially UniFrac. Here is an example comparing the UniFrac-PCoA results with and without "rarefying" the abundances (Requires phyloseq v1.1.28+)
GP20 <- prune_taxa(names(most_abundant_taxa), GP)
# Exploratory tree #1 (Note, too ma) plot_tree(ex2, color='SampleType',# label.tips='Family', size='abundance')ntaxa(GP20)
## [1] 20
length(get_taxa_unique(GP20, "Phylum"))
## [1] 5
That was prune_taxa, applicable when we know (or can provide) the OTU IDs of the OTUs we want to retain in the dataset, or a logical vector with the same length as ntaxa reseulting from a test (useful for genefilter or`genefilter_sample results (see next section)).
Alternatively, you can subset based on taxonomy expressions, using subset_taxa.
This tool takes after the genefilter function from the genefilter package, but emphasizes within-microbiome conditions. The following code illustrates . The function topp is a filter function that returns the most abundant p fraction of taxa. The filterfun_sample function takes one or or more functions like topp and binds them in order to define a filtering protocol function, in this case called: f1. This function, f1, is then passed to genefilter_sample along with the dataset that is going to be pruned as well as a value for A, the number of samples in which an OTU must pass the filtering conditions.
topp(0.1)
## function (x)
## {
## if (na.rm) {
## x = x[!is.na(x)]
## }
## x >= sort(x, decreasing = TRUE)[ceiling(length(x) * p)]
## }
## <environment: 0x109ab9d20>
f1 <- filterfun_sample(topp(0.1))
print(f1)
## function (x)
## {
## fun = flist[[1]]
## fval = fun(x)
## for (fun in flist[-1]) {
## fval = fval & fun(x)
## }
## return(fval)
## }
## <environment: 0x109250808>
## attr(,"class")
## [1] "filterfun"
wh1 <- genefilter_sample(GP, f1, A =round(0.5 * nsamples(GP)))
sum(wh1)
## [1] 793
ex2 <- prune_taxa(wh1, GP)
print(GP)
## phyloseq-class experiment-level object
## OTU Table: [18988 taxa and 26 samples]
## taxa are rows
## Sample Data: [26 samples by 8 sample variables]:
## Taxonomy Table: [18988 taxa by 7 taxonomic ranks]:
## Phylogenetic Tree: [18988 tips and 18987 internal nodes]
## rooted
print(ex2)
## phyloseq-class experiment-level object
## OTU Table: [793 taxa and 26 samples]
## taxa are rows
## Sample Data: [26 samples by 8 sample variables]:
## Taxonomy Table: [793 taxa by 7 taxonomic ranks]:
## Phylogenetic Tree: [793 tips and 792 internal nodes]
## rooted
Filtering low-variance OTUs
Suppose we wanted to use the variance of OTUs across samples as a condition for filtering. For example, to remove OTUs that do not change much across all (or most) samples. Note that the value of the variance is highly-dependent on the sequencing effort of each sample (the total number of reads sequenced from a particular sample). Thus, we must first transform the sample counts to relative abundance, which is shown in more detail in the next section. The following code will create a version of the GP dataset in which the abundance values have been transformed to relative abundance within each sample, and then OTUs have been filtered to keep only those with variance greater than 0.00001 (assuming we wanted to pick an arbitrary threshold in this way).
Useful for: Standardization / Normalization / Smoothing / Shrinking. Second-order function: transform_sample_counts. Also the threshrank and threshrankfun functions.
For more advanced normalization features (like shrinkage, etc.), also consider features in the edgeR package, the DESeq package, and for standardization the decostand function in the vegan-package; as well as probably many others that could be useful in this context.
Graphics for Inference and Exploration
In the following section(s) we will illustrate using graphical tools provided by the phyloseq package. These are meant to be flexible ways to explore and summarize the data.
For further details, there is a collection of "show and tell" wiki-pages describing the available graphics options supported by the phyloseq-package. These include example code for reproducing the figures shown. Many of the default settings are modifiable within the function arguments directly, and virtually everything about these plots can be further modified via the layers interface of ggplot2.
For quick reference (even though some have been described already), the key graphics-producing functions in phyloseq are:
Let's (re)load the Global Patterns dataset, prune the empty taxa, and add a custom sample variable called "human", a logical indicating whether or not the samples are human-associated.
We are going to show some examples that would take a lot of time to calculate and render on a dataset as large as GlobalPatterns. For the sake of time in re-running these examples, let's subset the data further to the most abundant 5 phyla.
Before we get to network plots, let's discuss distances. Many tools use distances to perform their calculations. In phyloseq, ordinations, heatmaps, and network plots all use the distance function for calculating OTU or Sample distance matrices (actually represented as a "dist" object) when needed, particularly when PCoA/MDS or NMDS is involved.
help("distance")
`?`(distance)
A relatively recent, popular distance method that relies heavily on the phylogenetic tree is UniFrac. It has been implemented in phyloseq as a fast parallel function, also wrapped by the distance function. See the phyloseq wiki page describing Fast Parallel UniFrac for further details, citations, and performance results.
The following code illustrates using the make_network and plot_network functions in phyloseq. In this context, we are using networks to graphically represent thresholded distances between samples or OTUs. The euclidean distances between points on the plot are essentially arbitrary, only the "edges" (lines) between "nodes" (OTUs/samples) are derived directly from the data. For further examples, it is recommended that you take a look at the plot_network wiki page
The GP variable is an only-slightly-modified version of the GlobalPatterns dataset. The threshold was determined empirically to show something interesting for demonstration. In practice, this value has a huge effect on the resulting network, and its usefulness, and it is highly recommended that you investigate the results from multiple values.
ig <- make_network(GP, type ="samples", distance ="bray", max.dist =0.85)
plot_network(ig, GP, color ="SampleType", shape ="human", line_weight =0.4,
label =NULL)
An example showing a network representation of OTUs, representing communities of bacteria that occurr in similar profiles of samples.
data(GlobalPatterns)
# prune to just the top 100 most abundant OTUs across all samples (crude).
GP100 <- prune_taxa(names(sort(taxa_sums(GlobalPatterns), TRUE))[1:100], GlobalPatterns)
jg <- make_network(GP100, "taxa", "jaccard", 0.3)
plot_network(jg, GP100, "taxa", color ="Phylum", line_weight =0.4, label =NULL)
plot of chunk plotnetwork-otus
Ordination Methods
"Ordination methods" in this context refers to methods for dimensional reduction of data -- usually the OTU abundance data, which is probably a large sparse matrix not so amenable to graphical display on its own. Graphically investigating the (usually) information-dense first few axes of an ordination result can be very useful for exploring and summarizing phylogenetic sequencing data. One of the resons for this is that many ordination methods are non-parametric, so they do not depend upon a prior hypothesis or model. This is essential for many microbiome investigations in which a model is only vaguely described or not available.
A good quick summary of ordination methods is provided in the introductory vignette for the vegan package:
This function wraps several commonly-used ordination methods for OTU abundance tables (as well as related tables, in some cases). The type of ordination performed depends upon the argument to method. Try ordinate("help") or ordinate("list") for the currently supported method options.
The output of this function will be an ordination class. The specific class depends upon the ordination method used, as well as the underlying function/package that is called internally by phyloseq to perform it. As a general rule, any of the ordination classes returned by this function, ordinate, will be recognized by downstream tools in the phyloseq package -- for example the ordination plotting function, plot_ordination (See next section for plot examples).
The plot_ordination function has many options, and supports many combinations of ordinations, including the mapping of sample and/or OTU variables to color and shape aesthetics. Many additional examples (with results) are included on the plot_ordination wiki page. For quicker reference, some example "1-liners" are also included at bottom of this section.
This combination of MDS/PCoA ordination of the UniFrac distance is recently very popular in microbiome analyses.
require("ggplot2")
ptitle <- "GP PCoA of UniFrac distance, GP most abundant 100 OTUs only"
p <- plot_ordination(GP100, GP.MDS, type ="samples", color ="SampleType",
title = ptitle)
p + geom_point(size =5) + geom_polygon(aes(fill = SampleType))
plot of chunk GP100-UniFrac-PCoA
Get the names of the most-abundant phyla, and use for subsetting.
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="taxa", color ="Phylum") +
geom_point(size =4)
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="split")
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="split", color ="SampleType")
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="biplot", shape ="Phylum")
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="split", color ="Phylum",
label ="SampleType")
plot_ordination(GP2, ordinate(GP2, "CCA"), type ="split", color ="SampleType",
shape ="Phylum", label ="SampleType")
Mapping Continuous Variables to Color
(Note: can't map continuous variables to shape with plot_ordination function)
p4title <- "Bushman dataset, PCoA/MDS ordination on Bray-Curtis distance"
Bushman.ord <- ordinate(Bushman, method ="MDS", distance ="bray")
plot_ordination(Bushman, Bushman.ord, "samples", color ="OMEGA3_FATTY_ACIDS_G_AVE",
title = p4title)
In a 2010 article in BMC Genomics, Rajaram and Oono describe an approach to creating a heatmap using ordination methods (namely, NMDS and PCA) to organize the rows and columns instead of (hierarchical) cluster analysis. In many cases the ordination-based ordering does a much better job than h-clustering at providing an order of elements that is easily interpretable. The authors provided an immediately useful example of their approach as the NeatMap package for R. The NeatMap package can be used directly on the abundance table ("otu_table"-class) of phylogenetic-sequencing data, but the NMDS or PCA ordination options that it supports are not based on ecological distances. To fill this void, and because phyloseq already provides support for a large number of ecological distances and ordination methods, phyloseq now includes the plot_heatmap() function: an ecology-oriented variant of the NeatMap approach to organizing a heatmap and build it using ggplot graphics tools. The distance and method arguments are the same as for the plot_ordination function, and support large number of distances and ordination methods, respectively, with a strong leaning toward ecology. This function also provides the options to re-label the OTU and sample axis-ticks with a taxonomic name and/or sample variable, respectively, in the hope that this might hasten your interpretation of the patterns (See the documentation for the sample.label and species.label arguments, and the examples below). Note that this function makes no attempt to overlay dendrograms from hierarchical clustering next to the axes, as hierarchical clustering is not used to organize these plots. Also note that each re-ordered axis repeats at the edge, and so apparent clusters at the far right/left or top/bottom of the heat-map may actually be the same. For now, the placement of this edge can be considered arbitrary, so beware of this artifact of the graphic and visually check if there are two "mergeable" clusters at the edges of a particular axis. If you benefit from this phyloseq-specific implementation of the NeatMap approach, please cite the NeatMap article, as well as phyloseq.
In this section, we will look at examples for using R to validate/test hypotheses we may have generated through some of the previous exploration.
Multiple Testing
In this example we will perform testing on fractional abundances to remove effect of differences in total sequencing across samples for same taxa. We first filter the low-variance taxa, avoiding the noise and "penalty" we pay for testing taxa that don't vary at much across samples. In practice, these OTU abundances probably need additional preprocessing prior to testing, and many good methods in microarray analysis and differential expression sequencing could probably apply to this data (but are not directly implemented/supported in phyloseq, yet). Otherwise, beware that the old motto "garbage-in, garbage-out" can definitely apply to your data if you are not careful.
We are going to use the multtest wrapper included in phyloseq, mt. Try ?mt for help on this function. The following code uses this wrapper to calculate the multiple-inference-adjusted P-values, using the "human" sample variable.
Re-order to original, and use raw p-values for adjustment via mt.rawp2adjp()
procedure <- c("Bonferroni", "Hochberg", "BH")
p.mtm <- mt.rawp2adjp(mtm[order(mtm[, "index"]), "rawp"], procedure)
# Re-order so that you can return original table (ordered p-value table)
p.adjp.ord <- p.mtm$adjp[order(p.mtm$index), ]
# Give it the original row names from mrownames(p.adjp.ord) <- taxa_names(GP3f)
# Return the table of adjusted p-values for each hypothesis.
GP3f.mt.table <- data.frame(p.adjp.ord, tax_table(GP3f)[rownames(p.adjp.ord),
jranks])
# Re-rorder based on BH
GP3f.mt.table <- GP3f.mt.table[order(GP3f.mt.table[, "BH"]), ]
subset(GP3f.mt.table, BH < 0.05)
A common question from many users related to how they can easily get phyloseq-formatted data into other R tools. The following examples are meant to illustrate doing that with some commonly-requested tasks.
The vegan package is a popular and well-maintained R package (hosted in CRAN) "for community and vegetation ecologists". It provides ordination methods, diversity analysis and other functions. Many of vegan's distance and ordination functions are wrapped by functions in phyloseq. Of course, not everything in vegan is wrapped by phyloseq, and it may turn out that you need to use some function that is not wrapped by phyloseq. What do you do? The following subsection provides example code for running just such a function by accessing and coercing the necessary data components from a phyloseq data object.
For OTU abundance tables, vegan expects samples as rows, and OTUs/species/taxa as columns (so does the picante package). The following is an example function, called veganotu, for extracting the OTU table from a phyloseq data object, and converting it to a properly oriented standard matrix recognized by the vegan package.
We will use the bioenv function from vegan to test for sample variables that correlate well with the microbial community distances.
veganotu <- function(physeq) {
require("vegan")
OTU <- otu_table(physeq)
if (taxa_are_rows(OTU)) {
OTU <- t(OTU)
}
return(as(OTU, "matrix"))
}
Now we can use this function for data input to vegan functions. Let's try this with the Bushman dataset, since it has ample continuous variables with which to correlate microbiom distances. First, let's coerce the Bushman sample data component into a vanilla "data.frame" class that we will call bushsd.
Now let's make a call to bioenv to see what happens... (not actually run in example, takes too long, be prepared to stop the run if you try it)
# bioenv(veganotu(Bushman), bushsd)
There are so many sample variables in the Bushman data that we would have to calculate 1.225996e+55 possible subsets in an exhaustive search calculation. That could take a long time! Note from the bioenv documentation that we could have expected that issue:
"There are 2^p-1 subsets of p variables, and an exhaustive search may take a very, very, very long time (parameter upto offers a partial relief)."
So one option is to use the upto parameter. Another is to specify a model formula when specifying the primary data argument to this function (see ?"~" for some details), which allows us to specify variables in the correlation search. We could also simply trim the number of columns in bushsd to just the few variables that we really care about. Let's try the second option here, the model formula approach, and focus on a few that we are interested in comparing. We'll look at the variable names again to remind us what is available
I arbitrarily chose a mixture of variables from the full list previewed above.
bioenv(veganotu(Bushman) ~ DEPTH + AGE + TOTAL_FAT_G_AVE + INSOLUBLE_DIETARY_FIBER_G_AVE,
bushsd)
##
## Call:
## bioenv(formula = veganotu(Bushman) ~ DEPTH + AGE + TOTAL_FAT_G_AVE + INSOLUBLE_DIETARY_FIBER_G_AVE, data = bushsd)
##
## Subset of environmental variables with best correlation to community data.
##
## Correlations: spearman
## Dissimilarities: bray
##
## Best model has 2 parameters (max. 4 allowed):
## AGE INSOLUBLE_DIETARY_FIBER_G_AVE
## with correlation 0.1537
##
Now that you know how to get phyloseq data into vegan, all of vegan's tools are now available to you to use as well. The following list is not exhaustive and focuses on the most popular tools, taken from the vegan website front page:
Diversity analysis: Shannon, Simpson, Fisher indices, Rényi diversities and Hill numbers.
Species abundance models: Fisher and Preston models, species abundance distributions.
Analysis of species richness: species accumulation curves, extrapolated richness.
Ordination: support and meta functions for NMDS, redundancy analysis, constrained correspondence analysis, constrained analysis of proximities (all three with partial analysis),
Support functions for ordination: dissimilarity indices, extended dissimilarities, Procrustes analysis, ordination diagnostics, permutation tests.
Ordination and environment: vector fitting, centroid fitting and smooth surface fitting, adding species scores as weighted averages, adding convex hull, SD ellipses, arrows etc. to ordination.
Dissimilarity analyses: ANOVA using dissimilarities, ANOSIM, MRPP, BIOENV, Mantel and partial Mantel tests.
Data standardization: Hellinger, Wisconsin, Chi-square, Beals smoothing.
Ordination Example on the Gap Statistic
Gap Statistic: How many clusters are there?
From the clusGap documentation: The clusGap function from the cluster package calculates a goodness of clustering measure, called the “gap” statistic. For each number of clusters k, it compares (W(k)) with E^*[(W(k))] where the latter is defined via bootstrapping, i.e. simulating from a reference distribution.
The following is an example performing the gap statistic on ordination results calculated using phyloseq tools, followed by an example of how a ggplot-based wrapper for this example might be included in the phyloseq package.
First perform the ordination using correspondence analysis
That's nice. Just in case it is useful, let's look at what the wrapper-function might look like.
gap_statistic_ordination <- function(ord, FUNcluster, type ="sites", K.max =6,
axes =c(1:2), B =500, verbose =interactive(), ...) {
require("cluster")
# If 'pam1' was chosen, use this internally defined call to pam
if (FUNcluster == "pam1") {
FUNcluster <- function(x, k) list(cluster =pam(x, k, cluster.only =TRUE))
}
# Use the scores function to get the ordination coordinates
x <- scores(ord, display = type)
# If axes not explicitly defined (NULL), then use all of them
if (is.null(axes)) {
axes <- 1:ncol(x)
}
# Finally, perform, and return, the gap statistic calculation using# cluster::clusGapclusGap(x[, axes], FUN = FUNcluster, K.max = K.max, B = B, verbose = verbose,
...)
}
# Define a plot method for results...
plot_clusgap <- function(clusgap, title ="Gap Statistic calculation results") {
require("ggplot2")
gstab <- data.frame(gs$Tab, k =1:nrow(gs$Tab))
p <- ggplot(gstab, aes(k, gap)) + geom_line() + geom_point(size =5)
p <- p + geom_errorbar(aes(ymax = gap + SE.sim, ymin = gap - SE.sim))
p <- p + opts(title = title)
return(p)
}
Now try out this function. Should work on ordination classes recognized by scores function, and provide a ggplot graphic instead of a base graphic. (Special Note: the phyloseq-defined scores extensions are not exported as regular functions to avoid conflict, so phyloseq-defined scores extensions can only be accessed with the phyloseq::: namespace prefix in front.)
The firehose of new-gen sequencing data is making possible "big" datasets in this realm. We are considering some of the best approaches to help a tool like phyloseq address computational issues that are arising from dealing with this data, without compromising some of the other features (interactivity, reproducibility, connection with existing R tools). Some promising tools already available for R that might help include:
Sparse matrix classes
Like in the Matrix package. This mainly applies to in-ram computations, especially when the full matrix is actually needed. In principle, this might only apply be required to represent the preprocessed data, which could still be sparse.
Store full dataset in a database
Some suggested packages at this conference so far, obviousl not-yet implemented in phyloseq, are:
