---
title: "Introduction to remart"
date: "`r BiocStyle::doc_date()`"
package: "`r BiocStyle::pkg_ver('remart')`"
vignette: >
  %\VignetteIndexEntry{"Introduction to remart"}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
output:
  BiocStyle::html_document
---

```{r}
#| label: setup
#| include: false

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

# Copied from testthat:::has_internet()
ensembl_offline <- !is.null(curl::nslookup(
  "rest.ensembl.org",
  error = FALSE
))
```

# Overview

The `r BiocStyle::Biocpkg("remart")` package provides a drop-in replacement for
`r BiocStyle::Biocpkg("biomaRt")` workflows. It keeps familiar function names
and signatures while replacing the BioMart backend with the Ensembl REST API.

`r BiocStyle::Biocpkg("remart")` is not (yet?) a complete implementation of the
BioMart query language. It currently supports a focused set of gene and
transcript annotation queries, sequence retrieval, and orthologue queries.
Interaction with non-Ensembl Marts should still be done with
`r BiocStyle::Biocpkg("biomaRt")`.

`r BiocStyle::Biocpkg("remart")` should not only be more stable, faster, but
also hopefully more user-friendly than `r BiocStyle::Biocpkg("biomaRt")`, as
many arguments are no longer necessary. These arguments are silently accepted
for compatibility, but they are ignored.

All annotation and sequence examples in this vignette query live Ensembl
services. They require an internet connection, and results can change as Ensembl
releases and annotations are updated.

# Installation and loading

The development version can be installed from GitHub with:

```{r}
#| label: install
#| eval: false

pak::pak("Huber-group-EMBL/remart")
```

Load the package in an R session with:

```{r}
#| label: load-package

library(remart)
```

# Discover supported attributes

`listAttributes()` reports the attributes currently implemented by
`r BiocStyle::Biocpkg("remart")`:

```{r}
#| label: list-attributes

listAttributes()
```

The supported filters for `getBM()` are currently limited to `ensembl_gene_id`
and `ensembl_transcript_id`, and only one filter can be used per call.

# Retrieve gene annotations

`getBM()` accepts the familiar `attributes`, `filters`, and `values` arguments.
The following query retrieves basic annotations for two human Ensembl genes. The
connection to Ensembl is handled internally, so a `mart` object is not required.

```{r}
#| label: gene-annotations
#| eval: !expr '!ensembl_offline'

gene_ids <- c("ENSG00000157764", "ENSG00000004939")

gene_annotations <- getBM(
  attributes = c(
    "ensembl_gene_id",
    "external_gene_name",
    "chromosome_name",
    "start_position",
    "end_position"
  ),
  filters = "ensembl_gene_id",
  values = gene_ids
)

head(gene_annotations)
```

Attributes are returned in the order requested. Duplicate identifiers are
handled internally, while identifiers that Ensembl cannot find are omitted with
a warning.

# Retrieve transcript-level annotations

Transcript-level attributes can be requested using gene identifiers. Because a
gene can have multiple transcripts, this may return multiple rows for one gene.

```{r}
#| label: transcript-annotations
#| eval: !expr '!ensembl_offline'

transcript_annotations <- getBM(
  attributes = c(
    "ensembl_gene_id",
    "ensembl_transcript_id",
    "ensembl_peptide_id",
    "transcript_biotype"
  ),
  filters = "ensembl_gene_id",
  values = "ENSG00000157764"
)

head(transcript_annotations)
```

Alternatively, transcript identifiers can be used as the filter when the query
is transcript-oriented:

```{r}
#| label: transcript-filter
#| eval: !expr '!ensembl_offline'

getBM(
  attributes = c(
    "ensembl_transcript_id",
    "ensembl_gene_id",
    "ensembl_peptide_id"
  ),
  filters = "ensembl_transcript_id",
  values = "ENST00000357654"
)
```

# Use the `getGene()` compatibility wrapper

For workflows written around `biomaRt::getGene()`, `getGene()` provides a
convenient gene-level interface:

```{r}
#| label: get-gene
#| eval: !expr '!ensembl_offline'

getGene(
  id = c("ENSG00000157764", "ENSG00000004939"),
  type = "ensembl_gene_id"
)
```

The returned columns include `ensembl_gene_id`, `hgnc_symbol`, genomic
coordinates, strand, and description. Ensembl's REST response does not provide
the `band` field used by `r BiocStyle::Biocpkg("biomaRt")`, so
`r BiocStyle::Biocpkg("remart")` returns `band` as `NA` with a warning.

# Retrieve sequences

`getSequence()` retrieves sequences by Ensembl gene, transcript, or peptide
identifier.

For example, a cDNA sequence can be retrieved for a transcript as follows:

```{r}
#| label: cdna-sequence
#| eval: !expr '!ensembl_offline'

cdna <- getSequence(
  seqType = "cdna",
  type = "ensembl_transcript_id",
  id = "ENST00000357654"
)

summary(cdna)
```

Upstream and downstream bases can be requested for supported sequence queries:

```{r}
#| label: expanded-sequence
#| eval: !expr '!ensembl_offline'

extended_cdna <- getSequence(
  seqType = "cdna",
  type = "ensembl_transcript_id",
  id = "ENST00000357654",
  upstream = 100,
  downstream = 100
)

summary(extended_cdna)
```

Coordinate-based requests using `chromosome`, `start`, and `end` are not yet
implemented.

# Find orthologues

`getLDS()` retains the familiar `r BiocStyle::Biocpkg("biomaRt")` name, but its
current implementation is specifically for orthology queries through Ensembl's
homology endpoint. The target species is supplied with `speciesL`; this replaces
the linked `martL` connection used by `r BiocStyle::Biocpkg("biomaRt")`.

The following example searches for mouse orthologues of a human gene:

```{r}
#| label: orthologues
#| eval: !expr '!ensembl_offline'

orthologues <- getLDS(
  attributes = c("ensembl_gene_id", "external_gene_name"),
  filters = "ensembl_gene_id",
  values = "ENSG00000157764",
  attributesL = c("ensembl_gene_id", "external_gene_name"),
  speciesL = "mouse"
)

orthologues
```

Only `ensembl_gene_id` is currently supported as the source filter. Source and
target attributes are gene-level attributes; transcript-level linked queries are
not yet supported.

# Migrating common `r BiocStyle::Biocpkg("biomaRt")` code

A typical `r BiocStyle::Biocpkg("biomaRt")` workflow creates a connection before
calling `getBM()`:

```{r}
#| label: biomart-workflow
#| eval: false

# A biomaRt workflow.
# This example is not actually run as the biomart service is unstable.
gene_ids <- c("ENSG00000157764", "ENSG00000004939")
attrs <- c(
  "ensembl_gene_id",
  "external_gene_name",
  "chromosome_name",
  "start_position",
  "end_position"
)
mart <- biomaRt::useEnsembl(
  "ensembl",
  dataset = "hsapiens_gene_ensembl",
  version = 114
)
biomaRt::getBM(
  attributes = attrs,
  filters = "ensembl_gene_id",
  values = gene_ids,
  mart = mart
)
```

With `r BiocStyle::Biocpkg("remart")`, the connection setup is removed and the
query can usually be adapted directly:

```{r}
#| label: remart-workflow
#| eval: !expr '!ensembl_offline'

gene_ids <- c("ENSG00000157764", "ENSG00000004939")
attrs <- c(
  "ensembl_gene_id",
  "external_gene_name",
  "chromosome_name",
  "start_position",
  "end_position"
)
getBM(
  attributes = attrs,
  filters = "ensembl_gene_id",
  values = gene_ids,
  version = 114
)
```

Several obsolete connection-related arguments are accepted through `...` for
source compatibility. Check `listAttributes()` and the function reference before
migrating a workflow that uses less common filters or attributes.

# Roadmap and future direction

We aim to implement additional filters and attributes, but we do not have full
visibility on how `r BiocStyle::Biocpkg("biomaRt")` is currently used in
workflows in production. If you have a specific use case that is not currently
supported, please open an issue on GitHub.

# Session information

```{r}
#| label: session-info

utils::sessionInfo()
```