Introduction

The Introduction to DuckDBGRanges vignette shows how a DuckDBGRanges behaves like an ordinary GRanges while keeping its data on disk. This vignette asks the follow-up question: what does that cost, and what does it buy? We compare the DuckDB backend against an in-memory GenomicRanges object on two range-analysis scenarios at scale:

1. scATAC-seq peaks — a million peak-like intervals a few hundred bases wide, the shape of multi-sample consensus peak sets;
2. variants — ten million SNV-like point positions, the shape of a large variant catalogue.

The headline results below were produced offline (see Benchmark setup) and are rendered here from a bundled results file, so this vignette builds quickly. The A small, live comparison section runs a miniature version at build time so you can see the mechanics end to end.

The short version: DuckDBGRanges wins decisively on memory and on the filtering that dominates the early stages of a workflow, while in-memory GRanges keeps its edge on the interval-tree operations — findOverlaps(), subsetByOverlaps(), nearest(). The two are complementary, which is why the idiomatic pattern is to filter with DuckDB, then materialize to GRanges for overlap work.

What DuckDBGRanges optimizes

DuckDBGRanges turns coordinate and metadata filters into SQL that DuckDB pushes into the Parquet reader — so a region restriction reads only the matching row groups instead of scanning every range. Existing Bioconductor code picks this up automatically: calling restrict() on a DuckDBGRanges is a WHERE clause, and chained filters compose into a single lazy query.

The design vignette (Design and extension of DuckDBGRanges) shows the SQL each operation translates to.

A small, live comparison

To show the mechanics without a large dataset, we build a small range set and run one operation on both an in-memory GRanges and a DuckDBGRanges, confirming the results agree.

library(DuckDBGRanges)
library(GenomicRanges)
library(arrow)

set.seed(1L)
n <- 2000L
df <- data.frame(
    seqnames = sample(paste0("chr", 1:5), n, replace = TRUE),
    start = sample(1:1e6, n),
    strand = sample(c("+", "-", "*"), n, replace = TRUE),
    score = runif(n))
df$end <- df$start + sample(100:500, n, replace = TRUE)

gr <- makeGRangesFromDataFrame(df, keep.extra.columns = TRUE)
path <- tempfile(fileext = ".parquet"); write_parquet(df, path)
ddb <- DuckDBGRanges(path, seqnames = "seqnames", start = "start",
                     end = "end", strand = "strand", mcols = "score")

## same answer, one in memory and one queried from disk
r_gr  <- restrict(gr,  start = 1L, end = 500000L)
r_ddb <- restrict(ddb, start = 1L, end = 500000L)
length(r_gr) == length(r_ddb)
#> [1] TRUE

At this size the in-memory object is faster — there is nothing to gain from going to disk, and the query has fixed overhead. The advantage appears at scale, which is what the offline benchmark measures.

Benchmark setup

The full benchmark generates synthetic ranges at scale across the human autosomes: 1,000,000 peaks for the scATAC-seq scenario and 10,000,000 SNVs for the variant scenario, plus a 50,000-feature annotation set used as the subject of the overlap operations. For each scenario it times six operations on both backends and records the in-memory footprint of each object.

The backends parallelize differently, so we give each its best effort on the same core budget: GRanges runs single-threaded C code (the in-memory baseline), while DuckDBGRanges autotunes DuckDB’s internal threads up to the budget with no configuration. The exact parameters are recorded with the results and shown beneath the tables.

Results

Rendered from the bundled offline results (inst/scripts/benchmark_results.rds); regenerate with inst/scripts/run_vignette_benchmarks.R (see that script’s header).

Memory footprint

scATAC-seq scenario (1M peaks)

Variant scenario (10M variants)

Configuration: scATAC-seq scenario = 1e+06 peaks, variant scenario = 1e+07 variants, 16-core budget. GRanges: in-memory, single-threaded. DuckDBGRanges: on-disk Parquet; DuckDB autotunes threads up to the core budget.

Main takeaways

• Memory is the headline. A DuckDBGRanges is a small, constant-size handle (~175 KB) whichever scenario it fronts, while a GRanges grows with the number of ranges — 150x smaller for a million peaks, over 1,500x smaller for ten million variants. This is what lets you reference a hundred-million-range catalogue without paying for it until a query touches it.
• Filtering favors DuckDB. restrict() and logical subsetting become WHERE clauses pushed into the Parquet reader, so they read only matching rows — the advantage grows with data size (a few times faster at a million peaks, ~13x at ten million variants).
• Nearest-feature distance favors DuckDB, strongly. distanceToNearest() runs as an SQL aggregation rather than an interval-tree walk, and is the largest single win measured (~10x at a million peaks, ~70x at ten million variants).
• Inter-range transforms are a wash that tips to DuckDB at scale. reduce() and disjoin() are close; in-memory GRanges edges reduce() at a million peaks, but DuckDB pulls ahead on both by ten million ranges (disjoin() ~2x).
• Overlap enumeration favors GRanges, decisively. findOverlaps() and subsetByOverlaps() are backed by in-memory interval trees that a set-based SQL join does not beat (GRanges is ~100x faster here). Hand these to a materialized GRanges.
• The two compose. The filter, then materialize pattern uses DuckDB to cut a large set down cheaply, then hands the small result to GRanges for the interval-tree step — each backend where it is strongest.

Backend comparison

Running your own benchmarks

inst/scripts/run_vignette_benchmarks.R reproduces these numbers on your hardware and scales to other range counts (BENCH_SCATAC_PEAKS, BENCH_VARIANTS, BENCH_CORES; set small counts to smoke-test first). It writes benchmark_results.rds, which this vignette renders via inst/scripts/make_timings_table.R.

For higher-level genomic workflows built on this backend, see the BiocDuckDB package.

Session information

sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 26.04 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so;  LAPACK version 3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats4    stats     graphics  grDevices utils     datasets  methods  
#> [8] base     
#> 
#> other attached packages:
#>  [1] arrow_25.0.0           DuckDBGRanges_0.99.1   GenomicRanges_1.65.1  
#>  [4] Seqinfo_1.3.0          DuckDBDataFrame_0.99.6 IRanges_2.47.2        
#>  [7] S4Vectors_0.51.5       BiocGenerics_0.59.10   generics_0.1.4        
#> [10] bit64_4.8.2            BiocStyle_2.41.0      
#> 
#> loaded via a namespace (and not attached):
#>  [1] sass_0.4.10           SparseArray_1.13.2    lattice_0.22-9       
#>  [4] digest_0.6.39         magrittr_2.0.5        evaluate_1.0.5       
#>  [7] grid_4.6.1            blob_1.3.0            fastmap_1.2.0        
#> [10] jsonlite_2.0.0        Matrix_1.7-5          DBI_1.3.0            
#> [13] BiocManager_1.30.27   purrr_1.2.2           jquerylib_0.1.4      
#> [16] duckdb_1.5.4.3        abind_1.4-8           cli_3.6.6            
#> [19] rlang_1.3.0           dbplyr_2.6.0          XVector_0.53.0       
#> [22] withr_3.0.3           cachem_1.1.0          DelayedArray_0.39.3  
#> [25] yaml_2.3.12           otel_0.2.0            S4Arrays_1.13.0      
#> [28] tools_4.6.1           dplyr_1.2.1           assertthat_0.2.1     
#> [31] buildtools_1.0.0      vctrs_0.7.3           R6_2.6.1             
#> [34] matrixStats_1.5.0     lifecycle_1.0.5       bit_4.6.0            
#> [37] pkgconfig_2.0.3       bslib_0.11.0          pillar_1.11.1        
#> [40] glue_1.8.1            xfun_0.60             tibble_3.3.1         
#> [43] tidyselect_1.2.1      sys_3.4.3             MatrixGenerics_1.25.0
#> [46] knitr_1.51            htmltools_0.5.9       rmarkdown_2.31       
#> [49] maketools_1.3.2       compiler_4.6.1