Package: OmicsLake 0.99.3

Yusuke Matsui

OmicsLake: OmicsLake: Versioned, On-Disk Omics Data Management for Bioconductor

A lightweight framework for versioned, on-disk omics data management using DuckDB, Arrow, and Parquet. It provides read/write-style functions, snapshots, and version-aware dataset lineage for tracked or explicitly annotated workflows. Adapters preserve supported Bioconductor containers, including SummarizedExperiment and MultiAssayExperiment, while retaining queryable tabular components.

Authors:Yusuke Matsui [aut, cre]

OmicsLake_0.99.3.tar.gz
OmicsLake_0.99.3.zip(r-4.7)OmicsLake_0.99.3.zip(r-4.6)OmicsLake_0.99.3.zip(r-4.5)
OmicsLake_0.99.3.tgz(r-4.6-any)OmicsLake_0.99.3.tgz(r-4.5-any)
OmicsLake_0.99.3.tar.gz(r-4.7-any)OmicsLake_0.99.3.tar.gz(r-4.6-any)
OmicsLake_0.99.3.tgz(r-4.6-emscripten)
manual.pdf |manual.html
DESCRIPTION |NEWS
card.svg |card.png
OmicsLake/json (API)

# Install 'OmicsLake' in R:
install.packages('OmicsLake', repos = c('https://biocstaging.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/matsui-lab/omicslake/issues

On CRAN:

Conda:

infrastructuredataimportdatarepresentation

4.08 score 139 exports 45 dependencies

Last updated from:0d2b2fc779. Checks:1 WARNING, 9 OK. Indexed: yes.

TargetResultTimeFilesSyslog
bioc-checksWARNING243
linux-devel-x86_64OK293
source / vignettesOK259
linux-release-x86_64OK304
macos-release-arm64OK259
macos-oldrel-arm64OK232
windows-develOK463
windows-releaseOK271
windows-oldrelOK325
wasm-releaseOK184

Exports:%!between%%!in%%>>%%between%%ilike%%iregex%%like%%regex%ATACAdapterChIPAdapterChromatogramsAdaptercoalescecontains_strcreate_pipelinedepsdropends_with_strEpigenomicsAdapterexport_datafetchfromGenomicsAdapterget_adaptersGlycomicsAdapterhistoryif_else_naimport_dataintois_not_nullis_nulllakeLakelake_auto_offlake_auto_onlake_doctorlake_existslake_findlake_getlake_haslake_putlake_reflake_repairlake_snaplake_statuslake_strict_onlake_taglake_tracklake_track_scriptlake_treelake_useLakeAdapterlinkLipidomicsAdapterMAEAdaptermarkMetabolomicsAdapterMethylationAdapterMsExperimentAdapterobjectsobserveobserve_sessionobserve_to_lakeol_add_rankol_aggregateol_checkoutol_commitol_compare_versionsol_create_viewol_cumulative_sumol_disable_transparent_trackingol_dropol_drop_objectol_drop_viewol_enable_strict_repro_modeol_enable_transparent_trackingol_export_parquetol_freadol_get_dependenciesol_import_parquetol_initol_labelol_list_labelsol_list_object_versionsol_list_objectsol_list_tablesol_list_tagsol_list_viewsol_loadol_logol_log_commitsol_moving_avgol_plot_lineageol_queryol_readol_read_maeol_read_objectol_read_seol_saveol_show_lineageol_tagol_tag_objectol_top_nol_writePhosphoproteomicsAdapterProteomicsAdapterputQFeaturesAdapterqueryQueryBuilderRaggedExperimentAdapterrecord_readrecord_writerefregister_adapterrestoresave_asSCEAdapterSEAdapterSeuratAdaptershow_migration_guidesnapSpatialExperimentAdapterSpectraAdaptersqlstarts_with_strtablestagtrace_callstrack_pipelinetrack_scriptTranscriptomicsAdaptertreeunlink_depuse_lakeVCFAdapterwith_trackingwrap_callwrap_fnXCMSAdapter

Dependencies:abindarrowassertthatBiobaseBiocGenericsbitbit64blobclicpp11DBIdbplyrDelayedArraydplyrduckdbgenericsGenomicRangesglueIRangesjsonlitelatticelifecyclemagrittrMatrixMatrixGenericsmatrixStatspillarpkgconfigpurrrR6rlangS4ArraysS4VectorsSeqinfoSparseArraystringistringrSummarizedExperimenttibbletidyrtidyselectutf8vctrswithrXVector

OmicsLake Comprehensive Guide
Introduction | Before you run all chunks | Main package features | 1. Basic operations | 1.1 Project initialization | 1.2 Writing and reading tables | 1.3 Advanced analysis with SQL queries (ol_query) | JOINs and subqueries | Lazy evaluation and integration with dplyr | 1.4 Advanced analysis with aggregation and window functions | Calculating gene expression statistics | Ranking genes | Moving average and cumulative sum | Combining with lazy evaluation | 1.5 Saving and loading R objects | 2. Tracking dependencies | 2.1 Saving data with specified dependencies | 2.2 Checking dependencies | 2.3 Importing and exporting Parquet files | Parquet export | Parquet import | Performance tips | 3. Version management | 3.1 Tagging and labeling | 3.2 Creating multiple versions | 3.3 Listing and comparing versions | 3.4 Loading a specific version | 4. Commits and history management | 4.1 Creating commits | 4.2 Viewing history | 5. Visualizing dependencies | 5.1 Creating dependency graphs | 6. Restoring a project | 6.1 Restoring state using labels | 7. Advanced features | 7.1 Filtered data loading | 7.2 Loading with lazy evaluation | 7.3 Deleting tables | 8. Bioconductor integration | 8.1 Creating a SummarizedExperiment | 8.2 Creating a MultiAssayExperiment | 9. Practical workflow example | 9.1 Complete differential expression analysis workflow | 10. Conclusion | List of key functions | Project management | Data saving and loading | Version management | History management | Dependencies | Listing | Bioconductor | Others | Best practices | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake Layer-by-Layer Use Cases
Introduction | Before you run all chunks | Covered layers | Initialization | 1. Bulk RNA-seq (SummarizedExperiment) | 2. Single-cell RNA-seq (SingleCellExperiment) | 3. Multi-omics cohort (MultiAssayExperiment) | 4. Proteomics / Metabolomics raw MS (Spectra) | 5. Proteomics quantification graph (QFeatures) | 6. Integrated LC-MS container (MsExperiment) | 7. Cross-layer release manifest | 8. Validation with Tutorial-Derived Data | 8.1 Official MultiAssayExperiment data (miniACC) | 8.2 Official Spectra data (fft_spectrum) | 9. Automated Effectiveness Checks | Summary | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake Practical Workflow: Integrating Phase 1-4 Functions
Introduction | Before you run all chunks | Project setup | Preparing the data | Phase 1: Filtering data with SQL queries | Phase 2: Statistical analysis with aggregation functions | Calculate statistics for each gene | Extract top expressed genes | Add rankings | Compute cumulative statistics | Phase 3: Sharing results with Parquet | Exporting results | Importing external data | Phase 4: Compare multiple versions with database views | Scenario: Compare two different analysis methods | Create a view: Compare methods | Using the view: Check concordance | Create multiple views | List views | Deleting views | Integrated workflow example: Combine all features | Practical tips and best practices | Use cases for Phase 1 (SQL queries) | Use cases for Phase 2 (aggregation functions) | Use cases for Phase 3 (Parquet) | Use cases for Phase 4 (views) | Combination patterns | Summary | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake v2.0 Quick Start Guide
Introduction | Before you run all chunks | Recommended Next Guides | Key Features | Installation | Basic Usage | Initializing a Lake | Storing and Reading Data | Filtering with Formula Syntax | dplyr Integration (Automatic Lineage Tracking) | Joining Multiple Tables | QueryBuilder | Version Control | Snapshots and Tags | Viewing History | Lineage (Data Provenance) | Checking Dependencies | Lineage Visualization | Lightweight Mode (Integrating with Existing Code) | observe: Track without Code Changes | wrap: Wrap Functions for Tracking | Defining Pipelines | Custom Operators | Bracket Notation | Import/Export | Bioconductor Integration | Global Shortcuts | Migration from Legacy API | Summary | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake v2.0 クイックスタートガイド
はじめに | 全コードを実行する前に | 次に読むガイド | 主な特徴 | インストール | 基本的な使い方 | Lakeの初期化 | データの保存と読み込み | Formula構文でフィルタリング | dplyr統合(自動リネージ追跡) | 複数テーブルのJOIN | QueryBuilder | バージョン管理 | スナップショットとタグ | 履歴の確認 | リネージ(データ系譜) | 依存関係の確認 | リネージの可視化 | 軽量モード(既存コードへの導入) | observe: コード変更なしで追跡 | wrap: 関数をラップして追跡 | パイプラインの定義 | カスタム演算子 | ブラケット記法 | インポート/エクスポート | Bioconductor統合 | グローバルショートカット | 旧APIからの移行 | まとめ | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake レイヤー別ユースケース集
はじめに | 全コードを実行する前に | 対象レイヤー | 初期化 | 1. Bulk RNA-seq (SummarizedExperiment) | 2. Single-cell RNA-seq (SingleCellExperiment) | 3. Multi-omics cohort (MultiAssayExperiment) | 4. Proteomics / Metabolomics raw MS (Spectra) | 5. Proteomics quantification graph (QFeatures) | 6. LC-MS統合コンテナ (MsExperiment) | 7. クロスレイヤーの公開マニフェスト | 8. チュートリアル由来データでの検証 | 8.1 MultiAssayExperiment公式データ (miniACC) | 8.2 Spectra公式データ (fft_spectrum) | 9. 有効性チェック(自動) | まとめ | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake 実践ワークフロー: フェーズ1-4機能の統合活用
はじめに | 全コードを実行する前に | プロジェクトのセットアップ | データの準備 | Phase 1: SQLクエリによるデータフィルタリング | Phase 2: 集計関数による統計解析 | 遺伝子ごとの統計量を計算 | 発現量上位の遺伝子を抽出 | ランキングを追加 | 累積統計の計算 | Phase 3: Parquetでの結果共有 | 結果のエクスポート | 外部データのインポート | Phase 4: データベースビューで複数バージョンを比較 | シナリオ: 2つの異なる解析手法を比較 | ビューの作成: 手法間の比較 | ビューの活用: 一致度の確認 | 複数のビューを作成 | ビューの一覧表示 | ビューの削除 | 統合ワークフロー例: 全機能を組み合わせる | 実践的なヒントとベストプラクティス | Phase 1 (SQLクエリ) の活用場面 | Phase 2 (集計関数) の活用場面 | Phase 3 (Parquet) の活用場面 | Phase 4 (ビュー) の活用場面 | 組み合わせのパターン | まとめ | Session Information

Last update: 2026-03-17
Started: 2026-03-17

OmicsLake 総合ガイド
はじめに | 全コードを実行する前に | パッケージの主な機能 | 1. 基本操作 | 1.1 プロジェクトの初期化 | 1.2 テーブルの書き込みと読み込み | 1.3 SQLクエリによる高度な分析 (ol_query) | JOINとサブクエリ | 遅延評価とdplyrとの統合 | 1.4 集計・ウィンドウ関数による高度な分析 | 遺伝子発現統計の計算 | 遺伝子のランキング | 移動平均と累積和 | 遅延評価との組み合わせ | 1.5 Rオブジェクトの保存と読み込み | 2. 依存関係の追跡 | 2.1 依存関係を指定してデータを保存 | 2.2 依存関係の確認 | 2.3 Parquet ファイルのインポート・エクスポート | Parquetエクスポート | Parquetインポート | パフォーマンスのヒント | 3. バージョン管理 | 3.1 タグ付けとラベル付け | 3.2 複数バージョンの作成 | 3.3 バージョン一覧と比較 | 3.4 特定バージョンの読み込み | 4. コミットと履歴管理 | 4.1 コミットの作成 | 4.2 履歴の確認 | 5. 依存関係の可視化 | 5.1 依存関係グラフの作成 | 6. プロジェクトの復元 | 6.1 ラベルを使った状態の復元 | 7. 高度な機能 | 7.1 データのフィルタリング読み込み | 7.2 遅延評価での読み込み | 7.3 テーブルの削除 | 8. Bioconductor統合 | 8.1 SummarizedExperimentの作成 | 8.2 MultiAssayExperimentの作成 | 9. 実践的なワークフロー例 | 9.1 完全な差次発現解析ワークフロー | 10. まとめ | 主要な関数一覧 | プロジェクト管理 | データ保存・読み込み | バージョン管理 | 履歴管理 | 依存関係 | 一覧表示 | Bioconductor | その他 | ベストプラクティス | Session Information

Last update: 2026-03-17
Started: 2026-03-17

Readme and manuals

Help Manual

Help pageTopics
OmicsLake: OmicsLake: Versioned, On-Disk Omics Data Management for BioconductorOmicsLake-package OmicsLake
Pipe operator that saves to lake%>>%
BETWEEN operator for range filtering%between%
Case-insensitive LIKE operator%ilike%
Case-insensitive regex match%iregex%
LIKE operator for SQL-style pattern matching%like%
Regex match operator%regex%
ATAC AdapterATACAdapter
ChIP AdapterChIPAdapter
Chromatograms AdapterChromatogramsAdapter
Coalesce - return first non-NA valuecoalesce
Check if string contains a substringcontains_str
Create a pipeline of tracked operationscreate_pipeline
Get dependencies from the default lakedeps
dplyr Compatibility Layerdplyr_compat
Drop data from the default lakedrop
Check if string ends with a suffixends_with_str
Epigenomics AdapterEpigenomicsAdapter
OmicsLake Evaluation Benchmarkseval_bench
OmicsLake Case Study (W3)eval_case_study
OmicsLake Evaluation Data Generatorseval_generate
OmicsLake Evaluation Metricseval_metrics
OmicsLake Evaluation Plottingeval_plot
Export data from the default lakeexport_data
Read data from the default lakefetch
Start a query from a table on the default lakefrom
Genomics AdapterGenomicsAdapter
Get registered adaptersget_adapters
Glycomics AdapterGlycomicsAdapter
Show history from the default lakehistory
If-else with NA handlingif_else_na
Import data into the default lakeimport_data
Create a pipe-compatible lake write functioninto
Check for non-NULL/non-NA valuesis_not_null
Check for NULL/NA valuesis_null
Get the current default lakelake
Lake - OmicsLake Core ClassLake
Simple `lake_*` Aliaseslake_aliases lake_auto_off lake_auto_on lake_get lake_has lake_put lake_ref lake_snap lake_strict_on lake_tag lake_track lake_track_script lake_tree lake_use
Run diagnostics for the default lake (or a specified project)lake_doctor
Check whether a data name exists in the default lakelake_exists
Find data names in the default lakelake_find
Repair workflow for the default lakelake_repair
Show one-line status for the default lake (or a specified project)lake_status
Lake Adapter Base ClassLakeAdapter
Create an explicit dependency linklink
Lipidomics AdapterLipidomicsAdapter
MultiAssayExperiment AdapterMAEAdapter
Mark data in lineage without storingmark
Metabolomics AdapterMetabolomicsAdapter
Methylation AdapterMethylationAdapter
MsExperiment AdapterMsExperimentAdapter
NOT BETWEEN operator%!between% not_between_operator
NOT IN operator%!in% not_in_operator
List objects in the default lakeobjects
Observation Modeobserve
Create an observed sessionobserve_session
Observe and record to a Lakeobserve_to_lake
Add ranking column to tableol_add_rank
Aggregate data with multiple statisticsol_aggregate
Restore entire project to a labeled stateol_checkout
Commit with metadata (note and parameters)ol_commit
Compare versions of an objectol_compare_versions
Create a database viewol_create_view
Calculate cumulative sumol_cumulative_sum
Disable transparent background tracking and optionally commit resultsol_disable_transparent_tracking
Drop (delete) a table from the projectol_drop
Drop (delete) an object from the projectol_drop_object
Drop a database viewol_drop_view
Enable Strict Reproducibility Modeol_enable_strict_repro_mode
Enable transparent background tracking for a sessionol_enable_transparent_tracking
Export a table or object to a Parquet fileol_export_parquet
fread-like reader for Parquet tablesol_fread
Get dependencies for a table or objectol_get_dependencies
Import a Parquet file into the projectol_import_parquet
Initialize an OmicsLake projectol_init
Label current state with a human-friendly aliasol_label
List all project-level labelsol_list_labels
List all versions of an objectol_list_object_versions
List all saved objects in the projectol_list_objects
List all tables in the projectol_list_tables
List all tags for a table or all project labelsol_list_tags
List all views in the projectol_list_views
Alias of 'ol_read()'ol_load
Return version log for a tableol_log
View commit historyol_log_commits
Calculate moving averageol_moving_avg
Visualize dependency lineage as a graphol_plot_lineage
Execute a custom SQL query on the OmicsLake databaseol_query
Read a table by name and referenceol_read
Compose a MultiAssayExperimentol_read_mae
Read a stored objectol_read_object
Build a SummarizedExperiment from long tableol_read_se
Save an R object via the backend metadata tableol_save
Show lineage (full dependency tree) for a table or objectol_show_lineage
Tag a table by creating a backupol_tag
Tag a stored object versionol_tag_object
Get top N rowsol_top_n
Write a table using the DuckDB backendol_write
Custom Query Operatorsoperators
Phosphoproteomics AdapterPhosphoproteomicsAdapter
Print method for lake_observationprint.lake_observation
Print method for lake_repair_reportprint.lake_repair_report
Proteomics AdapterProteomicsAdapter
Write data to the default lakeput
QFeatures AdapterQFeaturesAdapter
Start a query builder on the default lakequery
QueryBuilder - Fluent Query InterfaceQueryBuilder
RaggedExperiment AdapterRaggedExperimentAdapter
Manually record a file read for observationrecord_read
Manually record a file write for observationrecord_write
Get a lazy reference from the default lakeref
Register a data adapterregister_adapter
Restore the default lake to a snapshotrestore
Save pipe result to lakesave_as
SingleCellExperiment AdapterSCEAdapter
SummarizedExperiment AdapterSEAdapter
Seurat AdapterSeuratAdapter
Global Lake Shortcutsshortcuts
Show migration guide for ol_* to Lake APIshow_migration_guide
Create a snapshot of the default lakesnap
SpatialExperiment AdapterSpatialExperimentAdapter
Spectra AdapterSpectraAdapter
Execute SQL on the default lakesql
Check if string starts with a prefixstarts_with_str
List tables in the default laketables
Tag data in the default laketag
Trace function calls for lineagetrace_calls
Track a pipeline block into a Lake with minimal boilerplatetrack_pipeline
Track an existing analysis script with one function calltrack_script
Transcriptomics AdapterTranscriptomicsAdapter
Show lineage tree from the default laketree
Unlink a dependencyunlink_dep
Set or get the default lakeuse_lake
VCF AdapterVCFAdapter
Track a code block with explicit lake integrationwith_tracking
Function Wrapping for Lineage Trackingwrap
Wrap a function call inlinewrap_call
Wrap a function with lineage trackingwrap_fn
XCMS AdapterXCMSAdapter