the cedar ledge

Apache Arrow

Date: October 26 2020

Summary: Language-agnostic in-memory columnar format for analytical query engines and data frames

Keywords: ##zettel #apache #arrow #dataframe #data #storage #format #archive

Bibliography

Not Available

Table of Contents

  1. How To Cite
  2. References
  3. Discussion:

[1]

The arrow format allows for larger-than-memory datasets. When writing a dataset to arrow format, the data and metadata is laid in a descriptive layout. The data is written in pre-determined, binary formats by supported type. When reading, Arrow memory maps data from arrow memory. This means the OS gives access to memory which is swapped into RAM upon requests. (Jacob Quinn – correspondence on Apache Arrow mailing list)

How To Cite

Zelko, Jacob. Apache Arrow. https://jacobzelko.com/10262020041544-apache-arrow. October 26 2020.

References

[1] Apache Arrow and the Future of Data Frames, (2020).Available: https://www.youtube.com/watch?v=fyj4FyH3XdU&t=281s

Discussion:

CC BY-SA 4.0 Jacob Zelko. Last modified: May 19, 2024. Website built with Franklin.jl and the Julia programming language.