cubed0.23.0
cubed0.23.0
Published
Scalable array processing with bounded memory
pip install cubed
Package Downloads
Authors
Project URLs
Requires Python
>=3.11
Dependencies
- aiostream
- array-api-compat
- cloudpickle
- donfig
- mypy_extensions
- ndindex
- networkx
!=2.8.3,!=2.8.4,!=2.8.5,!=2.8.6,!=2.8.7,!=2.8.8,!=3.0.*,!=3.1.*,!=3.2.* - numpy
>=1.22 - psutil
- tenacity
- toolz
- zarr
!=3.1.2 - tqdm
; extra == "diagnostics" - graphviz
; extra == "diagnostics" - jinja2
; extra == "diagnostics" - pydot
; extra == "diagnostics" - pandas
; extra == "diagnostics" - matplotlib
; extra == "diagnostics" - rich
; extra == "diagnostics" - seaborn
; extra == "diagnostics" - apache-beam
; extra == "beam" - gcsfs
; extra == "beam" - dask
<2024.12.0; extra == "dask" - distributed
<2024.12.0; extra == "dask-distributed" - icechunk
; extra == "icechunk" - lithops
[aws]>=3.3.0; extra == "lithops" - cubed
[diagnostics]; extra == "lithops-aws" - lithops
[aws]>=3.3.0; extra == "lithops-aws" - s3fs
; extra == "lithops-aws" - cubed
[diagnostics]; extra == "lithops-gcp" - lithops
[gcp]>=3.3.0; extra == "lithops-gcp" - gcsfs
; extra == "lithops-gcp" - cubed
[diagnostics]; extra == "modal" - modal
; extra == "modal" - s3fs
; extra == "modal" - cubed
[diagnostics]; extra == "modal-gcp" - modal
; extra == "modal-gcp" - gcsfs
; extra == "modal-gcp" - cubed
[diagnostics]; extra == "coiled" - coiled
; extra == "coiled" - dask
[complete]; extra == "coiled" - s3fs
; extra == "coiled" - cubed
[diagnostics]; extra == "test" - dill
; extra == "test" - numpy_groupies
; extra == "test" - obstore
; extra == "test" - pytest
; extra == "test" - pytest-cov
; extra == "test" - pytest-mock
; extra == "test"
Scalable array processing with bounded memory
Cubed is a Python library for scalable out-of-core multi-dimensional array processing with bounded memory.
- Cubed provides NumPy and Xarray APIs for processing your multi-dimensional array data
- Cubed is a drop-in replacement for Dask's Array API
- Cubed will tell you if your computation would run out of memory before running it
- Cubed is designed to be robust to failures and will reliably complete a computation
- Cubed can process hundreds of GB of array data on your laptop using all available cores
- Cubed is horizontally scalable and stateless, and can scale to multi-TB datasets in the cloud