dask-sql can be installed via
conda (preferred) or
pip - or in a development environment.
You can continue with the Quickstart after the installation.
Create a new conda environment or use your already present environment:
conda create -n dask-sql conda activate dask-sql
Install the package from the
conda install dask-sql -c conda-forge
GPU support is currently tied to the RAPIDS libraries.
It generally requires the latest cuDF/Dask-cuDF nightlies.
Create a new conda environment or use an existing one to install RAPIDS with the chosen methods and packages. More details can be found on the RAPIDS Getting Started page, but as an example:
conda create --name rapids-env -c rapidsai-nightly -c nvidia -c conda-forge \ cudf=22.10 dask-cudf=22.10 ucx-py ucx-proc=*=gpu python=3.9 cudatoolkit=11.5 conda activate rapids-env
Note that using UCX is mainly necessary if you have an Infiniband or NVLink enabled system. Refer to the UCX-Py docs for more information.
Install the stable package from the
conda install -c conda-forge dask-sql
Or the latest nightly from the
dask channel (currently only available for Linux-based operating systems):
conda install -c dask/label/dev dask-sql
pip install dask-sql
If you want to have the newest (unreleased)
dask-sql version or if you plan to do development on
dask-sql, you can also install the package from sources.
git clone https://github.com/dask-contrib/dask-sql.git
Create a new conda environment and install the development environment:
conda env create -f continuous_integration/environment-3.9-dev.yaml
It is not recommended to use
pip instead of
After that, you can install the package in development mode
pip install -e ".[dev]"
To compile the Rust code (after changes), run
python setup.py build_ext
You can run the tests (after installation) with
GPU-specific tests require additional dependencies specified in continuous_integration/gpuci/environment.yaml:
conda env create -n dask-sql-gpuci -f continuous_integration/gpuci/environment.yaml
GPU-specific tests can be run with
pytest tests -m gpu --rungpu
This repository uses pre-commit hooks. To install them, call