slurm

A DSL for data-driven computational pipelines

Slurm: A Highly Scalable Workload Manager

Python 3.6+ toolbox for submitting jobs to Slurm

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.

Python Interface to Slurm

Create clusters of VMs on the cloud and configure them with Ansible.

Essential guides and programming tools in my toolbox (with focus on ML Training)

Simplify HPC and Batch workloads on Azure

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

A Slurm cluster using docker-compose

Prometheus exporter for performance metrics from Slurm.

Tools for computation on batch systems

An open-source toolkit for deploying and managing high performance clusters for HPC, AI, and data analytics workloads.

A toolset for black-box hyperparameter optimisation.

R package to send function calls as jobs on LSF, SGE, Slurm, PBS/Torque, or each via SSH

A scheduler for GPU/CPU tasks

SEML: Slurm Experiment Management Library

Funnel is a toolkit for distributed task execution via a simple, standard API.

Singularity implementation of k8s operator for interacting with SLURM.

Slurm in Docker - Exploring Slurm using CentOS 7 based Docker images