In this talk we’ll discuss how Pipekit provisions thousands of Kubernetes clusters for their customers so that they can submit Argo Workflows within 3 minutes of registering. In this session, we’ll dive into our challenges and how Pipekit, co-maintainers of Argo Workflows, partnered with Loft Labs, the maintainers of vCluster, to create a fully-managed experience for new users of Argo Workflows who need to run data workloads at scale (thousands of steps/ pods). This session provides a deep-dive on the architectural challenges, design choices, missteps, and scaling limits Pipekit has dealt with while building its Argo Workflows offering, including:
- Secure isolation of tenants on a shared K8s cluster
- On-demand and fully-automated cluster & compute provisioning for tenants
- Ideal configurations for Argo Workflows and Argo CD with vCluster
- Enabling useful metrics & observability features for each tenant’s vCluster
Speakers
