Batch processing refers to performing units of work, referred to as a job in a repetitive and unattended fashion. Jobs are typically grouped together and processed in batches (hence the name).
Kubernetes includes native support for running Jobs. Jobs can run multiple pods in parallel until receiving a set number of completions. Each pod can contain multiple containers as a single unit of work.
Argo enhances the batch processing experience by introducing a number of features:
In this module, we will build a simple Kubernetes Job, recreate that job in Argo, and add common features and workflows for more advanced batch processing.