Modal

Modal is a serverless cloud platform designed specifically for running Python-based AI, machine learning, and data engineering workloads. With a focus on developer experience and fast iteration, Modal enables data scientists and ML engineers to run compute-intensive workloads in the cloud using a remarkably simple Python SDK, without configuring servers, containers, or Kubernetes.

The Modal SDK is the primary interface for the platform. Developers decorate Python functions with modal.function() to designate them as cloud functions that run in Modal's infrastructure. The function's dependencies, base Docker image, and resource requirements (CPU, memory, GPU type and count) are specified as arguments to the decorator. When the function is called, Modal provisions the appropriate compute, runs the function, and returns the result, all transparently from the developer's perspective.

GPU support is central to Modal's value proposition for AI workloads. Any Python function can be assigned one or multiple NVIDIA A100, H100, A10G, or T4 GPUs by adding a single argument to the decorator. This makes it trivial to run GPU-accelerated workloads including LLM inference, model fine-tuning, image generation, and embedding computation without managing GPU infrastructure or worrying about GPU availability.

Fast cold starts are a critical feature for serverless inference workloads. Modal has invested significantly in container caching and image optimization techniques that reduce cold start times to seconds rather than minutes. This fast cold start performance makes Modal viable for interactive inference applications where even brief delays are unacceptable.

Container customization in Modal is done through the Image builder API. Developers chain image construction steps such as pip installations, apt package installs, Docker commands, and file copies using fluent Python syntax. Modal caches each layer of the image build process, ensuring that re-deployments only rebuild the layers that changed, keeping deployment times minimal even for complex environments.

Scheduled functions run on cron-style schedules, enabling periodic data processing, model evaluation, metric computation, and other recurring tasks without managing scheduler infrastructure. Web endpoints allow Modal functions to be exposed as HTTP APIs with automatic HTTPS and subdomain assignment, enabling rapid deployment of model serving APIs.

The Modal Dashboard provides visibility into running and completed function calls, resource consumption, logs, and usage metrics. The pricing model charges for actual GPU and CPU seconds consumed, with no minimum fees and no charges for idle capacity.

Modal is particularly well-suited for ML engineers who want to iterate quickly on model experiments, researchers running batch inference jobs, and production teams building AI-powered features who want the simplicity of serverless with the power of GPUs.