> ## Documentation Index
> Fetch the complete documentation index at: https://gligabao.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Providers and extending

> How the provider abstraction keeps the training code cloud-neutral, how to add a provider, and how to add a dependency, a model size or a function.

`sidecar/ai_train/providers/`, `sidecar/ai_train/app.py`

This page is about the Modal app behind the sidecar (how the sidecar deploys and stops it is on [Modal app lifecycle](https://gligabao.mintlify.site/training/modal.md)). To make the [sidecar](https://gligabao.mintlify.site/operations/sidecar.md#backends) answer from something other than that Modal app, add a backend in `sidecar/backends/` instead; it needs no change in Go or in `ai_train`.

## How the provider abstraction works

`app.py` only says *what* to run. It builds a `FunctionSpec` (image, GPU, timeout, volumes, secrets) and registers functions with `@provider.function(spec)` (or `@provider.asgi_app(spec)` for the web API). It runs them with `provider.call(fn, ...)` or `provider.spawn(fn, ...)` and persists/refreshes volumes with `provider.commit_volumes(paths)` / `provider.reload_volumes(paths)`. Images are described by a provider-neutral `ImageSpec` (GPU or not, with or without the full CUDA toolkit, requirements file); the provider decides how to build them, and mounts the `ai_train/` directory into the container at `/pkg`. Code in `ai_train_core/` and `api/` gets what it needs from the provider as injected callables (e.g. `commit_fn`).

Rules that keep it swappable:

* Nothing outside `providers/` imports a provider SDK, and `ai_train_core/` / `api/` (the code that runs in the container) never do.
* `app = provider.native_app` is a module-level object because provider CLIs such as `modal run` look for it.
* Code that runs remotely imports `ai_train_core` inside the function body, since it only exists in the container's `/pkg` mount, and the heavy ML imports are per image (only the training image has Unsloth; the inference image has torch, transformers and peft).

To add a provider (e.g. AWS): implement `Provider` in `providers/aws_provider.py` and add one branch to `get_provider` in `providers/__init__.py`. If the new provider cannot express something in `FunctionSpec`, extend `FunctionSpec` rather than leaking provider details into `app.py`.

## Growing it

1. Add a dependency: put it in the requirements file of the image that needs it (an image is rebuilt only when its file changes; code changes do not rebuild it).
2. Add a model size: an entry in `ai_train_core/model_config.py`, a train/generate function pair in `app.py`, and a pipeline module if it needs a new backend.
3. Add another remote function: declare it in `app.py` with a `FunctionSpec` and a matching action in `main`.
4. Add an inference endpoint: a Go endpoint over trained models, a sidecar route, or a new GPU generation function; see [Adding an inference endpoint](https://gligabao.mintlify.site/training/trained-models.md#adding-an-inference-endpoint).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.