# KAI Scheduler Deep Dive: How NVIDIA Schedules GPUs on Kubernetes

## Why GPU clusters need a different scheduler

The default `kube-scheduler` places one pod at a time, on one node, using a filter-and-score loop. That model is excellent for stateless microservices and wrong for most AI work. A 16-GPU PyTorch job is useless until all 16 workers run. A team that paid for 8 H100s expects to get them back when another team borrows them. And a notebook that needs a quarter of a GPU should not lock a whole one.

[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is NVIDIA's open-source answer to those problems. It is a Kubernetes-native batch scheduler, built on the ideas of `kube-batch`, that adds hierarchical queues with quotas, gang scheduling, fair share, reclaim and preemption, fractional GPUs, topology-aware placement and DRA. It runs as a second scheduler beside `kube-scheduler`, so you opt workloads in with `schedulerName: kai-scheduler`.

This post walks through how it actually works: every component, the path a pod takes from `kubectl apply` to a running container, the scheduling cycle and its five actions, the fair-share maths, how fractional GPUs are faked on a device plugin that only knows whole GPUs, and how topology shapes placement. Then a hands-on install and a comparison with Volcano and Kueue.

> **Version baseline (checked 21 October 2026):** KAI Scheduler v0.18.x (v0.18.0 released 23 September 2026, latest patch v0.18.6). The project is a CNCF Sandbox project, Apache 2.0 licensed. Several features covered here (semi-preemptible workloads, `queuePriorityInQuotaReclaim`, NvFractions) are alpha or new in 0.18, so pin versions and read the [migration guides](https://github.com/kai-scheduler/KAI-Scheduler/tree/main/docs/migrationguides) before upgrading.

### The short answer

*   **Use KAI** when several teams share a GPU pool and you need guaranteed quotas, borrowing of idle GPUs, and the ability to take those GPUs back automatically.
    
*   **Use KAI** when distributed training or multi-pod inference must start all-or-nothing, especially across racks or NVLink domains.
    
*   **Use KAI** when small jobs (notebooks, dev inference, evals) waste whole GPUs and you want fractional sharing.
    
*   **Stay on** `kube-scheduler` when you run a handful of single-GPU inference Deployments with no contention. A second scheduler is real operational surface.
    

## What KAI Scheduler is

KAI is the scheduling core of Run:ai, which NVIDIA acquired and then open-sourced in 2025. You can still see the lineage in the APIs: queues and pod groups live under `scheduling.run.ai`, while newer CRDs (Config, SchedulingShard, Topology) live under `kai.scheduler`. The project is now a [CNCF Sandbox project](https://github.com/kai-scheduler/KAI-Scheduler), integrates natively with [KubeRay](https://docs.ray.io/en/master/cluster/kubernetes/k8s-ecosystem/kai-scheduler.html), and provides the gang and topology layer underneath [Grove and Dynamo](https://developer.nvidia.com/blog/streamline-complex-ai-inference-on-kubernetes-with-nvidia-grove/).

Three design choices set it apart from `kube-scheduler`:

1.  **It schedules workloads, not pods.** Every pod belongs to a PodGroup, and the PodGroup is the unit of admission, fairness and eviction.
    
2.  **It schedules in periodic cycles over a snapshot.** Instead of reacting pod by pod, it freezes cluster state, simulates decisions in memory, and only commits what it has proven will work.
    
3.  **It separates deciding from binding.** The scheduler writes a `BindRequest`; a separate binder does the slow, failure-prone work of binding, volumes, DRA claims and GPU-sharing setup.
    

| Capability | What it gives you | Where it's configured |
| --- | --- | --- |
| Hierarchical queues | Quota, over-quota weight, limit and priority per team, nested to any depth | `Queue` CRD |
| Gang scheduling | All-or-nothing start via `minMember`; multi-level gangs via SubGroups and `minSubGroup` | `PodGroup` (auto-created) |
| Fair share + DRF | Guaranteed quota first, surplus by priority then weight | `proportion` plugin |
| Reclaim / preempt | Take borrowed GPUs back across queues; enforce priority inside a queue | Actions, PriorityClasses |
| Time-based fair share | Penalise queues that used a lot recently, with half-life decay | Prometheus + `proportion` args |
| GPU sharing | Fractions or MiB of a GPU, optional memory enforcement (HAMi-core, NvFractions) | Pod annotations |
| Bin-pack / spread | Node-level and device-level placement strategy | `SchedulingShard.placementStrategy` |
| Consolidation | Repack running pods to defragment GPUs | `consolidation` action |
| Topology-aware scheduling | Required / preferred rack, block or zone locality | `Topology` CRD + annotations |
| Elastic + semi-preemptible | Run between min and max size; protect the minimum, burst the rest | PodGroup spec |
| Min runtime, stale grace | Don't evict a job too soon; give broken gangs time to recover | Queue spec, annotations |
| DRA | ResourceClaims, including GB200 ComputeDomains / IMEX | Feature gate |
| Scheduling shards | Independent scheduler instances per node pool | `SchedulingShard` CRD |

It is supported on any conformant Kubernetes, including OpenShift, and works with node autoscalers such as Cluster Autoscaler and Karpenter. For GPUs it expects the [NVIDIA GPU Operator](https://github.com/NVIDIA/gpu-operator) (or the DRA driver).

## Architecture: nine components, one API server

KAI is not one binary. It is a set of controllers that never call each other directly: each one watches and writes Kubernetes objects, and the API server is the only bus. That makes each piece independently restartable, and it means every decision leaves a trace you can `kubectl get`.

![KAI Scheduler architecture: 9 components around the API server](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/04764005-b019-47ff-a516-457e6c196944.svg align="center")

Left of the API server sit the components that prepare work; right of it sit the ones that decide and act. Everything is installed by one Helm chart into the `kai-scheduler` namespace and reconciled by the operator.

| Component | Binary (`cmd/`) | Reads | Writes |
| --- | --- | --- | --- |
| **Operator** | `operator` | `Config`, `SchedulingShard` | Deployments, ConfigMaps, webhooks for every component |
| **Admission** | `admission` | Pods, Queues, PodGroups | Pod mutations (runtimeClass, GPU-sharing env), admit/deny |
| **Pod Grouper** | `podgrouper` | Pods and their owner chain | `PodGroup` objects, `pod-group-name` annotation on pods |
| **Scheduler** | `scheduler` | Everything (via informers) | `BindRequest`s, pod evictions, PodGroup conditions, events |
| **Binder** | `binder` | `BindRequest`s | Pod bindings, PVC binding, ResourceClaim allocation, reservation pods |
| **Pod Group Controller** | `podgroupcontroller` | Pods, PodGroups | PodGroup status (phase, resource usage) |
| **Queue Controller** | `queuecontroller` | PodGroups, Queues | Queue status (allocated, requested), metrics |
| **Resource Reservation** | `resourcereservation` | Its own pod + NVML | Annotation with the physical GPU index it holds |
| **Node Scale Adjuster** | `nodescaleadjuster` | Unschedulable fractional pods | Scaling pods that request whole GPUs |

Two things are worth calling out. First, the **operator** is the single source of configuration: you change behaviour by editing the cluster-scoped `Config` (`kai-config`) or a `SchedulingShard`, not by patching Deployments. Second, the **scheduler/binder split** is what lets the scheduler run a tight loop over thousands of nodes. Binding a pod with a PVC or a DRA claim can take seconds and fail transiently; the binder retries with backoff while the scheduler has already moved on, treating the `BindRequest` as the expected outcome in its next snapshot.

## The life of a pod: from `kubectl apply` to a running container

Follow one 3-pod PyTorchJob through the system. Nothing reaches a node until the scheduler has proven the whole gang fits, and nothing is bound until the binder has prepared every dependency.

![Lifecycle of a gang-scheduled pod: 12 steps across 7 actors](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/fe0b776b-d3c1-496d-930f-9dd5f6e7fffc.svg align="center")

1.  **Submit.** The workload carries two opt-ins: `schedulerName: kai-scheduler` in the pod template and the `kai.scheduler/queue: <leaf-queue>` label. The training operator creates the master and worker pods.
    
2.  **Admission.** KAI's webhooks validate the pod (a fractional request can't be combined with `nvidia.com/gpu`, `gpu-fraction` must be between 0 and 1, and so on) and mutate it, for example injecting a runtime class for GPU-sharing pods.
    
3.  **Grouping.** The pod grouper sees each pod, walks `ownerReferences` up to the PyTorchJob, and picks the plugin for that kind.
    
4.  **PodGroup.** It creates one PodGroup with `minMember` equal to the replica count (3), the queue, and a default priority class (`train` for PyTorchJob). It annotates each pod with `pod-group-name`.
    
5.  **Cache.** The scheduler's informers pick up pods, nodes, queues, PodGroups and BindRequests.
    
6.  **Cycle.** At the next `schedulePeriod`, the scheduler snapshots state and runs its actions. Allocate only commits if all 3 pods can be placed in the same simulation.
    
7.  **Decision.** For each pod it writes a `BindRequest` (pod, node, GPU allocation, DRA claims, retry policy). The pod itself is not touched yet.
    
8.  **Pickup.** The binder watches BindRequests.
    
9.  **PreBind.** Binder plugins run in priority order: `volumebinding` (300), `dynamicresources` (200), `gpusharing` (100), and optionally `hamicore` (50).
    
10.  **Bind.** The binder creates the `Binding` subresource and marks the BindRequest succeeded. On failure it retries up to `BackoffLimit`, then marks it failed so the scheduler can try again.
     
11.  **Watch.** The pod now has a `nodeName`.
     
12.  **Run.** kubelet starts the containers; the pod group controller moves the PodGroup to Running.
     

One consequence matters for debugging: if pods are `Pending` with no events from `kube-scheduler`, check the PodGroup's conditions and events first, then any BindRequests in the pod's namespace (`kubectl get bindrequests -n <ns>`). KAI writes scheduling explanations as Kubernetes events at each major step.

## The scheduling cycle: snapshot, session, actions, plugins

`kube-scheduler` is event-driven: a pod appears, it gets scored and bound. KAI is cycle-driven. Every `schedulePeriod` it freezes the cluster into a snapshot and makes all decisions against that frozen view, so a 64-pod gang is judged against one consistent picture rather than a moving target.

![Scheduling cycle: 3 setup steps, 5 actions](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/cba47490-b174-42da-bab8-cd7e3c074e74.svg align="center")

The pieces, in order:

*   **Cache.** Built from informers over pods, nodes, queues, PodGroups, BindRequests, topologies, ResourceClaims and more. It is also where committed decisions are applied back to the cluster.
    
*   **Snapshot.** A point-in-time copy. It guarantees consistency within a cycle, avoids repeated API calls, and can be dumped for offline debugging (the `snapshot` plugin plus the `snapshot-tool` binary replay a cycle locally).
    
*   **Session.** The context for one cycle. Plugins hook in at `OnSessionOpen` and register functions; actions call those functions; `OnSessionClose` cleans up.
    
*   **Actions.** Run in sequence. Each processes jobs one at a time and creates a statement per job.
    

### The five actions

| Action | Scope | Evicts? | Commits when |
| --- | --- | --- | --- |
| **Allocate** | Pending jobs, ordered by queue then job | Never | The job's gang (or its `minMember` floor) fits on free capacity |
| **Consolidate** | Running pods that fragment GPUs | Temporarily, to relocate | Every moved pod has a new home *and* the pending job fits. Disabled when any placement strategy is `spread` |
| **Reclaim** | Across sibling queues | Preemptible, over-quota pods of other queues | The reclaimer is below its fair share or quota and the victim's queue stays at or above its own |
| **Preempt** | Inside one queue | Preemptible pods of strictly lower priority | The preemptor fits after evictions |
| **StaleGangEviction** | Running gangs | Whole gangs below `minMember` | The stale grace period (default 60s) has expired |

The action list and order are configurable per shard through `SchedulingShard.spec.actions`. Each action has a priority (allocate 500, consolidation 400, reclaim 300, preempt 200, stalegangeviction 100) and can be disabled. `queueDepthPerAction` caps how many jobs per queue the expensive actions consider in one cycle.

### Simulate first, commit later

Disruptive actions never evict on a hunch. They generate *scenarios* ("evict these victims, then place this job"), run cheap *scenario filters* to discard hopeless ones, then simulate the survivors inside a **Statement**: a transaction log over the snapshot.

![Inside an action: scenario, simulation, commit or rollback](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/852533e8-7f3e-4514-b5b5-09699bebc8b8.svg align="center")

```go
cp := stmt.Checkpoint()          // rollback point
stmt.Evict(victimPod, "reclaim") // virtual eviction
stmt.Pipeline(pod, node)          // place on resources that are being freed
stmt.Allocate(pod, node)          // place on free resources
if ok { stmt.Commit() } else { stmt.Rollback(cp) } // or stmt.Discard()
```

`Pipeline` is the clever part: a pod can be assigned to a node whose resources are still held by pods being evicted. That lets an action count capacity that is being freed in the same cycle, so the job that triggered the evictions has a claim on exactly that capacity instead of racing other jobs for it later.

### Plugins: where policy lives

Actions are mechanics; plugins are policy. During `OnSessionOpen` each plugin registers functions on the session: `AddJobOrderFn`, `AddTaskOrderFn`, `AddQueueOrderFn`, `AddNodeOrderFn` (scoring), `AddGpuOrderFn` (which device on a node), `AddPrePredicateFn` / `AddPredicateFn` (filtering), `AddReclaimableFn` and friends (victim validation), plus `AllocateFunc` / `DeallocateFunc` event handlers that track virtual changes during simulation.

| Role | Default plugins (priority) |
| --- | --- |
| Filtering | `predicates` (1900), `nodeavailability` (1600), `resourcetype` (1500), `podaffinity` (1400), `dynamicresources` (700) |
| Fairness and ordering | `proportion` (1800), `priority` (1700), `subgrouporder` (1000), `taskorder` (900), `nominatednode` (800) |
| Workload integrations | `elastic` (1300), `kubeflow` (1200), `ray` (1100) |
| Protection | `minruntime` (600) |
| Placement | `topology` (500), `gpupack` or `gpuspread` (300), `nodeplacement` (200), `gpusharingorder` (100) |
| Debugging | `snapshot` (400) |

Higher priority runs first. You can disable, reorder, re-argument or add plugins in `SchedulingShard.spec.plugins`; supplying `arguments` replaces that plugin's defaults entirely, so copy the defaults you still want.

## Queues and hierarchical fair share

A queue is KAI's unit of ownership. Every workload must name a **leaf** queue, and each queue carries four knobs per resource (GPU, CPU, memory):

| Field | Meaning | Special values |
| --- | --- | --- |
| `quota` | Deserved (guaranteed) amount. Always handed out first | `-1` unlimited, `0` none |
| `overQuotaWeight` | Share of surplus relative to siblings at the same priority | Default 1 |
| `limit` | Hard cap, including borrowed capacity | `-1` no limit, `0` nothing above quota |
| `priority` | Who gets surplus first among siblings | Higher wins |

Units are GPUs (1 = one device), CPU millicores and memory in MB. Queues nest through `spec.parentQueue`, so you can model departments, teams and projects.

![Hierarchical queues: 1 root, 2 parents, 5 leaves](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/a546d1ac-3f0a-44e9-b1fe-96cf6ec6cd86.svg align="center")

```yaml
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: ml-org
spec:
  resources:
    gpu: { quota: 24, overQuotaWeight: 3, limit: -1 }
    cpu: { quota: -1, limit: -1 }
    memory: { quota: -1, limit: -1 }
---
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: training
spec:
  parentQueue: ml-org
  priority: 1
  resources:
    gpu: { quota: 12, overQuotaWeight: 2, limit: -1 }
    cpu: { quota: -1, limit: -1 }
    memory: { quota: -1, limit: -1 }
```

A fresh install creates `default-parent-queue` and its leaf `default-queue`, both with zero quota and no limit, so you can submit immediately and design real queues later.

### How fair share is computed

Every cycle, the `proportion` plugin divides capacity top down. At each level it runs two phases:

1.  **Deserved quota.** Each queue gets `min(quota, requested)`. Priority is ignored here; every queue is treated equally.
    
2.  **Over-quota.** Whatever remains goes to the highest-priority bucket first. Inside a bucket it is split by `overQuotaWeight`, capped by each queue's remaining request and `limit`. Only then does the next priority bucket see anything.
    

```python
# phase 1
for q in queues:
    q.fair_share = min(q.quota, q.requested)
remaining = total - sum(q.fair_share for q in queues)
# phase 2, per priority bucket (highest first)
while remaining > 0:
    w = sum(q.over_quota_weight for q in bucket)
    for q in bucket:
        give = min(remaining * q.over_quota_weight / w, q.remaining_request)
        q.fair_share += give; remaining -= give
```

The result then recurses into each parent's children.

![Fair-share example: 16 GPUs, 3 queues, 2 phases](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/df913dc3-cb1b-4a3c-b7e7-9c1b533da0ba.svg align="center")

In this example, phase 1 hands out 6 + 4 + 2 = 12 GPUs. Of the 4 left, `inference` (priority 2) is served first and only needs 1 more. The remaining 3 go to the priority-1 bucket and split 2:1 by weight. The consequence people miss: a queue with **priority 2 and weight 1 beats a queue with priority 1 and weight 100** for surplus. Weight only matters between equals.

Two more details make this work on real clusters. Fair share is computed per resource and combined with **Dominant Resource Fairness (DRF)**, so a CPU-heavy queue and a GPU-heavy queue are compared on the resource each uses most. And every queue gets a **saturation ratio**, `allocated / fairShare`. Queues below 1 are scheduled first; queues above 1 are the candidates when someone else needs capacity back (next sections).

One rule ties fairness to workload priority: **non-preemptible workloads (priority 100 or higher) can only use in-quota resources.** Surplus is lent, so only workloads that can be evicted may borrow it.

### Time-based fair share

Plain fair share is memoryless: a queue that hogged the cluster all night is treated the same as one that ran nothing. Since v0.10, KAI can feed historical usage back in. The pod group controller publishes per-PodGroup usage, the queue controller aggregates it into a metric, Prometheus stores it, and the scheduler queries it. Usage decays with a configurable half-life:

$$U = 0.5^{\Delta t / t_{1/2}} \cdot A$$

Usage is normalised to cluster capacity over the window (24 GPU-hours in a 10-GPU cluster over 10 hours scores 0.24), and the `kValue` argument controls how strongly it shrinks a queue's over-quota share. Deserved quota is never affected. Enable it with `spec.prometheus.enabled: true` on `kai-config` and point the scheduling shard at that Prometheus; the repo ships a [simulator](https://github.com/kai-scheduler/KAI-Scheduler/tree/main/cmd/time-based-fairshare-simulator) to tune parameters offline.

## Gang scheduling, PodGroups and SubGroups

Gang scheduling means a workload starts with at least its minimum set of pods, or not at all. In KAI the gang is a **PodGroup**, and its floor is `minMember`.

![Why gang scheduling matters: 6 GPUs, 2 jobs of 4](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/7e84ddb8-d250-41b6-b898-ef40205565f0.svg align="center")

The failure on the left is not hypothetical. NCCL-based training blocks in initialisation until every rank joins, so partially placed jobs hold GPUs forever. With gangs, the scheduler only commits job A when all 4 workers fit in one simulation, and job B stays pending without holding anything.

### PodGroups are created for you

You rarely write a PodGroup. The pod grouper walks each pod's owner chain to the top-level workload and uses a per-kind plugin to fill in the queue, `minMember` and a default priority class. Plugins exist for batch Jobs and CronJobs, Deployments, Kubeflow (PyTorchJob, MPIJob and friends), JobSet, LeaderWorkerSet, Ray, Knative, Spark, Grove and more.

| Workload | Default PodGroup shape | Default priority class |
| --- | --- | --- |
| `batch/v1` Job | `minMember: 1` (no gang); override with `kai.scheduler/batch-min-member` | `train` |
| Deployment | One PodGroup per Deployment, `minMember: 1` (new default in v0.18) | `inference` |
| PyTorchJob / MPIJob | All replicas, or `minAvailable` from the scheduling policy | `train` |
| JobSet | One PodGroup; a parent SubGroup per replicatedJob and a leaf per replica | `train` |
| Kubeflow Notebook | Single pod | `build` |

The grouper can also "look through" orchestration layers. An Argo Workflow that launches a PyTorchJob is grouped by the PyTorchJob's rules, not by the Workflow. If you need several workloads in one gang, or your own controller owns the lifecycle, create the PodGroup yourself, set `pod-group-name` on the pod template, and add `kai.scheduler/skip-podgrouper: "true"`.

```yaml
apiVersion: scheduling.run.ai/v2alpha2
kind: PodGroup
metadata:
  name: hpo-sweep
  namespace: ml
spec:
  queue: training
  minMember: 2              # start once any 2 trials fit
  priorityClassName: train
```

### Hierarchical PodGroups (SubGroups)

A flat gang is too blunt for disaggregated inference or multi-role training. Since v0.10, a PodGroup can contain a tree of **SubGroups**. Leaves carry `minMember`; the root and mid-level nodes carry `minSubGroup` (how many direct children must be satisfied). Each node can carry its own topology constraint.

![Hierarchical PodGroup: disaggregated serving, 2 stages, 4 leaf SubGroups](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/71dd1e84-dab2-42da-a0c8-c7c10567ffb6.svg align="center")

The scheduler orders SubGroups so the ones below their floor are filled first, then spreads extra pods to the SubGroup with the lowest `allocated / minMember` ratio. This is the layer that NVIDIA's Grove uses to express a Dynamo deployment as one schedulable unit.

### When a gang breaks

If a running gang loses pods (a node dies, a pod OOMs) and drops below `minMember`, it is **stale**. Holding its remaining GPUs would recreate the deadlock above, so the StaleGangEviction action evicts the rest. Because transient failures are common, KAI first waits a **stale grace period**: 60s by default, configurable globally (`globalDefaultStalenessGracePeriod`) or per workload with the `kai.scheduler/staleness-grace-period` annotation. A negative value disables stale eviction for that workload.

## GPU sharing: fractions of a GPU on a device plugin that only counts whole ones

The NVIDIA device plugin advertises `nvidia.com/gpu` as an integer. KAI lets a pod ask for part of one anyway, through an annotation, and keeps the bookkeeping itself.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: notebook-a
  labels:
    kai.scheduler/queue: notebooks
  annotations:
    gpu-fraction: "0.5"        # or gpu-memory: "20000" (MiB)
spec:
  schedulerName: kai-scheduler
  containers:
    - name: jupyter
      image: nvcr.io/nvidia/pytorch:25.09-py3
      # no nvidia.com/gpu here: mixing the two is rejected by admission
```

The trick is a placeholder. To kubelet, a GPU used for sharing is owned by one **reservation pod** that requests `nvidia.com/gpu: 1`. To KAI, that GPU is a *GPU group* with capacity 1.0 that it packs fractional pods into.

![GPU sharing: two views of the same GPU](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/d7837965-c89a-4573-9d58-fad180d095fb.svg align="center")

Binding a fractional pod is where the separate binder earns its keep:

![Fractional GPU binding: 5 steps](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/8921a1cb-fe6d-4345-b8ac-8bf4d7f807fb.svg align="center")

The reservation pod is a tiny program that asks NVML which physical GPU it was given and annotates itself with the index (`run.ai/reserve_for_gpu_index`). The binder's `gpusharing` plugin then labels the user pod with the GPU group, sets `NVIDIA_VISIBLE_DEVICES` (or a CDI device name when CDI is on) and `GPU_PORTION` through a per-pod ConfigMap, and binds it.

### Isolation is a separate decision

Placing pods on a shared GPU and stopping them from using each other's memory are different problems. Admins pick one cluster-wide mode with `global.gpuSharingMode`:

| Mode | Scheduling | Memory enforced? | Request annotations |
| --- | --- | --- | --- |
| `NonMemoryEnforced` (default) | Fractions and MiB | No. A container can see and use the whole GPU | `gpu-fraction`, `gpu-memory` |
| `HamiCore` | Fractions and MiB | Yes, via HAMi-core injected `CUDA_DEVICE_MEMORY_LIMIT` | `gpu-fraction`, `gpu-memory` |
| `NvFractions` (new in v0.18) | Per-container memory quantities | Yes, via gpu-fractioning (needs GPU Operator 26.7.1+, r615 driver, containerd 2.0 with NRI) | `nvidia.com/container.<name>.gpu-memory.request` / `.limit` |
| `Disabled` | Rejected | n/a | n/a |

In the default mode, a fraction is a scheduling promise, not a fence. That is fine for trusted notebooks and evals and risky for multi-tenant inference: one process that allocates the whole GPU will OOM its neighbours. If tenants don't trust each other, use `HamiCore` or `NvFractions`, or MIG.

A few practical rules. Only one container per pod can request a fraction. `gpu-fraction-num-devices: "2"` asks for the same fraction on two devices. KAI does not support the device plugin's time-slicing `replicas` setting; use these modes instead. And because fractional pods request no `nvidia.com/gpu`, a cluster autoscaler cannot see their demand. Enable `global.clusterAutoscaling=true` and the **node scale adjuster** sums pending fractions (two 0.5 pods become one) and launches whole-GPU scaling pods that the autoscaler does see.

## Placement: bin-pack, spread and topology

Once a job is allowed to run, KAI still has to choose *where*. Three layers of plugins answer that: `nodeplacement` picks nodes, `gpupack` / `gpuspread` pick devices inside a node, and `topology` constrains both to a rack, block or zone.

### Bin-pack vs spread

![Placement strategies: binpack vs spread on 4 nodes](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/8d273961-598e-4ff1-804e-4fe415e45a80.svg align="center")

The strategy is set per shard and per resource:

```yaml
apiVersion: kai.scheduler/v1
kind: SchedulingShard
metadata:
  name: default
spec:
  placementStrategy:
    gpu: binpack   # node level: fill the busiest node that fits
    cpu: spread
```

Node-level and device-level placement are independent. You can spread pods across nodes for resilience but still pack them onto the same GPUs inside each node by setting `gpu: spread` and enabling `gpupack` instead of `gpuspread`. With fractional GPUs, `gpusharingorder` prefers GPUs that are already shared, which keeps whole GPUs free for whole-GPU jobs.

One coupling to remember: the **consolidation** action only runs when every placement strategy is `binpack`. Spreading and defragmenting pull in opposite directions.

### Topology-aware scheduling (TAS)

A 32-GPU training job spread over four racks spends its life waiting on the spine network. TAS fixes that. You describe the physical hierarchy once, from coarsest to finest, using node labels you already have (cloud providers and NVIDIA's topology tooling publish them):

```yaml
apiVersion: kai.scheduler/v1alpha1
kind: Topology
metadata:
  name: cluster-topology
spec:
  levels:
    - nodeLabel: cloud.provider.com/topology-block
    - nodeLabel: cloud.provider.com/topology-rack
    - nodeLabel: kubernetes.io/hostname
```

Workloads then opt in with annotations. **Required** placement is a hard boundary; **preferred** placement is a target that may fall back to a larger domain. Combining them gives "must stay in one block, try hard to stay in one rack":

```yaml
metadata:
  annotations:
    kai.scheduler/topology: cluster-topology
    kai.scheduler/topology-required-placement: cloud.provider.com/topology-block
    kai.scheduler/topology-preferred-placement: cloud.provider.com/topology-rack
```

The algorithm deliberately pulls in two directions:

![Topology-aware scheduling: 2 zones, 7 racks, 1 job](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/b4588e5a-19b8-4421-8549-9ba740d39ba3.svg align="center")

Domain selection **bin-packs** (choose the domain with the least free capacity that still fits) to keep large empty domains for the next big job. Node ordering inside the domain does the opposite at the preferred level (choose the sub-domain with the *most* free capacity) to keep this job's pods together. In hierarchical PodGroups, each SubGroup can carry its own constraint, so prefill can be pinned to one rack and decode to another while the PodGroup is still admitted as one gang. On GB200/GB300 systems the same mechanism is how you keep a job inside one NVLink domain.

For finer locality inside a node, KAI also has a `numa` plugin and a NUMA placement exporter that align GPUs, CPUs and memory to the same socket.

## Priority, preemption, reclaim and consolidation

KAI has two priorities that are easy to confuse. **Queue priority** decides which queue gets surplus first (previous section). **Workload priority**, a normal Kubernetes `PriorityClass`, decides order and eviction *inside* a queue. The chart installs four:

| PriorityClass | Value | Preemptible | Typical use | Default for |
| --- | --- | --- | --- | --- |
| `train` | 50 | Yes | Training, batch, HPO | Jobs, PyTorchJob, MPIJob, everything else |
| `build-preemptible` | 75 | Yes | Interactive work you're willing to lose |  |
| `build` | 100 | No | Notebooks, dev pods | Kubeflow Notebook |
| `inference` | 125 | No | Serving | Deployment, Knative Service |

The threshold is 100: anything at or above it is non-preemptible by default and, in exchange, can only run inside its queue's quota. Since recent releases you can decouple the two with an explicit `preemptibility` on the PodGroup (`preemptible`, `non-preemptible`, or the alpha `semi-preemptible`), so "high priority but evictable" is expressible.

### Two eviction paths, never mixed

![Eviction rules: preempt vs reclaim, 4 checks each](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/605fdc91-130c-48fb-aea8-4bbf7ae1f8a2.svg align="center")

The split is strict. A workload can never preempt across queues, and reclaim never targets a workload in the same queue. That is what gives teams a real guarantee: **as long as a queue runs only inside its deserved quota, nothing outside it can evict its pods.** Reclaim only ever takes back what was lent.

Reclaim also preserves order. After a reclaim, the queue that had the lowest saturation ratio among its siblings still has the lowest, and a queue that was under quota is still under quota. That stops two queues from evicting each other forever. Two knobs tune it on the `proportion` plugin:

*   `reclaimerUtilizationMultiplier` (default `1.0`, must be at least 1). Values above 1 make reclaim more conservative.
    
*   `queuePriorityInQuotaReclaim` (alpha, default `false`). Lets a strictly higher-priority queue reclaim *in-quota* preemptible pods from a lower-priority one, as long as it stays within its own quota. It weakens the guarantee above, so enable it per shard deliberately.
    

### Don't evict too early

Killing a job 30 seconds after it started wastes the start-up cost and can cause thrashing. The `minruntime` plugin lets queues set a floor, inherited down the hierarchy:

```yaml
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: training
spec:
  parentQueue: ml-org
  preemptMinRuntime: "10m"   # same-queue preemption waits 10 minutes
  reclaimMinRuntime: "30m"   # cross-queue reclaim waits 30 minutes
  resources:
    gpu: { quota: 12, overQuotaWeight: 2, limit: -1 }
```

Shard-level defaults live in `SchedulingShard.spec.minRuntime`. For reclaim you can resolve the value from the victim's queue or from the lowest common ancestor of the two queues.

### Consolidation: defragmenting without losing work

Reclaim and preempt make room by *removing* work. Consolidation makes room by *moving* it.

![Consolidation: before and after one cycle](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/0e6c17ed-3f2a-4692-8c63-928c6f86e72a.svg align="center")

It only commits if every displaced pod can be re-placed in the same simulation, so nothing is lost except a restart. That still matters: a pod that is moved is evicted and restarted, so workloads without checkpointing pay for it. Disable the action per shard (`spec.actions.consolidation.enabled: false`) if your jobs can't tolerate restarts.

## Elastic workloads, DRA and scheduling shards

### Elastic and semi-preemptible workloads

An elastic workload declares a floor and runs with whatever extra capacity is available. In KAI the floor is `minMember` (or `minSubGroup` for hierarchical gangs) and the ceiling is the number of pods the workload creates. Pods above the floor are scheduled opportunistically and, under pressure, evicted individually, leaders last, while the job keeps running. A PyTorchJob with `elasticPolicy` `minReplicas: 1, maxReplicas: 3` starts on one worker and grows to three when the cluster is idle.

The gap this leaves: an elastic *inference* service is usually high priority, which makes it non-preemptible and confined to quota, so it can't burst at all. v0.18 adds an alpha answer, **semi-preemptible**:

![Semi-preemptible PodGroup: 3 core pods, 3 elastic](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/f4c42a51-858d-4c05-afc1-cf223b3103cd.svg align="center")

```yaml
apiVersion: scheduling.run.ai/v2alpha2
kind: PodGroup
metadata:
  name: llm-serving
spec:
  queue: inference
  preemptibility: semi-preemptible
  minMember: 3          # 3 protected pods; pods 4+ are elastic
```

With `minMember: 0` it behaves like a fully preemptible job; with `minMember` equal to the pod count, like a non-preemptible one. Raising `minMember` on a running semi-preemptible PodGroup is rejected by admission, because it would silently re-charge elastic pods to quota. It is alpha, so keep it off production paths for now.

### Dynamic Resource Allocation

DRA replaces integer device counts with `ResourceClaim`s that a driver satisfies. KAI's snapshot includes claims, the `dynamicresources` plugin filters nodes on them, and the binder's `dynamicresources` plugin writes the allocation before binding. The headline use case is GB200/GB300: NVIDIA's `k8s-dra-driver-gpu` exposes Multi-Node NVLink through a `ComputeDomain`, and pods request IMEX channels as claims. The scheduler records the claim allocation in the BindRequest:

```yaml
spec:
  podName: gpu-imex-pod
  receivedResourceType: Regular
  receivedGPU: { count: 1, portion: "1.00" }
  resourceClaimAllocations:
  - allocation:
      devices:
        config:
        - opaque:
            driver: compute-domain.nvidia.com
```

DRA support is behind a feature gate at install time: pass `--feature-gates=DynamicResourceAllocation=true` to both scheduler and binder through `scheduler.additionalArgs` and `binder.additionalArgs`.

### Scheduling shards

One scheduler instance does not have to own the whole cluster. A `SchedulingShard` creates a separate scheduler for nodes, queues and PodGroups that share a `kai.scheduler/node-pool` label value.

![Scheduling shards: 2 shards, one operator](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/81c475d3-e629-45b7-86a8-2e45ab1c5724.svg align="center")

Shards are how you give an H100 pool bin-packing and aggressive reclaim while a mixed CPU/GPU pool spreads, or isolate a noisy research pool from production. The trade-off: capacity in one shard can never be borrowed by another.

### Everything else worth knowing

*   **Background pods** (v0.18): maintenance pods are ignored when scheduling and evicted when they're in the way.
    
*   **In-place pod resize** (v0.18): KEP-1287 resizes are accounted against queue quota and limits, with a queue admission webhook.
    
*   **NUMA-aware placement**: a `numa` plugin plus a NUMA placement exporter keep GPU, CPU and memory on one socket.
    
*   **Explainability**: every major scheduling step is an event, and `detailed-fit-errors` (off by default, costly on big clusters) explains exactly why a pending pod didn't fit a node.
    
*   **Snapshot tool**: dump a cycle's snapshot and replay it locally to debug a decision.
    
*   **Operations**: HA replicas with PDBs for binder and queue controller, JSON logging, a `fipsMode`, and public [scale-test](https://kai-scheduler.github.io/KAI-Scheduler/scale-tests/) and [benchmark](https://kai-scheduler.github.io/KAI-Scheduler/dev/bench/) dashboards.
    

## Hands-on: install KAI and run real workloads

You need a Kubernetes cluster, Helm, and the NVIDIA GPU Operator on the GPU nodes. Everything below works on a single GPU node; the gang example wants three GPUs.

### 1\. Install

```bash
helm upgrade -i kai-scheduler \
  oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler \
  -n kai-scheduler --create-namespace \
  --version v0.18.6

kubectl -n kai-scheduler get pods
kubectl get config kai-config -o jsonpath='{.status.conditions}' | jq
kubectl get queues      # default-parent-queue, default-queue
```

Useful flags at install time: `--set global.gpuSharingMode=HamiCore` for memory-isolated sharing, `--set global.clusterAutoscaling=true` with an autoscaler, and on OpenShift with GPU Operator older than 25.10 add `--set admission.gpuFractionRuntimeClassName=null`. Never submit workloads into the `kai-scheduler` namespace.

### 2\. Create team queues

```yaml
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: ml-org
spec:
  resources:
    gpu: { quota: -1, overQuotaWeight: 1, limit: -1 }
    cpu: { quota: -1, overQuotaWeight: 1, limit: -1 }
    memory: { quota: -1, overQuotaWeight: 1, limit: -1 }
---
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: training
spec:
  parentQueue: ml-org
  resources:
    gpu: { quota: 2, overQuotaWeight: 2, limit: -1 }
    cpu: { quota: -1, overQuotaWeight: 1, limit: -1 }
    memory: { quota: -1, overQuotaWeight: 1, limit: -1 }
---
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: notebooks
spec:
  parentQueue: ml-org
  resources:
    gpu: { quota: 1, overQuotaWeight: 1, limit: 2 }
    cpu: { quota: -1, overQuotaWeight: 1, limit: -1 }
    memory: { quota: -1, overQuotaWeight: 1, limit: -1 }
```

### 3\. A whole-GPU pod and a fractional pod

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
  namespace: ml
  labels: { kai.scheduler/queue: training }
spec:
  schedulerName: kai-scheduler
  containers:
    - name: cuda
      image: nvidia/cuda:13.0.2-base-ubi8
      command: ["sh", "-c", "nvidia-smi -L && sleep 3600"]
      resources: { limits: { nvidia.com/gpu: 1 } }
---
apiVersion: v1
kind: Pod
metadata:
  name: half-gpu
  namespace: ml
  labels: { kai.scheduler/queue: notebooks }
  annotations: { gpu-fraction: "0.5" }
spec:
  schedulerName: kai-scheduler
  containers:
    - name: cuda
      image: nvidia/cuda:13.0.2-base-ubi8
      command: ["sh", "-c", "nvidia-smi && sleep 3600"]
```

After applying, look at what KAI created around them:

```bash
kubectl -n ml get podgroups                      # one per pod
kubectl -n ml get bindrequests                   # the scheduler's decisions
kubectl -n kai-resource-reservation get pods     # the GPU holder for half-gpu
kubectl -n ml exec half-gpu -- env | grep -E 'NVIDIA_VISIBLE_DEVICES|GPU_PORTION'
```

### 4\. A gang: distributed PyTorch

With the Kubeflow training operator installed, the only KAI-specific lines are the queue label and `schedulerName`. The pod grouper sets `minMember: 3`.

```yaml
apiVersion: kubeflow.org/v1
kind: PyTorchJob
metadata:
  name: ddp-mnist
  namespace: ml
  labels: { kai.scheduler/queue: training }
spec:
  pytorchReplicaSpecs:
    Master:
      replicas: 1
      restartPolicy: OnFailure
      template:
        spec:
          schedulerName: kai-scheduler
          containers:
            - name: pytorch
              image: ghcr.io/kubeflow/training-v1/pytorch-dist-mnist:latest
              args: ["--backend", "nccl", "--epochs", "5"]
              resources: { limits: { nvidia.com/gpu: 1 } }
    Worker:
      replicas: 2
      restartPolicy: OnFailure
      template:
        spec:
          schedulerName: kai-scheduler
          containers:
            - name: pytorch
              image: ghcr.io/kubeflow/training-v1/pytorch-dist-mnist:latest
              args: ["--backend", "nccl", "--epochs", "5"]
              resources: { limits: { nvidia.com/gpu: 1 } }
```

On a node with fewer than three free GPUs, all three pods stay `Pending` and hold nothing. Delete `gpu-pod` and watch all three start together.

### 5\. When something stays Pending

![Pending-pod triage: 5 checks in order](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/78e6e5aa-404b-4b08-9477-5514604e194a.svg align="center")

The commands behind each check: `kubectl get pod <p> -o jsonpath='{.spec.schedulerName}'`, `kubectl get queue <q> -o yaml`, `kubectl -n <ns> get podgroups`, `kubectl -n <ns> describe podgroup <pg>`, and `kubectl -n <ns> describe bindrequest <br>`. For deeper digging, turn on `detailed-fit-errors` temporarily or use the snapshot tool to replay the cycle offline. The repo also ships an agent skill that walks this same diagnosis.

## KAI vs Volcano vs Kueue vs plain kube-scheduler

All four can run GPU jobs. The real difference is which layers each one owns.

![Layer ownership: 4 options, 4 layers](https://cdn.hashnode.com/uploads/covers/63cc2392587fc59d3cd765f9/bc0e4e8d-76ce-430b-9ff4-348f443adf37.svg align="center")

**Kueue** (a Kubernetes SIG project) is a job-level admission controller. It holds jobs suspended until their ClusterQueue has quota, then lets `kube-scheduler` place the pods. That keeps the upstream scheduler and every plugin you rely on, but placement and admission are separate decisions, so "admitted" is not the same as "all pods fit on nodes right now" unless you also use Kueue's topology-aware admission. **Volcano** (CNCF) and **KAI** both descend from `kube-batch` and replace the scheduler for the pods they own: queueing, gang checks and placement happen in one session over one snapshot. KAI additionally splits binding into its own controller.

|  | kube-scheduler | Kueue | Volcano | KAI Scheduler |
| --- | --- | --- | --- | --- |
| Model | Pod at a time | Job admission, then kube-scheduler | Batch scheduler, session + actions | Batch scheduler, session + actions, separate binder |
| Quota borrowing and take-back | No | Cohorts, borrowing, preemption | Queue capacity, reclaim | Quota, over-quota weight, priority, reclaim |
| Hierarchical queues | No | Hierarchical cohorts | Supported | Native, any depth |
| Gang scheduling | Via scheduler-plugins | At admission | PodGroup | PodGroup with SubGroups |
| Fractional GPUs | No (time-slicing or MIG at driver level) | No (relies on devices) | Via HAMi vGPU integration | Native: fraction, MiB, optional enforcement |
| Topology-aware | Pod topology spread | Topology-aware admission | Network-topology aware | Required/preferred, per SubGroup |
| Time-based fairness | No | Fair sharing with usage decay | Not the main model | Prometheus-backed, half-life |
| Best fit | Few, uncontended GPU services | Keep kube-scheduler, add multi-tenant admission | General batch and HPC, Spark, MPI | Shared GPU pools for training + inference, NVIDIA stack (Grove, Dynamo, KubeRay) |

Feature details move every release in all three projects, so check current docs before deciding on a single row.

### How I'd choose

**KAI first** if GPUs are the scarce resource and several teams share them. Hierarchical quotas with automatic reclaim, fractional GPUs, topology-aware gangs and a binder that handles DRA are all in one place, and it's the scheduler NVIDIA's own serving stack (Grove/Dynamo) is designed around.

**Kueue first** if you want to keep `kube-scheduler` untouched, mostly run whole-GPU batch jobs, or need multi-cluster dispatch. It's the least invasive way to add quotas and borrowing.

**Volcano first** if your workloads are broad batch and HPC (Spark, MPI, Flink) with GPUs as one resource among many, and you value its long production history and job CRD.

**None of them** if you have a few inference Deployments on dedicated nodes and no contention. A second scheduler is something you have to upgrade, monitor and debug.

### Production considerations for KAI

*   **Two schedulers, two views.** Pods without `schedulerName: kai-scheduler` are invisible to KAI's queues. Mixing schedulers on the same GPU nodes means KAI's fair share can be wrong. Dedicate GPU nodes to KAI or enforce `schedulerName` with policy (Kyverno, Gatekeeper).
    
*   **Quotas are a promise you must back.** Sum of leaf quotas should not exceed real capacity, or "guaranteed" isn't. v0.18 validates sibling sums against parents.
    
*   **Pick a sharing mode deliberately.** `NonMemoryEnforced` is a scheduling convention, not isolation.
    
*   **Plan for evictions.** Anything that borrows surplus (priority below 100) can be reclaimed. Train with checkpoints, set `reclaimMinRuntime` to protect start-up, and use `build`/`inference` classes for work that must not die.
    
*   **Size the scheduler.** Memory scales with cluster objects; the repo has [resource-sizing guidance](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/operator/resource-sizing.md). Run replicas with PDBs.
    
*   **Watch upgrades.** Pre-1.0 releases ship behaviour changes (v0.18 changed Deployment grouping to one PodGroup per Deployment). Read the migration guide for every minor bump.
    

## Where I landed

KAI's design comes down to three ideas: schedule *workloads* through PodGroups, decide over a *frozen snapshot* with simulate-then-commit actions, and hand the slow part to a *separate binder*. Almost every feature above (gangs, fair share and reclaim, fractional GPUs, topology, DRA) is a plugin or action on top of those three.

For a shared GPU platform I'd adopt it in this order:

1.  Install with defaults, dedicate GPU nodes to it, and move one team onto a leaf queue with a real quota.
    
2.  Add the other teams' queues and let over-quota borrowing run with `train`\-class jobs. Set `reclaimMinRuntime` before anyone complains about being reclaimed.
    
3.  Turn on GPU sharing for notebooks and evals, with `HamiCore` or `NvFractions` if tenants don't trust each other.
    
4.  Describe the network with a `Topology` CRD and add required/preferred placement to multi-node training and disaggregated serving.
    
5.  Only then reach for the alpha features: semi-preemptible inference, `queuePriorityInQuotaReclaim`, time-based fair share.
    

The test I'd hold it to is the same as for any control plane: can you explain, from `kubectl` output alone, why a pod is pending, why another was evicted, and who owns the GPUs right now? With PodGroups, BindRequests, queue status and events, KAI makes that answerable. That, more than any single feature, is why I'd run it.

## Sources

**KAI Scheduler repository (main branch, checked 21 October 2026)**

*   [KAI Scheduler on GitHub](https://github.com/kai-scheduler/KAI-Scheduler) and [CHANGELOG](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/CHANGELOG.md) (v0.18.0, 23 September 2026)
    
*   [Scheduler core concepts](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/developer/scheduler-concepts.md), [Action framework](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/developer/action-framework.md), [Plugin framework](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/developer/plugin-framework.md)
    
*   [Scheduling deep dive](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/scheduling-deep-dive/README.md)
    
*   [Binder](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/developer/binder.md) and [Pod Grouper](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/developer/pod-grouper.md)
    
*   [Queues](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/queues/README.md), [Fairness](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/fairness/README.md), [Time-based fair share](https://github.com/kai-scheduler/KAI-Scheduler/tree/main/docs/time-based-fairshare), [Priority](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/priority/README.md)
    
*   [Batch / gang scheduling](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/batch/README.md), [Hierarchical PodGroup design](https://github.com/kai-scheduler/KAI-Scheduler/tree/main/docs/developer/designs/hierarchical-podgroup), [Elastic and semi-preemptible](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/elastic/README.md)
    
*   [GPU sharing](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/gpu-sharing/README.md) and [GPU sharing with autoscalers](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/gpu-sharing/autoscaling/README.md)
    
*   [Topology-aware scheduling](https://github.com/kai-scheduler/KAI-Scheduler/tree/main/docs/topology), [DRA](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/dra/README.md), [Min runtime plugin](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/plugins/minruntime.md), [Stale grace period](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/staleness-grace-period/README.md)
    
*   [Operator](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/operator/README.md) and [scheduler config customization](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/operator/scheduler-config-customization.md)
    
*   [Quick start](https://github.com/kai-scheduler/KAI-Scheduler/blob/main/docs/quickstart/README.md)
    

**Ecosystem**

*   [KubeRay integration with KAI](https://docs.ray.io/en/master/cluster/kubernetes/k8s-ecosystem/kai-scheduler.html)
    
*   [NVIDIA Grove blog post](https://developer.nvidia.com/blog/streamline-complex-ai-inference-on-kubernetes-with-nvidia-grove/)
    
*   [NVIDIA GPU Operator](https://github.com/NVIDIA/gpu-operator)