# InferenceClass Source: https://docs.modelplane.ai/v0.6/reference/inferenceclasses/ Hardware recipe defining devices and provisioning for a node pool. Apply instances as `apiVersion: modelplane.ai/v1alpha1`, `kind: InferenceClass`. [Concept guide: Define Hardware Classes](https://docs.modelplane.ai/v0.6/platform/inference-class/index.md) ## Example ```yaml apiVersion: modelplane.ai/v1alpha1 kind: InferenceClass metadata: name: gke-l4-1x-g2 spec: description: "GKE g2-standard-8, 1x NVIDIA L4" provisioning: provider: GKE gke: machineType: g2-standard-8 diskSizeGb: 100 accelerator: type: nvidia-l4 count: 1 devices: - name: gpu claim: DRA driver: gpu.nvidia.com deviceClassName: gpu.nvidia.com count: 1 attributes: architecture: { string: Ada Lovelace } capacity: memory: { value: "23034Mi" } ``` ## Definition The CompositeResourceDefinition this reference is generated from, with the complete OpenAPI schema, validation rules, and defaults: ```yaml apiVersion: apiextensions.crossplane.io/v2 kind: CompositeResourceDefinition metadata: name: inferenceclasses.modelplane.ai spec: group: modelplane.ai names: categories: [crossplane, modelplane, platform] kind: InferenceClass plural: inferenceclasses shortNames: [icl] scope: Cluster versions: - name: v1alpha1 served: true referenceable: true schema: openAPIV3Schema: type: object required: [spec] properties: spec: type: object required: [devices] x-kubernetes-validations: # The pool's role follows accelerator - compose-inference-cluster # gives a pool the GPU role when it's set and the CPU role when # it isn't - # while the scheduler and ServingStack spec.draDrivers follow # spec.devices. These rules keep the two in step, one pair per # cloud because CEL can't address the provider's block # dynamically: a class claiming a GPU driver without accelerator # would compose a CPU pool whose advertised GPUs nothing can # allocate, and accelerator on a class claiming only dra.cpu would # provision GPUs nothing claims. - rule: >- !has(self.provisioning) || !has(self.provisioning.gke) || has(self.provisioning.gke.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.gke.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.gke) || !has(self.provisioning.gke.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.gke.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.eks) || has(self.provisioning.eks.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.eks.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.eks) || !has(self.provisioning.eks.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.eks.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.aks) || has(self.provisioning.aks.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.aks.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.aks) || !has(self.provisioning.aks.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.aks.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.nebius) || has(self.provisioning.nebius.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.nebius.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.nebius) || !has(self.provisioning.nebius.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.nebius.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.vultr) || has(self.provisioning.vultr.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.vultr.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.vultr) || !has(self.provisioning.vultr.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.vultr.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.civo) || has(self.provisioning.civo.accelerator) || !(has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.civo.accelerator is required when a device claims a DRA driver other than dra.cpu. - rule: >- !has(self.provisioning) || !has(self.provisioning.civo) || !has(self.provisioning.civo.accelerator) || (has(self.devices) && self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu')) message: >- spec.provisioning.civo.accelerator must be omitted when no device claims a DRA driver other than dra.cpu. # dra-driver-cpu runs only on CPU pools, so a provisioned GPU # pool never publishes dra.cpu devices for a claim to allocate. - rule: >- !has(self.provisioning) || !self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver == 'dra.cpu') || !self.devices.exists(d, (!has(d.claim) || d.claim == 'DRA') && d.driver != 'dra.cpu') message: >- A class with spec.provisioning can't claim dra.cpu alongside another DRA driver: Modelplane runs dra-driver-cpu only on CPU pools. properties: description: type: string description: >- Human-readable description of the class. provisioning: type: object description: >- How to provision a node pool of this class. Omit for classes that describe BYO node pools that already exist. required: [provider] x-kubernetes-validations: - rule: "self.provider != 'GKE' || has(self.gke)" message: spec.provisioning.gke is required when spec.provisioning.provider is GKE. - rule: "self.provider != 'EKS' || has(self.eks)" message: spec.provisioning.eks is required when spec.provisioning.provider is EKS. - rule: "self.provider != 'AKS' || has(self.aks)" message: spec.provisioning.aks is required when spec.provisioning.provider is AKS. - rule: "self.provider != 'Nebius' || has(self.nebius)" message: spec.provisioning.nebius is required when spec.provisioning.provider is Nebius. - rule: "self.provider != 'Vultr' || has(self.vultr)" message: spec.provisioning.vultr is required when spec.provisioning.provider is Vultr. - rule: "self.provider != 'Civo' || has(self.civo)" message: spec.provisioning.civo is required when spec.provisioning.provider is Civo. properties: provider: type: string enum: [GKE, EKS, AKS, Nebius, Vultr, Civo] gke: type: object required: [machineType] properties: machineType: type: string minLength: 1 diskSizeGb: type: integer default: 100 minimum: 10 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node pool. Provisioning input only: the scheduler matches against spec.devices, not this block, so count here is the GCP machine's GPU count and need not be restated in devices. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU pool: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type passed to GCP (e.g. nvidia-l4, nvidia-h100-80gb). minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 eks: type: object required: [instanceType] properties: instanceType: type: string description: >- EC2 instance type (e.g. g6.xlarge, p4d.24xlarge). The instance family determines the GPU model, if any. The accelerator block below doesn't change the hardware, but whether it's set decides whether the group is provisioned as a GPU or a CPU pool. minLength: 1 diskSizeGb: type: integer default: 100 minimum: 10 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node group. Provisioning input only: the scheduler matches against spec.devices, not this block. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU group: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type (e.g. nvidia-a10g, nvidia-h100). Informational - reported on the consuming InferenceCluster's status. minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 aks: type: object required: [vmSize] properties: vmSize: type: string description: >- Azure VM size (e.g. Standard_NC24ads_A100_v4, Standard_ND96isr_H100_v5). The VM size determines the GPU model and count, if any. The accelerator block below doesn't change the hardware, but whether it's set decides whether the pool is provisioned as a GPU or a CPU pool. minLength: 1 diskSizeGb: type: integer default: 100 minimum: 10 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node pool. Provisioning input only: the scheduler matches against spec.devices, not this block. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU pool: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type (e.g. nvidia-a100, nvidia-h100). Informational - reported on the consuming InferenceCluster's status. minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 nebius: type: object required: [platform, preset] properties: platform: type: string description: >- Nebius compute platform (e.g. gpu-h100-sxm, gpu-l40s-a). Together with preset this determines the GPU model and count, if any. The accelerator block below doesn't change the hardware, but whether it's set decides whether the group is provisioned as a GPU or a CPU pool. minLength: 1 maxLength: 63 preset: type: string description: >- Resource preset within the platform (e.g. 8gpu-128vcpu-1600gb). Determines the GPU, vCPU, and memory shape of each node. minLength: 1 maxLength: 63 diskSizeGb: type: integer default: 100 minimum: 10 driversPreset: type: string default: cuda13.0 description: >- NVIDIA driver stack mk8s preinstalls on the pool's nodes (e.g. cuda12.4, cuda13.0). Valid values depend on the platform and Kubernetes version, and mk8s validates them, so new presets work without a Modelplane update. Defaults to the only preset mk8s implements on the default Kubernetes version; older presets remain for older versions. minLength: 1 maxLength: 63 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node group. Provisioning input only: the scheduler matches against spec.devices, not this block. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU group: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type (e.g. nvidia-h100, nvidia-l40s). Informational - reported on the consuming InferenceCluster's status. minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 vultr: type: object required: [plan] properties: plan: type: string description: >- Vultr plan ID (e.g. vcg-l40s-16c-180g-48vram). The plan determines the GPU model and count, if any. The accelerator block below doesn't change the hardware, but whether it's set decides whether the pool is provisioned as a GPU or a CPU pool. minLength: 1 maxLength: 63 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node pool. Provisioning input only: the scheduler matches against spec.devices, not this block. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU pool: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type (e.g. nvidia-l40s, nvidia-a100). Informational - reported on the consuming InferenceCluster's status. minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 civo: type: object required: [size] properties: size: type: string description: >- Civo Kubernetes size ID (e.g. an.g1.l40s.kube.x1). The size determines the GPU model and count, if any. The accelerator block below doesn't change the hardware, but whether it's set decides whether the pool is provisioned as a GPU or a CPU pool. minLength: 1 maxLength: 63 accelerator: type: object description: >- GPU accelerator to attach when provisioning the node pool. Provisioning input only: the scheduler matches against spec.devices, not this block. Set this when and only when a device claims a DRA driver other than dra.cpu; the API rejects a mismatch. Omitted, Modelplane provisions a CPU pool: a standard node image, the modelplane.ai/cpu label and taint, and no GPU stack. required: [type, count] properties: type: type: string description: >- GPU accelerator type (e.g. nvidia-l40s, nvidia-a100). Informational - reported on the consuming InferenceCluster's status. minLength: 1 maxLength: 63 count: type: integer minimum: 1 maximum: 16 devices: type: array description: >- Devices a node of this class has, following DRA's model (KEP-4381). Each entry describes one kind of device with a count, mirroring what a DRA driver publishes in a ResourceSlice (one entry per kind rather than per physical device). ModelDeployment.nodeSelector matches against these, and claim: DRA devices are emitted as requests in a DRA ResourceClaim when scheduling a worker to this pool. A scheduled worker pod is pinned to its pool with a nodeSelector on the modelplane.ai/pool node label. Modelplane-provisioned (EKS, GKE) pools carry this label automatically. On a BYO (Existing) cluster Modelplane doesn't provision the nodes, so the operator must label the pool's nodes modelplane.ai/pool= themselves, or worker pods for this class will stay Pending. minItems: 1 maxItems: 16 x-kubernetes-list-type: map x-kubernetes-list-map-keys: [name] items: type: object required: [name, driver] properties: name: type: string description: >- Name of this device within the class (e.g. gpu, nic). minLength: 1 maxLength: 63 claim: type: string description: >- How Modelplane treats this device. DRA emits it as a request in a ResourceClaim, so DRA binds a matching device to the pod at admission time; use it for hardware a real DRA driver exposes. Synthetic describes the device for fleet scheduling only and never claims it; use it for hardware that matters for placement but has no DRA driver yet, like an InfiniBand fabric. enum: [DRA, Synthetic] default: DRA driver: type: string description: >- DRA driver that owns this device (e.g. gpu.nvidia.com). Becomes the attribute/capacity domain a nodeSelector reads as device.attributes[""].. minLength: 1 maxLength: 253 deviceClassName: type: string description: >- Name of the cluster-scoped DRA DeviceClass to claim this device through. Required for claim: DRA devices; the DRA driver install creates the DeviceClass (e.g. gpu.nvidia.com). Ignored for Synthetic devices. minLength: 1 maxLength: 253 count: type: integer description: How many of this device a node has. default: 1 minimum: 1 maximum: 64 attributes: type: object description: >- DRA-style typed attributes for this device. Keys are bare names (e.g. architecture); the domain comes from the device's driver. Each value sets exactly one typed field. maxProperties: 32 additionalProperties: type: object properties: string: type: string maxLength: 253 version: type: string description: Semantic version (e.g. "9.0.0"). maxLength: 32 bool: type: boolean int: type: integer format: int64 x-kubernetes-validations: - rule: "[has(self.string), has(self.version), has(self.bool), has(self.int)].filter(x, x).size() == 1" message: "exactly one of string, version, bool, int must be set" capacity: type: object description: >- DRA-style capacity quantities for this device. Keys are bare names (e.g. memory); values are Kubernetes Quantities. maxProperties: 32 additionalProperties: type: object required: [value] properties: value: type: string description: A Kubernetes Quantity (e.g. "141Gi"). minLength: 1 maxLength: 32 x-kubernetes-validations: - rule: "self.claim != 'DRA' || has(self.deviceClassName)" message: "deviceClassName is required when claim is DRA" status: type: object properties: conditions: type: array items: type: object ```