Embedding pooling strategy in Aspose.llm for .NET — how per-token embeddings are reduced to a single vector....style Mean Example using Aspose.LLM.Abstractions.Models ; var...
GPU device index in Aspose.llm for .NET — selects which GPU holds the model when SplitMode is None.... Example using Aspose.LLM.Abstractions.Parameters ;...
RoPE frequency scaling factor in Aspose.llm for .NET — multiplicative factor applied to RoPE frequencies; null or 0 uses the model default.... Example using Aspose.LLM.Abstractions.Models ; var...
Flash attention mode in Aspose.llm for .NET — Auto lets the runtime pick, Enabled forces flash attention, Disabled turns it off....automatically when Auto Example using Aspose.LLM.Abstractions.Models ; var...
Force a specific acceleration backend in Aspose.llm for .NET — null auto-detects; set to CUDA, HIP, Metal, Vulkan, or a CPU variant to pin....AVX2 vs AVX512 Example using Aspose.LLM.Abstractions.Acceleration...
KV cache dtype for the V (values) tensor in Aspose.llm for .NET — tolerates more aggressive quantization than TypeK; Q8_0 is a safe default for tight memory.... Example using Aspose.LLM.Abstractions.Models ; var...
Policy for KV cache trimming when context fills up in Aspose.llm for .NET — five strategies ranging from full reset to preserving the first user turn....KeepSystemPromptAndH Example using Aspose.LLM.Abstractions.Models ; var...
Pre-seeded conversation history for new chat sessions in Aspose.llm for .NET — inject user/assistant turns for few-shot priming....Application-specific Example using Aspose.LLM.Abstractions.Models ; var...
KV cache dtype for the K (keys) tensor in Aspose.llm for .NET — F16 default, quantized types save memory at long contexts....quality drop) Example using Aspose.LLM.Abstractions.Models ; var...
Original trained context length for YaRN scaling in Aspose.llm for .NET — set this to the model's native context when extending beyond it....training context Example using Aspose.LLM.Abstractions.Models ; var...