Falcon_OPT / AutoAWQ_kernels /awq_ext /vllm /topk_softmax_kernels.h
BechirTrabelsi1's picture
Training in progress, step 500
25b4ce2 verified
raw
history blame
196 Bytes
#pragma once
#include <torch/extension.h>
void topk_softmax(
torch::Tensor& topk_weights,
torch::Tensor& topk_indices,
torch::Tensor& token_expert_indices,
torch::Tensor& gating_output);