Write a GPU program that implements top-p (nucleus) sampling for LLM inference.
Top-p sampling is a text generation technique where you sample from the smallest set of tokens whose cumulative probability exceeds threshold p. This balances randomness and quality better than pure top-k or greedy sampling.
Given logits (unnormalized scores) from a language model:
solve function signature must remain unchangedInput: logits = [1.0, 2.0, 3.0, 0.5] p = 0.9 seed = 42 Output: sampled_token = 2 or 1 (tokens with highest probabilities, sampled randomly)
Input: logits = [10.0, 1.0, 1.0] p = 0.5 seed = 123 Output: sampled_token = 0 (single token dominates the probability mass)
vocab_size ≤ 50,000logits[i] ≤ 100.0p ≤ 1.0sampled_token < vocab_sizevocab_size = 50,000