o .i@sddlZddlmZddlmZddlmZdejdejdejdejd ejd eee d ejfd d Z d eee de dej d ejfddZ dS)N)apply_penalties)is_pin_memory_available)make_tensor_with_padlogitsprompt_token_idspresence_penaltiesfrequency_penaltiesrepetition_penaltiesoutput_token_idsreturnc Cs:|j\}}t|||j}||dk|t||||||S)zM Applies presence, frequency and repetition penalties to the logits. )shape_convert_to_tensorsdevice masked_fill_r) rrrrr r _ vocab_sizeoutput_tokens_trY/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/v1/sample/ops/penalties.pyapply_all_penalties s rrrcCs$t||dtjtd}|j|ddS)z@ Convert the different list data structures to tensors. cpu)padrdtype pin_memoryT) non_blocking)rtorchint64rto)r rroutput_tokens_tensorrrrr*s r) r vllm.model_executor.layers.utilsrvllm.utils.platform_utilsrvllm.utils.torch_utilsrTensorlistintrrrrrrrs8