o €wÖiº ã@s"ddlmZddlmZddlmZddlmZmZm Z m Z m Z ddl m Z mZmZmZmZmZddlmZe e¡ZGd d „d eƒZGd d „d eƒZGd d„deƒZGdd„deƒZGdd„de ƒZGdd„de ƒZGdd„deƒZGdd„de ƒZGdd„deƒZ Gdd„deƒZ!gd¢Z"dS)é)ÚCausalLMOutputWithPast)ÚUnpack)Úloggingé)ÚDeepseekV3DecoderLayerÚ DeepseekV3MLPÚ DeepseekV3MoEÚDeepseekV3PreTrainedModelÚDeepseekV3TopkRouter)ÚKwargsForCausalLMÚQwen3AttentionÚQwen3ForCausalLMÚ Qwen3ModelÚ Qwen3RMSNormÚQwen3RotaryEmbeddingé)Ú Dots1Configc@ó eZdZdS)Ú Dots1RMSNormN©Ú__name__Ú __module__Ú __qualname__©rrúd/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/transformers/models/dots1/modular_dots1.pyr'órc@r)ÚDots1RotaryEmbeddingNrrrrrr+rrc@r)ÚDots1AttentionNrrrrrr/rrc@r)ÚDots1MLPNrrrrrr3rrc@r)ÚDots1MoENrrrrrr7rrc@r)ÚDots1TopkRouterNrrrrrr ;rr cs&eZdZdedef‡fdd„ Z‡ZS)ÚDots1DecoderLayerÚconfigÚ layer_idxcstƒ ¡|j||_dS)N)ÚsuperÚ__init__Ú layer_typesÚattention_type)Úselfr"r#©Ú __class__rrr%@s zDots1DecoderLayer.__init__)rrrrÚintr%Ú __classcell__rrr)rr!?sr!c@r)ÚDots1PreTrainedModelNrrrrrr-Err-c@r)Ú Dots1ModelNrrrrrr.Irr.cs*eZdZdeedef‡fdd„ Z‡ZS)ÚDots1ForCausalLMÚ super_kwargsÚreturnc stƒjdi|¤ŽS)a~ labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Dots1ForCausalLM >>> model = Dots1ForCausalLM.from_pretrained("rednote-hilab/dots1.llm1.inst") >>> tokenizer = AutoTokenizer.from_pretrained("rednote-hilab/dots1.llm1.inst") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```Nr)r$Úforward)r(r0r)rrr2NszDots1ForCausalLM.forward)rrrrr rr2r,rrr)rr/Ms þýr/)r-r.r/N)#Úmodeling_outputsrÚprocessing_utilsrÚutilsrÚ deepseek_v3.modeling_deepseek_v3rrrr r Úqwen3.modeling_qwen3r r r rrrÚconfiguration_dots1rÚ get_loggerrÚloggerrrrrrr r!r-r.r/Ú__all__rrrrÚs$