o }oi @s<ddlmZmZmZmZddZddZddZdd Zd S) )Gemma3Config1BGemma3Config4BGemma3Config12BGemma3Config27BcCs0t}|jdks J|jdksJ|jdksJ|jdksJ|jdks&J|jdks-J|jdks4J|jdks;J|j d ksBJ|j d ksIJ|j d ksPJ|j d usWJ|j d ks^J|jd useJ|jdkslJ|jdussJ|jdkszJ|jdksJ|jd usJ|jdusJ|jdksJdS)Niiii'i@Bg?iRMSNormTư>ropeFi)r num_layers hidden_sizenum_attention_headsnum_query_groups kv_channelsffn_hidden_size window_size rotary_baserope_scaling_factor seq_length normalizationlayernorm_zero_centered_gammalayernorm_epsilongated_linear_unitposition_embedding_typeadd_bias_linearhidden_dropoutattention_dropout#share_embeddings_and_output_weightsis_vision_language vocab_sizeconfigr&_/home/ubuntu/.local/lib/python3.10/site-packages/tests/collections/llm/gpt/model/test_gemma3.pytest_gemma3_1b_config,r(cC"t}|jdks J|jdksJ|jdksJ|jdksJ|jdks&J|jdks-J|jdks4J|jdks;J|j d ksBJ|j d ksIJ|j d ksPJ|j d usWJ|j d ks^J|jd useJ|jdkslJ|jdussJ|jdkszJ|jdksJ|jd usJ|jd usJdS)N"i rr i(r @r Tr r Fr)rrrrrrrrrrrrrrrrrrr r!r"r$r&r&r'test_gemma3_4b_config+*r0cCr*)N0ir,r i<r-r r.r/r Tr r Fr)rrrrrrrrrrrrrrrrrrr r!r"r$r&r&r'test_gemma3_12b_configCr1r4cCs0t}|jdks J|jdksJ|jdksJ|jdksJ|jdks&J|jdks-J|jdks4J|jdks;J|j d ksBJ|j d ksIJ|j d ksPJ|j d ksWJ|j d us^J|jdkseJ|jd uslJ|jdkssJ|jduszJ|jdksJ|jdksJ|jd usJ|jd usJdS)N>i r3gP6?iTr-r r.r/r Tr r Fr)rrrrrr softmax_scalerrrrrrrrrrrrr r!r"r$r&r&r'test_gemma3_27b_config[r)r9N) %nemo.collections.llm.gpt.model.gemma3rrrrr(r0r4r9r&r&r&r's