o i@slddlmZddlZddlmZddlmZddlmZddl m Z ddl m Z ee ZGd d d e ZdS) )IterableN)SymIntstatically_known_true) init_logger)is_func)VllmInductorPassc@speZdZdZejdejjddfddZ de e Bde e Bde fd d Z d ee e Bd ee e Bde fd dZdS)NoOpEliminationPassa_ This is an inductor pass that removes redundant reshape/slice operations. It is required for RMSNorm-quant fusion to work properly. That's because apply_fp8_linear adds a reshape, which is redundant in the 2D-case. Additionally, torch internal no-op elimination pass does not handle certain slice variants. Cases handled: 1. A chain of reshapes is equivalent to the last reshape called on the base tensor (input of the first reshape). 2. A reshape that produces the shape of the input is redundant 3. A slice that produces the shape of the input is redundant Example graph 1: mul_1: "f16[s0, 4096]" = ... view_1: "f16[s0, 128, 32]" = torch.reshape(mul_1, [-1, 128, 32]) view_2: "f16[s0, 4096]" = torch.reshape(view_2, [-1, 4096]) view_3: "f16[s0, 128, 32]" = torch.reshape(view_3, [-1, 128, 32]) Can be replaced with: mul_1: "f16[s0, 4096]" = ... view_3: "f16[s0, 128, 32]" = ... Example graph 2: getitem_1: "f16[s0, 4096]" = ... view_1: "f16[s0, 4096]" = torch.reshape(getitem_1, [-1, 4096]) at = auto_functionalized(static_scaled_fp8_quant, input = view_1, ...) out: "f8e4m3fn[s0, 4096]" = at[1] Can be replaced with: getitem_1: "f16[s0, 4096]" = ... at = auto_functionalized(static_scaled_fp8_quant, input = getitem_1, ...) out: "f8e4m3fn[s0, 4096]" = at[1] Example graph 3: arg0: "s0" = SymInt(s0) scaled_mm: "f16[s0, 4096]" = ... slice_1: "f16[s0, 4096]" = torch.slice(scaled_mm, -1, 0, arg0) at = auto_functionalized(fused_add_rms_norm, input = slice_1, ...) out: "f16[s0, 4096]" = torch.slice_scatter(scaled_mm, at[1], 0, 0, arg0) Can be replaced with: arg0: "s0" = SymInt(s0) scaled_mm: "f16[s0, 4096]" = ... at = auto_functionalized(fused_add_rms_norm, input = scaled_mm, ...) out: "f16[s0, 4096]" = at[1] graphreturnNcCsZd}|jD]}t|tjjjjr7|jd}t|tjjjjr7|d|jdt |j dkr7| ||d7}t|tjjjjsIt|tjjj j ro|jd}|jdj}|jdj}|||rn||| ||d7}qt|tjjjjr|jdd\}}} } } |jdj} |jdj} || | r||| ||d7}qtd|dS)Nrvalz$Removed %s no-op reshapes and slices)nodesrtorchopsatenreshapedefaultargs update_arglenusers erase_nodesliceTensormetashapeall_dims_equivalentreplace_all_uses_with slice_scatterloggerdebug)selfr countnodeinput input_shape output_shapebaseview dim_indexstartend base_shape view_shaper1f/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/compilation/passes/utility/noop_elimination.py__call__Cs<               zNoOpEliminationPass.__call__dimi_dimcCs t||kS)a This function checks if two dimensions are equivalent. :param dim: The dimension arg to reshape/slice :param i_dim: The corresponding dimension in the input tensor :return: Are the dimensions equivalent? There are two cases in which the dimensions are equivalent: 1. The dimensions are equal (both integers) 2. The dimensions both correspond to the same SymInt r)r$r4r5r1r1r2dims_equivalentls z#NoOpEliminationPass.dims_equivalentdimsi_dimscs@t|}t|}t|t|krdStfddt||DS)NFc3s |] \}}||VqdS)N)r6).0si_sr$r1r2 sz:NoOpEliminationPass.all_dims_equivalent..)listrallzip)r$r7r8dims_i_dims_r1r<r2rzs z'NoOpEliminationPass.all_dims_equivalent)__name__ __module__ __qualname____doc__r time_and_logrfxGraphr3intrboolr6rrr1r1r1r2r s0(  r )collections.abcrtorch.fxrr%torch.fx.experimental.symbolic_shapesr vllm.loggerrfx_utilsrvllm_inductor_passr rCr"r r1r1r1r2s