o wi-@sUddlmZddlZddlmZmZmZddlmZm Z m Z ddl m Z ddl mZddlmZer9ddlmZd#d dZd$ddZgaded<gaded<d%ddZd%ddZd&ddZd&ddZd'd d!Zed"krqedSdS)() annotationsN)IO TYPE_CHECKINGAny)AttributeProtodefsload)collect_snippets)load_model_tests)Runner)SequenceschemasSequence[defs.OpSchema]returnboolcCstdd|DS)Ncss|]}|jdkVqdS ai.onnx.mlN)domain.0sr\/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/onnx/backend/test/stat_coverage.py szis_ml..)any)r rrris_mlsrfIO[Any]mlNonecCsP|d|r |dn|d|d|d|d|ddS)Nz# Test Coverage Reportz (ONNX-ML Operators) z (ONNX Core Operators) z ## Outlines z,* [Node Test Coverage](#node-test-coverage) z.* [Model Test Coverage](#model-test-coverage) z2* [Overall Test Coverage](#overall-test-coverage) write)rrrrr gen_outliness      r"z Sequence[str]common_coveredexperimental_coveredc sthdttfdd|Datfdd|D}tfdd|D}tfdd|Datfdd|D}tfdd|D}ttt|t|}ttt|t|}|d |d |r|d ttd |d ttt|dddt|d n|d|r|dtt|ttt|dt|n|dgd} t|t|g} | D]} |d| ddd| dd dddq|dt | | D]o\} } |d| d| D]Z} |d| | vr>|dt| d t| D]!\}}|d!|d"|d#|d$|d%|d&qn| vrI|d'n|d(|d)q|d*qdS)+N> Multinomial RandomNormal RandomUniformRandomNormalLikeRandomUniformLikec3>|]}|jvr|jtjjjkr|jdkkr|jVqdSrname support_levelrOpSchema SupportTypeCOMMONrrr node_testsrrr7 z)gen_node_test_coverage..c3>|]}|jvr|jtjjjkr|jdkkr|jVqdSrr+rr1rrr>r3c3|] }|vr|VqdSNrrr, generatorsrrrEsc3r*rr,r-rr.r/ EXPERIMENTALrrr1rrrFr3c3r4rr:rr1rrrMr3c3r5r6rr7r8rrrTsz# Node Test Coverage z ## Summary zNode tests have covered /z (dz.2fz%, z) generators excluded) common operators. z6Node tests have covered 0/0 (N/A) common operators. zYNode tests have covered {}/{} ({:.2f}%, {} generators excluded) experimental operators. z;Node tests have covered 0/0 (N/A) experimental operators. )z!💚Covered Common Operatorsz"💔No Cover Common Operatorsz'💚Covered Experimental Operatorsz(💔No Cover Experimental Operatorsz* [ z](# -z)  ## z### z There are z" test cases, listed as following:
z z ```python z ```
z (random generator operator) z (call for test cases) z z
) setr sortedr#r$lenr!floatformatlowerreplacezip)r rrcommon_no_covercommon_generatorexperimental_no_coverexperimental_generator num_commonnum_experimentaltitles all_liststlrsummarycoder)r9rr2rgen_node_test_coverage(s      6        rZcCs4|di}|D]}|||j<q i}g}tddD]>}|jdrItjtjtj t ddd|j}tj |sCt d|d| |qt|} | tj| dq|d } |D]&} t| } |r~d } | jD] }|jd krxd } qo| s|qad } |d | jjd d}| jjD]m}|jtvs|jtvr|d7}|jD]W}|j|vri||j<|j||jvrg||j|j<|jtjkr|j||j|jvr||j|j |jq|jtjkr|j||j|jvr||j|j |jq|jtj kr|j!||j|jvr||j|j |j!q|jtj"kr=|j#||j|jvr<||j|j |j#q|jtj$kr]|j%||j|jvr\||j|j |j%q|jtj&kr}|j'||j|jvr|||j|j |j'q|jtj(kr|j)||j|jvr||j|j |j)q|jtj*kr|j+||j|jvr||j|j |j+q|jtj,kr|j-||j|jvr||j|j |j-q|jtj.kr|j/||j|jvr||j|j |j/qq|d | jjd|dt0| jjddt1|t1t0| jjd |d|dt2|D]R}|d|d|dt0||dt0||j3dt2||j3D]%}|||vrr||dt0|||d qV||dqV|dq0|dqa| s|r|ddSdSdS)Nz# Model Test Coverage real)kindzonnx/backend/test/data/light/z..zUnable to find model .z model.onnxFrTrBrArz has z nodes. Of these, z are covered by node tests (gY@z%) rCznodes rDz: z out of z attributes covered z: 0 rEz z+No model tests present for selected domain )4r!r,r url startswithospathnormpathjoindirname__file__existsFileNotFoundErrorappendr prepare_model_datasortr opset_importrgraphnodeop_typer#r$ attributetyperFLOATrINTiSTRINGrTENSORrVGRAPHgFLOATSfloatsINTSintsSTRINGSstringsTENSORStensorsGRAPHSgraphsrHrIrG attributes)r rr schema_dictschemaattrs model_pathsrt model_name model_dir model_written model_pb_pathmodel ml_presentopset num_coveredrnattroprprrrgen_model_test_coverages                   (&  rcCs|d|ddS)Nz# Overall Test Coverage z## To be filled. r rrrrgen_overall_test_coverages rcCs|ddS)Nz. r rrrrgen_spdxsrc Cs8tjtjtjtjtjt}tj|d}t}t|}tj|d}t |dddd!}t |t |dt ||dt ||dt|Wdn1sXwY|rtj|d}t |dddd"}t |t |d t ||d t ||d t|WddS1swYdSdS) NdocszTestCoverage.mdzw+zutf-8)newlineencodingFzTestCoverage-ml.mdT)rarbrerealpathrfrdrget_all_schemasropenrr"rZrr)base_dirdocs_dirr has_mlfnamerrrrmain s0"        "r__main__)r rrr)rrrrrr)r rrrrrrr)rrrr)rr) __future__rratypingrrronnxrrronnx.backend.test.caser onnx.backend.test.loaderr onnx.backend.test.runnerr collections.abcr rr"r#__annotations__r$rZrrrr__name__rrrrs*         p i