o  ¢iã@s´ddlZej ddg¡dd„ƒZej ddg¡dd„ƒZej dd g¡d d „ƒZej dd g¡d d„ƒZej ddg¡dd„ƒZej ddg¡dd„ƒZej ddg¡dd„ƒZ dS)éNÚtextz lorem ipsumcCó||ƒ}t|ƒdks J‚dS)Né©Úlen©Ú tokenizerrÚtokens©r úY/home/ubuntu/.local/lib/python3.10/site-packages/spacy/tests/tokenizer/test_whitespace.pyÚ"test_tokenizer_splits_single_spaceór z lorem ipsumcCó.||ƒ}t|ƒdks J‚|djdksJ‚dS)Nééú ©rrrr r r Ú"test_tokenizer_splits_double_space órz lorem ipsum cCs"||ƒ}t|jƒt|ƒksJ‚dS)N)ÚreprÚ text_with_wsrr r r Ú)test_tokenizer_handles_double_trailing_wssrz lorem ipsumcCr)NrrÚ rrr r r Útest_tokenizer_splits_newlinerrz lorem ipsumcCr©Nrrrr r r Ú#test_tokenizer_splits_newline_spacer rz lorem ipsumcCrrrrr r r Ú*test_tokenizer_splits_newline_double_space$r rz lorem ipsumcCrrrrr r r Ú(test_tokenizer_splits_newline_space_wrap*r r) ÚpytestÚmarkÚ parametrizer rrrrrrr r r r Ús