Wrapper around the stdlib tokenize which roundtrips.
This Python library is a wrapper around tokenize
from the Python standard library. It provides two additional tokens ESCAPED_NL
and UNIMPORTANT_WS
, and a Token
data type. Use src_to_tokens
and tokens_to_src
to roundtrip.
Linter | Message | Location |
---|---|---|
No lint warnings ✓ |