The lifetime constraint in the enum is mandatory, the code won't compile without it.

The tokenization function itself doesn't have to explicitly specify lifetimes like it does in the blog post. It would also work without them and Rust would still ensure that you don't end up with a use-after-free.