tokenizers releases
PyPI package, released from huggingface/tokenizers on GitHub · No. 154 of the most used PyPI packages in the paper's catalog
Latest release the paper recorded: tokenizers v0.23.3, Oct 9, 2026. Its newest GitHub release when last read: Oct 9, 2026.
Latest releases
Newest first, each linking to its notes on GitHub; the excerpt is the notes' own opening, where they have text.
- tokenizers v0.23.3
From the release notesRelease v0.23.3 Python package release, other packages are unchanged :rotating light: Fixs a breaking change introduced in v0.23.1 What's changed Patch for the python package with three fixes: Loosen the huggingface hub…