Live page ยท Day archive

ttok 1.0

Read the original at Simon Willison

Summary

The developer releases ttok 1.0, which defaults to the GPT-5 tokenizer after confirming that GPT-6 models share the same 44,794-token vocabulary.

Carried by: Simon Willison. First seen: .