TECH NEWS
I Built a Python Compressor Using OpenAI's Tokenizer. Here's What Happened
LLMs spent billions of dollars learning the best subword dictionary that has ever existed. Somewhere around 200,000 pieces of language, ranked by how much they compress. And I kept looking at that dictionary and thinking: nobody is allowed to just use that for compression? TokPress is a pure-Python lossless