I Built a Tiny GPT That Speaks Sanskrit in a Weekend — Here’s What Broke

A from-scratch transformer, ~250 lines of Python, and one Unicode lesson that humbled me.

GitHub repository link

I’ve understood transformers “at a high level” for a while now. Attention, embeddings, the usual diagram with the arrows. But I’d never built one. And there’s a specific kind of knowledge you only get from typing out the code yourself and watching it break.

So, I gave myself a weekend and a constraint: build a GPT small enough to understand completely, but on a language that would actually fight back — Sanskrit. I have a pile of Devanagari text and an NVIDIA DGX Spark sitting on my desk, so why not?

The Goal: Predict the next syllable.

A GPT does exactly one thing: given some text, predict what comes next.Everything else — chat, translation, reasoning — is built on top of that single reflex. So, my whole model is just: feed it...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE