1989 Mac SE/30 Runs AI Transformer
Artificial intelligence has become synonymous with massive GPU clusters and data centers consuming enormous amounts of power. That can make the technology feel almost completely out of reach, as though training a neural network requires hardware that only the world's largest companies can afford. But Sean Lavigne's MacMind project shows that with some clever engineering, all sorts of hardware can get in on the action. He has shrunk a transformer model down to the point that it can run on a Macintosh SE/30 from 1989.
MacMind is a fully functional transformer neural network written entirely in HyperTalk, the scripting language behind Apple's HyperCard. The project includes token embeddings, positional encoding, scaled dot-product self-attention, cross-entropy loss, backpropagation, stochastic gradient descent, and a residual connection. Every line of the algorithm is written in HyperTalk.
Running the model on an SE/30 obviously places some limitations on it. The model contains just 1,216 parameters...
Copyright of this story solely belongs to hackster.io. To see the full text click HERE
