Scientists create AI that 'watches' videos by mimicking the brain

Imagine an artificial intelligence (AI) model that can watch and understand moving images with the subtlety of a human brain. Now, scientists at Scripps Research have made this a reality by creating MovieNet: an innovative AI that processes videos much like how our brains interpret real-life scenes as they unfold over time.

This brain-inspired AI model, detailed in a study published in the Proceedings of the National Academy of Sciences on November 19, 2024, can perceive moving scenes by simulating how neurons -- or brain cells -- make real-time sense of the world. Conventional AI excels at recognizing still images, but MovieNet introduces a method for machine-learning models to recognize complex, changing scenes -- a breakthrough that could transform fields from medical diagnostics to autonomous driving, where discerning subtle changes over time is crucial. MovieNet is also more accurate and environmentally sustainable than conventional AI.

"The brain doesn't just see still frames ...