I Made My iPhone's Neural Engine and GPU Run Inference Together as an Experiment. It Got Slower.
This story started from a very simple idea. Maybe too simple, I understand this now.
The modern iPhone has three different processors inside, if we can call them processors. There is a normal CPU, there is a GPU, and there is also ANE, the Apple Neural Engine, a special chip only for neural networks. When you run a Core ML model, the framework takes one of them and works only on it. The other two are just sitting and doing nothing. And I always feel a bit bad when I see this. Expensive hardware, and half of it is sleeping.
I am a PhD student, I research how to run neural networks on mobile devices, and I also work part-time on an iOS app. So this question is not theoretical for me. When you process a video stream and detect objects frame after frame, you want more FPS, always more;...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
