Inside ARK-ASR-3B’s Whisper and Qwen Architecture
Overview
ARK-ASR-3B is a 3-billion-parameter multilingual automatic speech recognition model built by Audio8 that combines a Whisper-style audio encoder with an MLP adapter and a Qwen decoder. The model operates at 16 kHz sampling rate and achieves state-of-the-art performance on the Hugging Face Open ASR Leaderboard English short-form benchmark with an average word error rate of 5.04% and a real-time factor (RTFx) of 490.98. The architecture uses custom arkasr remote code to inject audio representations into the decoder by replacing audio placeholder token embeddings before transcript generation. The model supports 19 languages including Chinese, English, German, Japanese, French, Korean, Spanish, Polish, Italian, Romanian, Hungarian, Czech, Dutch, Finnish, Croatian, Slovak, Slovene, Estonian, and Lithuanian. It is distributed in safetensors checkpoint format and requires loading with trust_remote_code=True in the Hugging Face Transformers library. The accompanying training, inference, and evaluation code is open-source and available on GitHub, along with vLLM serving infrastructure for...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
