Yeah definitely switch over to using ggerganov's whisper implementation, I use it in a little home brewed python app on my M1 for handling speech transcripts. The base EN model chews through minutes of audio in seconds, it's insanely fast.