Thinking of quitting speech therapy for my 2 1/2 year old

Andy has had to miss the last few speech therapy appointments due to being sick, or having terrible temper tantrums, to where I don't see the point in driving him 30 minutes to speech therapy. He's been in speech therapy for 3 or 4 months now, and is really going NOWHERE. Sadly. I was so excited that it would help him, and it just isn't. He will say "mmm" to the therapist to continue play, and that is IT. It's just so maddening, I don't even want to do it anymore. I just want to take a break from it, and maybe return in the summer. It feels like such a waste of our time. A 20 year experienced speech therapist once told me that she wouldn't even bother doing any speech therapy until Andy was three, and his language base was bigger.

He currently says, mama, dada, and up. He will use the sign "eat" and claps for more. He will also say "mmm", if you ask him to say that to get more food, or a toy. That's about it.