Last week I got 2 ill reviews removed from my add-on.
And I noticed that the review count didn’t drop by 2 but only by 1.
Having a further look it seems that I should only have 35 reviews but the count is at 39.
There is bug 592466 and bug 1198436 already, not quite your scenario however. I suspect that the count is only being updated when a new review is added.
I don’t think so, because that would mean my count wouldn’t be off by more than 2.
It’s a bit worrying if this bug does affect the overall rating.
It would mean that you could potentially bombard an add-on with low or high ratings, and thus manipulate the system through the ratings that secretly stick even though being removed.