are you sure you using the right wire combination. On all the head units i've wired the colours were green and green and black for one rear speaker. And purple and purple and black for the other. Then grey for one front and white for the other.
So that might be worth checking. Unless the colours your talking about are the ones from the car itself not the head unit cos then your probably right.

However if your using the standard speaker wiring and you have a pre 85 rolla then that is your problem. The stock system uses a common negative and there should be a warning about this somewhere in your head unit booklet that says don't do it or i will explode in your face!
Just run new wires the whole way and i bet it will fix it.
I didn't fully understand your post have you interchanged the speakers? Cos maybe one of them is just fooked