with reference to the reference

It doesn't matter whether the colour you perceive to be blue is the same colour that another person perceives it to be. That is not what we are taught colour is all about. We give a name, lets say "green" to this particular colour that is there. As long as we can differentiate the green from the blue and other colours, and everyone else understands that the colour in question has the name "green" tagged to it, the purpose of having colours has been achieved. In a colourblind world, the ability to differentiate the hypothetical green from another colour is lost, thus the "green" may actually be another differnet colour. But thats on a case by case basis and it cannot be concluded that our eyes play tricks on us, but it is that they are unableto preceive some things as the majority does. Same goes for hearing.