Is your perception the same as mine? There are the extremes of Tetrachromacy & Colour Blindness that create unique abilities and problems, but for the rest of use it does raise questions of how we perceptive colour.
Looking at developmental neuro-science for a child I assume that the neural pathways in the visual cortex that handle colour perception are formed early on. Our brain makes the links between the perceived colour and the linguistic label for the colour. So it seams extremely unlikely that the pattern of neurons that formed the visual cortex are identical between people.
Tetrachromacy (tetra = four, chroma = color) in humans is a visual oddity which allows the perception of 100 millions colours (See this from IFLS). This is a boon for artists, like Concetta Antico who can see the colours to produce them in art. (BTW her website has a few pages on Tetrachromancy including a Preliminary report of her ability and the genetics involved) It appears to be X-chromosome-linked, so it will be more common in women, but still exceedingly rare.
On the flip-side there is colour blindness. I’ve used colour-blindness tests to head head off potential visually based problems with assessment. To ensure that it is a fair assessment, based on student skill and accounting for disability. There is Color Vision Testing Made Easy, which as a bunch of images online, and there is the EnChroma Color Blindness Test an interactive online test and a mobile app. Disclaimer: Neither of these should replace professional expertise.