X sounds like Z. When you say X-Xavier you are saying the letter of the alphabet versus sounding it out phonically. Just like K and C have similar sounds. If there was a hyphen after the X then it becomes EX. For example: x-ray

No one says:

ex- Xerox for Xerox

ex- xylophone for xylophone

ex-xanadu for xanadu

Please see the dictionary to see and hear the proper pronunciation of the name Xavier. You can also hear the proper pronunciation on dictionary.com.