First off a disclaimer; I don't know shit. Maybe, everyone else is right and I'm wrong. Could be. And what I'm offering is only one possible explanation of many. But, it seems to me, given the context, it has some merit.
To understand where I am coming from I need to explain one of the fundamental goals of mysticism. All traditions, Christian and otherwise have their mystics; for Christians it's people like Augustine, Theresa of Avila, Meister Eckhart, and even C.S. Lewis. Muslims have the Sufis like Rumi. Buddhist have ... well Buddha ... but others as well. etc. The thing to understand about all these mystics is that their ultimate goal - the reason they are called mystics - is that they are so in tune with God that there is no distinction between them and God. Here are a couple of Sufi quotes to give you the idea;
The Beloved gives us the water of life,
which cures every illness.
In the Beloved's rose garden of Oneness
no thorns survive.
I have heard it said that there is a window
between one heart to another.
But what supports the window
if walls have ceased to exist?
Rumi
Remove me from myself ,
so that all that remains is you, Beloved.
Take my life so that I can stand in your Presence.
Let all that remains be you.
Hallaj
Nothing exists but you, Beloved.
You are my speech. You are the silence of my mind.
You sleep with me. You walk the path with me.
There is nowhere I can go where you are not.
I have disappeared. Only you remain.
Bulleh Shah
Back to John 14. The key section here is the one where Jesus says that in his father's house are many room, and he goes to prepare a place for [his disciples].
If you start reading this section at John 13 and keep reading to the end of John 17 you get a better idea of what this section is all about. There are two things that Jesus keeps harping about throughout these 4 chapters; One is that his disciples need to follow his example, and the other is that he and the Father are one; if you've seen Him you've seen the Father and vice versa. There are also some mentions of the Holy Spirit coming after he is gone so they know what to do when he's not there.
But it isn't about going to heaven.
To try and get a little clarity on John 14:2, I think you need to jump all the way to John 17:24-26
Father, I desire that they also, whom you have given me, may be with me where I am, to see my glory that you have given me because you loved me before the foundation of the world. O righteous Father, even though the world does not know you, I know you, and these know that you have sent me. I made known to them your name, and I will continue to make it known, that the love with which you have loved me may be in them, and I in them.
In John 14:2, and throughout these 4 chapters, I think Jesus is speaking as a mystic. He is saying, in parabolic language, that as he and the Father are one, so the disciples are to be as well. That he wants them in the same place (relationally speaking) as he is where there is no distinction between God, or Jesus, or them. When he says to them; "In my father's house are many rooms ... I go to prepare a place for you" he is saying come live with me in complete union with the Beloved where you disappear and only the Father remains; where not only the windows between you and him disappear, but also the walls.