Until 2yrs ago, I had an enormous oak tree growing on the land opposite my home. Each year I found sprouted acorns which took root anywhere in my garden, which I had to pull up or I would be living in a forest. About 5yrs ago, I did transplant one in the field opposite me which is now a beautiful tree about 8ft tall.
My one bit of advice is that it should be removed as soon as possible as the roots go down a long way and are not easy to get out one established. I think you will have a problem in moving it away from the bay without doing that tree some damage if left for too long. If you can get it out without damaging the root, you should be able to transplant it to wherever wanted without a problem.
The one that I put into the field needed a bit of shaping when about 3ft tall to give it a nice straight trunk with a good shape at the top.
I am now a widow and live with my memories.