Stress comes with the dream. Anxiety, self doubt, and over thinking are going to happen. It may freeze you for a while, but never let it stop you. Just keep back into your worlds and let your characters keep talking, they know what to do.
I don't know if I saw myself doing it, but I wanted it. I wrote Dean Koontz in third or fourth grade asking, him how I could be like him and have his job. It was always there, the "someday" idea. Then after having our fourth child my husband asked me what I wanted to do for me. I said, "Write" full time. He went out and bought me a laptop and said, "Get to work." It's what I've been doing ever since.
I've always written. There never was a time that I wasn't coming up with worlds in my head and living in them, talking about them, or writing them. Its just who I am.