Let's get this out of the way right off: World building the correct way is hard(tm). Insanely hard. Just writing this series is looking insanely hard.
That said, so is writing anything longer than a short story. Which is what's brought us to here: if you are writing fantasy that's not based in a familiar environment (like, oh, say, documented historical or current Earth), or a pre-developed universe (Arda, Dragonlance, Forgotton Realms, etc), and longer than a novella, you are choosing to build your own world.
World building isn't a bad thing to do. Just understand that you *need* to do more work in one form or another for your writing to hang together well. Some people can hold all of the details of their created world in their head. I prefer a separate document that I can refer to, since my memory doesn't serve well due to the volume of stuff that's on my mind at any one time. Documenting it for me solidifies the concepts, and sometimes will help me develop parallel ideas that are related to the concept I'm working with at the time.
No matter what tracking method you choose to do, be certain you can track a volume of minutiae that can (read: will probably) grow very large, depending on how much detail you want to put into your world. I'm personally of the opinion that "details matter" (for reasons I will touch on later), so usually when I'm working on something like world building, I usually wind up with a large volume of stuff that never makes it into anything I'm writing, but does influence my writing both directly and indirectly.
Firstly, a well thought out world makes your future writing MUCH easier.
You know the rules, and what will work and what is improbable. You won't stomp on yourself in the future with stupid mistakes like Hollywood does all the time when it comes to computer technology, which those of us in the tech industry can find insanely distracting, because "It makes no sense". Which brings me to my second point.
A well constructed world helps us prevent distractions to the reader.
Let me be perfectly clear: You *NEVER* want a reader to say "that doesn't make sense". EVER. Keep this in mind as you write. If someone reading one of your works gets to the point of saying "That doesn't make sense", you, as a communicator, have failed to communicate something very important, or have transgressed something either stated or preconceived in your writing. Your story loses credibility, and as entertainers (yes, if we are writing fiction, we are entertainers, as well as communicators), if we break the suspension of disbelief, the reader will lose interest.
Details matter. Don't think otherwise.
Even if you don't use the details in your writing directly, you have them to refer back to. Details bring your writing alive. Knowing the minutiae of your world down to the shape of the average blade of grass may be a bit on the excessive side, but if you can do that, I encourage it. An example of where details come in handy:
Say that gravity on your new world is 1/6th Earth normal. This is roughly normal gravity on the moon for a reference.
Knowing this, we let the reader in on a few things without ever telling them "the gravity is 1/6 Earth normal", like the average human(oid) is close to 8' tall and very lanky, because vertical growth is less inhibited. They could bound with large leaps due to their long legs and light weight due to the lack of gravity. Flora would be the same: tall and lanky. The planet could either be very small, or be large and have very low density. If the planet is very small, it would rotate slower than the regular 24 hour day as to not sling off/tear the atmosphere. Perhaps it is a moon of a Earth sized planet, which brings up a whole NEW line of difficulties that I won't get into here. If you *do* have your new world as a moon of a larger planet, you can use photos of "Earthrise" as reference for describing the rise of the parent planet on your world.
Your planet's gravity is a detail. Not a huge detail (I say, tongue firmly planted in cheek), but a detail that can shape your writing. Details matter.