I share your enthusiasm for lemon scented gums, they really are a lovely tree. We have a couple here. I once sent a leaf to a friend who was overseas, with the instructions to crush it, hold it under the nose, and think of Australia. Apparently they nearly jumped on the next plane home...
However, I'd share everybody's reservations about them. They eventually get massive and then can be difficult and expensive to remove from small blocks. The roots and leaves can be a problem, and they can be a real fire risk, depending on where you are.
Many of our trees here can be limb droppers - either through wind or due to a process sometimes known as "hydraulicking". This can cause trees to drop limbs on a perfectly calm day. If I remember rightly, it works something like this. On a hot Australian day a big tree can be sucking up moisture at a pretty hefty rate. If the leaves sudddenly sense a temperature change (even a suddenly clouding over can reputedly do the trick) they can shut off the flow very rapidly. In extreme cases the stress of this can cause a branch to break - a bit like shutting your hose off quickly and having it pop off at the tap end.
Fire can be a big consideration too, although it depends where you live of course. When planning our tree balance round the house I studied material about the Ash Wednesday fires, and also the recent Canberra fires where several suburban houses were lost. Apparently the Canberra houses weren't lost through direct attack from the main fire front. They fell foul of airborne burning material that fell on them and caused mostly smallish fires that then developed into big ones.
Having a tree near a house is great for shade and wildife, but it simply provides a wick to light your house with if you have a fire. Friends of ours nearly lost their house in that way. Trees near the house caught fire, the burning leaves feel onto the roof and onto the air conditioner, which then collapsed into the roof, setting that on fire.
If you think you can cope with the side effects (as with anything you grow) then they shouldn't be a reason not to go ahead. I guess it's just good to know what the possible negatives are, and be forewarned.