OP - I observe that you are suffering from crushing anxiety in the interpersonal aspects of discussing your work. Perhaps you could take stock in your successes and see what others clearly see in you? You are clearly successful as a researcher - two book contracts do not come to someone with nothing to say!
I really don't know any academics who stay current outside their field. Fields are too big in themselves, and time is short. I basically am current only on what I'm writing about (and not always even then).
Thank you hegemony for this - The amount of information that we must manage to stay current in and around our fields has risen exponentially since 1990, and few really are "read up" exhaustively.
I also value my personal life and my relationship with nature, and I find that placing priority on eating well, breathing fresh air, and travel reinvigorate my work. Don't feel guilty doing these things - once you are rested and recharged you won't feel so stifled by the profession.
Conferences can be drudgery, or they can be convergences of like minded people sharing expansive thought, very stimulating.
True intellectuals do not posture, they think and speak their thoughts from a place of deep interest and sharing. Perhaps you have been around more insecure folk, or have been intimidated by others' seemingly complete knowledge?
Take some space, and own your successes. Academia should be a means to a life, not a life to a means.
Wishing you the best, ~lnw