Here's what I recommend...
Buddhism is sort of a "come and see" religion. In order to really get a feel for it, you need to attempt to practice it somewhat.
What the Buddha Taught by Walpola Rahula is the "what" of Buddhism, simply the best intro to Buddhism out there IMHO! Not just for Theravadins but for Mahayanists as well, by the way
Mindfulness in Plain English by Henepola Gunaratana ("Bhante G") is the best meditation manual for Vipassana to date (again, IMHO). That will give you the "how"
As far as holy books, check out The Dhammapada first. Buddhism does not have a "bible" as it were. The Pali Canon is several volumes and thousands upon thousands of pages long. A set would run you in the $1000s, definitely not a good idea for someone who's just starting out! However, there have been several attempts to create or condense major Buddhist teachings into a single volume like a "Bible" over the years, mainly for beginners like you (and me!) The Dhammapada has been championed as one of those all purpose Buddhist "Bibles", because it's short, to-the-point, and contains teachings on so many of the Buddha's main themes (generosity, virtue, heaven, Nibbana, etc.) Translations abound, but I would go for one that was translated by a Buddhist. Access to Insight has several available free online. The translation by Acharya Buddharakkhita is the one I am currently studying, and so far it's the best I've seen. If you like one you can follow the links to order/purchase it!
"Indeed, the Blessed One is worthy and rightly self-awakened, consummate in knowledge & conduct, well-gone, an expert with regard to the world, unexcelled as a trainer for those people fit to be tamed, the Teacher of divine & human beings, awakened, blessed." — AN 11.12