On this page:http://readingfaithfully.wordpress.com/ ... talibrary/
there is a PDF you can download with a bibliography for building a solid English sutta library. The selection puts a priority on accuracy, readability, and availability. Those texts marked with a star are recommended to start with. As you will see, they include the two sutta books already recommended on this thread. You will see other pages on that site that go into more detail about good books to start with.
One of the purposes of this PDF is for people like the OP who want to get the books but may not have a sense of where to begin. If it is helpful or you have suggestions for improvements that would be welcome.
As far as Pali language texts, most are now available for free on the Kindle. And there are other resources on the computer to get the texts so having the Pali print books is not so urgent.