Well I don't know how 'simple' of a library you are looking for, but XNA is a robust and fully featured library, with everything from 3D graphics and shaders, to audio, to networking, to data storage, with the added bonus of deploying to Windows, Xbox, and windows phone from the same code. There are even numerous free specialized engines built on top of XNA that you can download. And Microsoft has a bunch of sample projects with example code that is well commented. I would definitely recommend it as a starting place to build an understanding of game programming before moving to a more complex and powerful library.
On the more serious route, OGRE is a more professional free engine and can built built in a variety of languages (C++, Python, C#, etc.). It's harder to work with, but much more powerful.
I'm pretty sure we have a topic like this somewhere, let me look for it.