According to the open source online encyclopedia, Wikipedia:
"John Quincy Adams swore on a book of law. Lyndon B. Johnson was sworn in on a Roman Catholic missal on Air Force One. Washington kissed the Bible afterwards, and subsequent presidents followed suit, up to and including Harry Truman, but Dwight D. Eisenhower broke that tradition by saying his own prayer instead of kissing the Bible."
In general, I think it is all quite silly, but we are a ritualistic people, and it still exists in the Constitution, so I guess these rituals matter.
Regardless, Nixon swearing on the Christian Bible did not stop him from ruining himself and the trust of the country with the Watergate Scandal, and I'm sure we could make a list a mile long of Presidents who have abandoned their oaths at one time or another.
Even the great Abraham Lincoln abandoned his oath when he suspended parts of the Constitution during the Civil War. However, on Lincoln's behalf, the oath doesn't say to unconditionally uphold the Constitution, it only says that he or she will try to "the best of their ability."
Specifically, it says in Article Two, Section One, Clause Eight, "I do solemnly swear (or affirm) that I will faithfully execute the Office of President of the United States, and will to the best of my ability, preserve, protect and defend the Constitution of the United States."
But, every President since Taft has sworn in on a Christian Bible, (if you count Johnson's Roman Catholic Missal in the absence of an available Bible on Air Force One after Kennedy's assassination).
Our most recent President, Barack Obama, was sworn in using Abe Lincoln's personal Christian Bible. It would be interesting to see if Romney swears in on the Christian Bible or the Book of Mormon if he were to be elected. As a matter of fact, I think that would be an interesting debate question for a moderator to ask.
I think if I were given the chance (not getting my hopes up) I would follow in John Quincy Adams footsteps and swear on a Book of Law or the Constitution itself. After all, your primary job as President of the United States is to uphold the Constitution and its principals.
If given the chance, as the newly elected President of The United States of America, which book or document would you place your hand to swear or affirm your oath?