How and why have U.S. interests and involvement in the Middle East changed over time?

The United States has been involved in the Middle East since its founding. One of the first official actions of the US government was the Treaty of Tripoli in 1796, which established a ceasefire between the fledgling US Navy and the Barbary States, which included Morocco and other parts of North Africa. Before the treaty, the Barbary States had been hiring privateers called the Barbary Corsairs to raid shipping across the Mediterranean, including US shipping. The treaty put an end to that, in exchange for concessions from the US. Because the Barbary States were predominantly Muslim (as the Middle East remains today), the treaty also established explicitly that freedom of religion in the United States includes Islam. (Some folks don't seem to have gotten the memo...)

Still, US involvement in the Middle East was relatively minimal for most of the 19th and early 20th century. It reached a much higher level after the founding of Israel in 1948, and the establishment of Israel as a key US ally. The resulting displacement of Palestinians, as well as the fact that Israel is a Jewish state surrounded by Muslim states, has led to a great deal of conflict in the region, and the US has generally taken Israel's side in such conflicts, supplying enormous amounts of economic and military support.

More recently the US became again embroiled in the Middle East after OPEC, and especially Saudi Arabia, embargoed oil exports to the US in 1973, dramatically raising the price of oil in the US and triggering inflation and recession (you may have heard it called "the Nixon stagflation"; it is a good example of "cost-push inflation"). This crisis triggered calls for the US to divest from foreign oil, though US oil imports didn't actually fall significantly until the 2010s. Instead, the main US response seems to have been attempts to use diplomacy--and, if necessary, force--to secure a steady supply of oil from the Middle East. Many subsequent conflicts and proxy wars, including those with Iran and Iraq, were at least in part motivated by this desire to protect oil interests.

After the 9/11 attacks in 2001, as a kind of vicarious retaliation, US policy in the Middle East became even more aggressive, changing from simply trying to ensure a steady flow of oil to actively trying to unseat governments perceived as the enemy. On the one hand, some of these governments---such as the Taliban's in Afghanistan and Saddam Hussein's in Iraq---were in fact quite terrible, engaged in tyranny, oppression, and genocide; and thus it is hard to feel bad that they are gone. But on the other hand, many equally-terrible regimes (such as that of Saudi Arabia) were not toppled by the US military, and it is hard to see why other than that the US only tends to attack countries that stop supplying it with oil. In addition, the costs of these wars and occupations has been extremely high---trillions of dollars spent, hundreds of thousands of people killed---and often greatly underestimated beforehand.

Most recently, under President Obama, the US has gone back to a less aggressive approach, and actually followed through on reducing foreign oil dependence. Under Obama US oil imports have fallen faster than ever before. Make no mistake, the US still deploys many troops (including quite literally robot troops, namely Predator drones) in the region, but the strategy has been much more targeted at undermining particular leaders and terrorist organizations rather than at conquering and occupying whole countries.