I just read an Atlantic Monthly article, As China's Role in the World Changes So Does Mandarin's Role in China, which begins by relaying the common experience of foreign visitors to China that are able to say anything in Chinese (even if its just a simple 你好, hello). Chinese people will appear surprised and often shocked that you can speak Chinese. However, this reaction might start changing before too long. An increasing number of foreigners visiting or working in China are learning Mandarin Chinese although its still only a very small minority that can speak with moderate fluency. A few days ago, I ran into a fellow foreign teacher at a local coffee shop reading a Chinese book. He's been studying Chinese for about 3 years which makes me quite jealous since my Chinese ability is considerably more limited.
Chinese can certainly be a difficult language to learn, especially for speakers of an alphabetical language such as English. However, I think the main reason very few Westerners can speak or read Chinese is that, until recently, there weren't many opportunities to do so. I remember studying French in high school. At that time, I think French and Spanish were the only options. Similarly, if I'd wanted to learn Chinese in college, it wouldn't have been possible since few universities at the time offered Chinese courses. Things are much different today - most colleges offer Chinese and some high schools are doing so as well. When I taught at an American university, I took some Chinese courses after I started visiting China. I was quite impressed with how quickly some American college students picked it up, especially as I struggled to keep up. Either there's some truth to the idea that its harder for older people to learn a new language or I'm just a 笨蛋 (dimwit, idiot or literally "stupid egg").
I think the Atlantic Monthly article is a bit inaccurate in some respects. For example, while its true that Chinese is one of the oldest written languages in the world, the notion that it allows "modern descendants to converse directly with their distant ancestors" is quite an exaggeration. In fact, written Chinese (汉字; hanzi or Chinese characters) has evolved substantially over 4500 years and ancient Chinese writing might be largely indecipherable to most Chinese as well as non-Chinese people.
Also contrary to the article, only a relatively small percentage of modern Chinese characters are pictorial and many of these are have changed substantially over time and take quite a bit of imagination to grasp what they're supposed to identify. For example, I doubt that most people could readily discern that this picture 心 is the Chinese character for the word "heart," let alone that this one, 家 means "family" or "home" (a picture of a pig under a roof). Chinese characters changed even further in the early 1950s when Chairman Mao decided they should be simplified to promote greater literacy among Chinese people.
To me, while a very difficult language to learn, Chinese is also a fascinating language. This is largely due to its differences from English. While I may never become fluent, there will probably be an increasing number of non-Chinese people that do. Maybe this shouldn't be too surprising - as China has grown economically and become more commonly mentioned in Western news, Westerners have become more interested in learning Chinese. Who knows - its even possible that the surprised reaction of Chinese people to foreigners speaking Chinese may become a thing of the past in the not too distant future.