Stems usually come in one of two materials, vulcanite or acrylic. Vulcanite is rubber that has undergone the vulcanization process, which means it has undergone high heat and the introduction of sulfur. The resulting material can be molded into a myriad of shapes, including pipe stems. Vulcanite pipe stems are usually black in color. Acrylic stems, also known by their brand name of Lucite, are made from acrylic resin or, essentially, plastic. They can be black or any number of colors.
Each material has its advantages. Vulcanite stems are softer than acrylic and provide better grip when being held by the teeth. Acrylic stems are resistant to oxidation, which can often afflict vulcanite stems.
My own preference is an acrylic stem. I usually don't hold a pipe in my mouth for an extended period, and I prefer the ease of maintenance that an acrylic stem offers. It doesn't affect the smoking quality of the pipe, but it does affect my overall enjoyment of my pipes because I'm not having to worry about polishing the stems and keeping them free of oxidation.
Apart from being unsightly, the oxidation that a vulcanite stem can definitely affect the enjoyment of a pipe. Just a little bit of oxidation will result in a very sour taste and result in a most unpleasant experience.
Oxidized stems can be restored however. A trip to your local tobacconist and some turns on the buffing wheel will get your vulcanite stem looking like new. The task can also be done at home with not too much effort. I use a stem restore product available from Walker Briar Works in Manchester, Michigan.
The choice between acrylic and vulcanite comes down to personal preference. I probably have an equal number of pipes with each type of stem and I usually don't let the stem material influence my buying decision.