My understanding is that the "black specs" on brass fittings is lead. You don't see it where the fitting has been machined (eg. to form threads). Machined surfaces show that beautiful buttery golden colour.
Brass is an alloy of copper and zinc. In manufacture lead is added to molten brass because it melts at a lower temp than the copper and zinc.
When the molten metal mixture is injected into the die (or mold) to form a cast of the fitting, the copper and zinc solidify first and their atoms arrange themselves in a repeating matrix with no space between the atoms for the lead, which is still in a liquid state. So the lead is forced to the outside of the casting. This where it fufils the reason why it was added in the first place. It acts as a mold release agent ensuring that the fitting does not stick to the tool steel of the die. The casting is easily ejected as the lead finally cools and soldifies on the external surface of the casting, leaving a black discolouration. (spray on mold release compounds are used too and may contribute to the discolouration)
What this means in brewing is that exposing your hot wort to machined surfaces of a brass fitting (where the lead contamination has been cut or drilled away) is probably OK. But exposing wort to extenal blackened surfaces (eg. a brass hose barb on a false bottom) risks getting the lead on the surface into the wort.
It is often said that there is only a tiny percentage of lead used in the brass casting, but be aware that this small amount can end up concentrated on the external surfaces. As has been said, Palmer's book How to Brew (free online) details how to remove the lead from the surface by a pickling process.