A couple of years ago I pulled my valves apart while i was rearranging some plumbing. Not a speck in any of them after a decade of all-grain brewing (OK, on the HLT valve that shouldn't be a surprise
).
One problem is that people use these valves in ways they are not designed to be used. Ball valves are not meant to be used for throttling flow -- they should be either fully open or fully closed. Most ball valves use disc shaped seals to form a seal around each end of the ball, rather than something that will both seal, and completely fill the space around the ball. As a result, a standard ball valve has hidden recesses where crap can collect, especially if it is used in a halfway position.
Use a type of valve that is designed to be used for throttling flow, like a butterfly or globe valve.