From my readings, it is the white layer that you are after. This indicates healthy yeast.
I brewed with a Wyeast Belgian Witbier (3944), washed the yeast, and got a terrific amount of nice, white yeast from that one.
I recently tried it with a dry pitched US-05, and still got a layer of white yeast, just not nearly as much.
I wouldn't think that it would be too detrimental if a bit of the off coloured yeast got into the starter.