Spent grains, trub and yeast sediment are all very high in nitrogen, which means they'll fire up biological activity in a compost heap or in the soil. That's why they soon turn sour. They will temporarily depress pH but have only a slight lasting effect on soil pH, easily counteracted with lime or wood ashes. They should be espeially useful in compost if the other ingredients are low in nitrogen, such as sawdust, dead leaves and straw. As Partial Man said, mix well. I dig them directly into soil well in advance of planting, mostly where leafy greens or sweetcorn are going in. In fruting plants such as tomatoes I found they encourage too much top growth and discourage fruiting.
I've seen no effects on worms, but it's possible. Probably the hops, which are toxic to so many forms of life other than us lucky few.
Look up the history of the word cloaca; it's a gem.