No matter what you change, everything makes a difference.
A lot of the time this difference is so small it is only noticable when tasted back to back.
Ever had one of those batches that initially you thought was not-quite-right? Then it seemed to "come good"? Chances are it wasn't aging ... it was your tastebuds forgetting the last batch it was being drunken with.
When I have a BB ale batch and a Wey ale batch side to side the BB ale tastes inferior and a bit funky. With just BB malts they're all good.
Comparison is bigger than absolute taste in the same way using the loo after someone has done a rancid dump is only revolting until your dump outstinks their's.
We all think our shit don't stink and whether we like it or not the proof is obvious when you stick your head under the covers to suck in that sweet, sweet fart and think "WHOA!". You know you love it.
Not saying your beer tastes like shit, just that taste is a matter of taste. One man's nectar is another's Pride of Ringwood.