Depends what you want from your brewing.
The diminishing returns thing hold true - It will take 2/3rds of the time and effort to make 5L that it would to make 50L - but, that might not be a problem for you. If you don't need a lot of volume, but want to try AG and learn about brewing, small batches are perfect.
I regularly brew batches in the 5-15L range and use the small batches to try out new recipes or ingredients, or if I just want to brew, but really don't need much more volume of beer.
Its a bit quicker to do a small volume, because the amounts of liquid are smaller and things take less time to heat up and cool down. You will be able to do it in the kitchen on a bog standard stove, and might well have a pot thats big enough sitting in your cupboard.
I do my "small" batches via BIAB and find that on the small scale it makes things super easy and even faster... but I've done small volume batch sparges too and they were fairly easy.
If you were to go the BIAB route, for 5 litres into the fermentor, all you would need is a fairly big pot (10+L) and a meter of the swiss voile fabric that us BIABers use, cut into a circle. I hold mine on the pot with a big rubber band. And thats it.
If you wanted to use a set-up with a mash tun and batch sparge. You'd need a pot (or small eski) for a mash tun, a pot for a kettle and a pot for a HLT, but all of them could be a bit smaller, say 8+litres for the kettle and MT and whatever you have around the place for a HLT. You could do some juggling with a bucket and reduce the need for pots. A braid set-up for lautering is pretty easy to make.
At that sort of batch size.. clean up can be done in your handy dandy dishwasher and I know that my small BIAB brews are over and done from set-up to cleaned up in about 3.5 hours. Less if I no chill.
I say give it a go. You'll learn a lot and might even convince yourself that it would be worth the money/effort to scale up in the future
Cheers
Thirsty