To answer OP question...
If the speakers that you would build turn out like mine, then yes, you should be ashamed
On a more serious note (rare for me!) it has already been emphasized that (1) you should do what is best for you and bugger what anyone else may think of it and (2) the value of DIY. Even if there is not agreement on "what is best?" (and there isn't, nor will there ever be...this is a great confusion to the newbie!) I will make a pitch for the value of another angle: Buy good quality* used gear. For someone like me with limited money, patience, tools and (especially) competence, this ia an excellent avenue. To use a common example, over many years I've bought (and sold some) very old, quite inexpensive, Bose 901 pairs, the American Icon speaker that makes audiophiles foam at the mouth at its very mention

, these are pretty durable and rarely have anything wrong with them (other than being Bose?)

Using the Bose 901 as an example, here is a speaker that was (and still is) quite over-priced if bought new. However, you can buy a working pair for anywhere from $300-$600, is very likely to be in good condition, and importantly, you can usually sell it and get back most of your money. If your audio snob friends see them and curl up into a little ball on the floor, twitching, do you really care? Is your happiness or your friends' approval more important to you?
A thirty- or forty- year old speaker cabinet probably is in need of some minor refinishing. Any imbecile, even me, can clean, sand, stain, varnish, etc. with pretty good results. I can buy screws, new feet, etc. Replacing defective parts (drivers, crossovers, etc.) is a bit more chalenging but easily within reach of a DIY-er. Also many speakers have re-foam kits, etc. In sum, buying decent used gives you a chance to f- er muck

around with a (presumably) good speaker and tweak it to your own likings.
All this long-winded post is to say a middle ground between pure DIY and buying new (with its 100% mark-up) is to buy used and fix it up a little. It is even likely you won't need to do anything to it. My most recent buy (three days ago) is my first-ever pair of "pro" speakers (Yorkville Unity U15) which were listed for "parts or repair" and I got about 1/4 cost of new, delivered. Other than expected wear and tear, I have yet to find anything wrong with them! In this case, they are speakes designed to be played VERY LOUD at a gig or for drunken patrons in a bar

but at least for light-duty home use they are if nought else, a step up from the despised Bose 901.
* Of course "quality" is whatever you mean it to be.