As long as it isn't used maliciously I don't think downvotes make this place worse.
I usually like to compare to facebook/youtube/x to Reddit as the formers have all removed dislikes/downvotes, the latter hasn't. The platform I find most interesting is usually reddit because it has filtered a lot of the "bad content" away and only what the majority have liked is visible to me.
It's difficult to compare because we use stake here and there's also way too few people doing active downvoting so there's never any real conflicts or cases where certain posts get rewarded more after a downvote occurs because of the downvote. We're just too small to see it come to fruition properly and to then decide if it is good the way it is now or if it needs fuether changes.