Unjustified whale flags are going to happen with or without free downvotes. Sure, they get a little more juice proportionally, but I don't think of it as breaking (and it's also in fact why the % is not as high as what some people have been pushing for, which is 100%).
The downvote committee can happen today. Don't see what's stopping it. Good luck convincing enough people though. Actually, the downvote committee has a better chance of forming with free downvotes, and funded accordingly.