I like the idea to impose a penalty for repeated votes for a single user. Not sure if it's feasible at scale or not, though.
What do you mean here? I think it would be possible to implement, but are you talking about knock on effects maybe and not just technical implementation?