Whenever anyone starts talking about "fairness" in a political context it raises a huge red flag to me, as it usually indicates a total lack of objective reasoning in favor of ideological propaganda. Most political commentary against a flat tax is a prime example, claiming that it wouldn't be fair since it would cut taxes for the rich while raising them for the poor. This type of reasoning is based only on the current system, ignoring the fact that it is harder to get much fairer than a tax where everyone pays exactly the same rate. The only way a flat tax isn't fair is if you believe that wealth should be seized from the rich and redistributed to the poor, that the wealthiest should bear a tax burden far greater than what is their fair share mathematically, and that the poorest should get tax "refunds" of money that they never even paid. Now you might think that's a good system, but it has nothing to do with fairness.
The problem with any flat tax proposal is its contrast with the current system, which taxes the rich at higher rates and transfers some of their money to the poor. Because the current system is so unfair, those who benefit from that unfairness, whether individually or politically, do not want anything fair. If you have gone for years paying no income tax, or have even received money that you never paid in from the earned income tax credit, why would you want a fair system that actually asks you to pay a proportional amount of your income? Likewise, if you see your political constituency as those who receive government benefits, rather than those who pay for them, you want to transfer even more benefits to your supporters, not force them to contribute.
In my opinion a true flat income tax is probably not politically viable in the U.S. And I'm not sure a flat tax that is loaded down with a maze of deductions and rules like the current system would be all that much better. But those who oppose a flat tax on the basis of "fairness" don't want anything fair. What they want is a system that conforms to their ideology, whereby the rich pay an ever-increasing proportion of their income, while the poorest receive greater and greater handouts of that money.