Fair point, but rabble or no, he was fighting in a battle while heavily outnumbered with worse-trained and less-disciplined forces, and performed exceedingly well. He deserves some credit, if not necessarily in duels then in general fighting without dying.
I won't argue that, but I will maintain that it's not nearly enough to expect he'd have any real chance of beating The Mountain in a 1v1.
Pretty hard to judge a guy as a "big deal" when his best 1v1 feat, historically or otherwise, is beating some one-off character with no grounding at all for his level of skill beyond the who promptly got rekt. As for his pit fighter history, even fucking Jorah Mormont managed to win some major pit fights in the show, and he's much older than Daario and generally not seen as a great fighter.
The majority of Meereen's pit fights are actually 1v1's of almost
exactly the same nature as Westeros's combat trials, and the city's Champion would've had a stellar record at them. In the chapters that go through detailing Danaerys's attempts at ruling the city, it ends up getting revealed that several of the man's lessers had far-reaching reputations of their own.
Jorah Mormont was considered to be a good fighter even before leaving Westeros, and has spent a great many years merc'ing his ass off since then. He's just not that great at jousting, and as such, has only a single tournament championship to his name in Westeros.
If he legitimately beat the Sword of the Morning in a 1v1, he's certainly got the skills to beat the Mountain.
Assuming he was as good a swordsman as a lancer, most certainly. It's hard to be really sure, though; unlike Dayne, who's got at least as many swordsmanship feats to his name as tourney wins, Rhaegar's most noteworthy melee battle was the one where Robert Baratheon quite literally crushed him in single combat. It's not impossible that he's kind of an inverse Robert that way, (as in, great at jousting and shit at melee), but it's also not terribly likely.