Rampant Frogantua's second ability counts players who lost the game before Rampant Frogantua entered the battlefield.

If the number of cards remaining in your library is less than the amount of combat damage dealt to a player by Rampant Frogantua, you can't choose to mill that many cards with Rampant Frogantua's last ability.

Rampant Frogantua's second ability counts every player who has lost, no matter how the player lost the game: due to a state-based action (having 0 or less life, trying to draw a card from an empty library, having 10 or more poison counters), because of a spell or effect that states they lose the game, because your awesome might has left them with no choice other than concession, or for any other reason.