Nix doesn’t check whether a spell’s mana cost is 0. Rather, it checks how much mana was actually spent to cast it. It can counter a spell with mana cost 0 (such as Slaughter Pact), a spell cast via an alternative nonmana cost (such as Allosaurus Rider) or alternative cost of (such as Basking Rootwalla), a spell whose cost to cast it was reduced to 0 (such as Frogmite), a spell cast without paying its cost due to an effect (such as suspend or Mindleech Mass’s ability), or a copy of a spell, among other possibilities. It will not counter a spell that had an additional or alternative mana cost paid to cast it, regardless of that spell’s mana value (such as a face-down spell cast with morph, or an Ornithopter cast while Trinisphere is on the battlefield, for example).

Nix checks whether any player paid mana to cast the spell, not whether its current controller did. If Commandeer is cast on a spell, then Nix is cast on that spell, Nix won’t counter it unless the player who originally cast that spell spent no mana to cast it.

Nix can target any spell. The condition is checked only on resolution.