The new species was identified from a partial skeleton unearthed at Hampden Beach in New Zealand's Otago region.
With an average height of 1.7 metres and a weight of 100 kilogrammes, the giant bird is thought to be one of the world's biggest extinct penguin species, easily dwarfing its cuddly-looking modern descendants.
The paleontologists who discovered the fossilized remains dubbed it "kumimanu biceae" which means "monster bird" in the Maori language.
Some 60 million years ago, a penguin as tall as Kanye West walked the shores of New Zealand.
The tallest penguin now living is the emperor penguin, which reach about 1.2m when fully grown.
"Dating back to the late Paleocene 60-56 million years ago, the newly named Kumimanu biceae was so big it stood 1.65 metres tall (5'5") and measured 1.77 metres in length (5'10") when swimming.
The discovery shows that the penguins "got big very rapidly", according to Mayr. Kumimanu tops all other giant penguins by at least 7 inches (18 centimeters) But what makes Kumimanu particularly exciting is its age - it's the earliest giant penguin scientists have found so far.
María Ríos et al PLOS One
Penguins are believed to have evolved from a flying ancestor perhaps resembling a cormorant, Mayr said. Once grounded, some penguin species became much larger, growing from about 80cm tall to twice the size.
Two other huge pre-historic penguins were previously discovered on the island, but they were slightly smaller than "kumimanu". After dinosaurs were wiped out 65 million years ago by a giant asteroid impact, so did many predatory marine reptiles, and the penguins were quick to seize the opportunity.
Kumimanu lived long before Antarctica's glaciation.
The humerus (top) and a bone from the shoulder girdle (coracoid, bottom) of the Paleocene giant penguin Kumimanu biceae, compared to the corresponding bones of one of the largest fossil penguins known to date (Pachydyptes ponderosus from the Eocene in New Zealand) and those of an Emperor Penguin (Aptendodytes forsteri).
"We examined the wing and leg bones of this penguin and quickly realized that we had a previously unknown species", he said. Credit: G. Mayr/Senckenberg Research Institute. Studies suggest early penguins were brownish, not the trademark black and white of today's penguins, Mayr said.
Scientific reference: Gerald Mayr et al.
"This fossil shows massive penguins were there right from the start when penguins first evolved - and that wasn't really known before now".