Despite Pence's hardline stance and words, he has also signaled the United States is possibly open to negotiations when he said in regards to the North Korean officials, "I haven't requested any meeting".
The invite, presented to South Korean President Moon Jae-in by Kim's younger sister, Kim Yo Jong, was delivered during a historic meeting between North and South Korean officials at Seoul's presidential palace Saturday, presidential spokesman Kim Eui-kyeom said.
The countries are still technically at war since a 1953 armistice, though they have resumed talks after a year-long standoff between North Korea and the United States in which an exchange of threats between the heads of state elevated tensions and prompted the North's continued missile and nuclear tests.
Moon is trying to use the games as an opportunity to restore regular communication with North Korea and eventually pull it into talks over resolving the global standoff over its nuclear program.
A senior USA government official says that Moon urged Pence to meet with the North Korean delegation, but Pence left the Olympic venue without doing so.
The highest profile member of the North Korean delegation to the Games, she is the first immediate member of the North's ruling family to visit the South since the 1950-1953 Korean war.
North and South Korean leaders swapped warm handshakes and smiles at Friday's opening ceremony, enabling the International Olympic Committee to present the Games as enablers of peace and take the media spotlight away from a Russian doping scandal that has plagued it for years.
Moon's office says the North Korean delegates were served traditional food from the Gangwon province, a mountainous coastal area that's home to the Olympic towns of Pyeongchang and Gangneung.
Up the road, however, an elderly husband and wife, each draped in the South Korean flag, denounced Moon as a "fake president". US Vice President Mike Pence, sitting a few seats away in the same Opening Ceremony VIP booth, looked stony-faced as the scene unfolded. "When my eyes met the North Korean athlete's I just smiled".
"We will not allow North Korea to hide behind the Olympic banner the reality that they enslave their people and threaten the wider region", he said. The Koreas could potentially talk about a South Korean special envoy meeting Kim Jong Un in Pyongyang or even a summit between Moon and Kim.
The figures from the two Koreas shared kimchi (Korean pickled cabbage) and soju rice liquor, and spoke for three hours. He criticized the Trump administration for straining too hard to signal disgust of Kim Jong Un's government. For his part, Abe said that Japan and the USA had "confirmed... that we can never accept a nuclear-armed North Korea".