Abe returned to Tokyo from a six-day Middle East tour slightly ahead of schedule and convened a Cabinet meeting soon after.
"We are fighting against time, and we'll make an all-out effort and use every diplomatic route that we have developed to win the release of the two," he said.
Abe said he was consulting with leaders in the region. A convoy carrying Japanese Vice-Foreign Minister Yasuhide Nakayama left the embassy in Jordan's capital Amman on Wednesday for an unknown location in the city. He was dispatched from Abe's Middle East entourage to handle the situation.
The Islamic State group demanded the $200 million ransom in a video posted online Tuesday that showed a knife-brandishing masked militant standing over the kneeling captives. It gave a deadline of 72 hours, which the video's release time suggests would expire sometime Friday.
Abe and other Japanese officials have not said directly whether Japan will pay ransom for the captives, 47-year-old freelance journalist Kenji Goto and 42-year-old Haruna Yukawa, the founder of a private security company.
Government spokesman Yoshihide Suga confirmed Japan believes the threat is authentic.
"Japan's aim is not to kill the Muslim people, as the militant group claims it to be," Suga said. "We strongly urge them not to harm the two Japanese and release them immediately."
Abe has limited choices, among them to openly pay the extremists or ask an ally like the United States to attempt a risky rescue inside Syria. Japan's military operates only in a self-defense capacity at home.
But officials are adamant that Japan will continue to provide non-military aid to the region.
"We will never give in to terrorism," Abe said.
In past international hostage situations involving Japanese captives, at least one has been killed while most have been released. It is unclear how many times Japan paid ransom. The only confirmed case was in Kyrgyzstan in 1999.
Japanese media reported Wednesday, citing unnamed officials, that Goto's wife received an email on her mobile phone in December demanding a ransom of more than 2 billion yen ($17 million).
She exchanged several emails with the unknown person, whose email address was similar to one used by the Islamic State group, the Kyodo News service and other media reported. The messages did not include a threat to kill Goto, it said.
Securing the hostages' release will be hard, "because all Japanese diplomats left Syria as the civil war escalated," Tsutomu Ishiai, foreign news editor for the major newspaper Asahi Shimbun, said in a commentary.
"Japan is in an extremely difficult situation. It needs to find a way to save the hostages by getting in touch with religious leaders and local heavyweights who are in a position to make contact with the extremists," he added.
Shuji Hosaka, a senior research fellow at Japan's Institute of Energy Economics in Tokyo, said he did not believe the Islamic State group was specifically targeting Japanese when Goto and Yukawa were captured.
The few Japanese taken hostage by extremists so far generally have been caught up coincidentally in ambushes and attacks.
But the ransom demand appears to have been timed to coincide with Abe's visit, he said.
"I think they were waiting for the best time to release the video and information about them," Hosaka said. "Then Abe happened to be making a visit to the Middle East and there was a reference to the Islamic State and Japanese government aid during the visit. So they may have seen this as a good time to use these two Japanese as negotiation cards."
Associated Press writer Ken Moritsugu contributed to this report.
Copyright 2015 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.