THE RETURN to power of Japan’s conservatives has once again spurred hope that the country can restore its role as an economic powerhouse and the northern anchor of a rebalanced American presence in East Asia. A revitalized Japan could act as a weighty counter to China’s apparent determination to assert itself as the new regional hegemon. The newly installed government, under the Liberal Democratic Party (LDP) banner, offers prospects for more stable and competent governance after years of turbulence and lack of leadership.

But there are grounds also for a high degree of caution, not only about the advent of stable leadership but also about where Japan is headed. The new government of Prime Minister Shinzo Abe embraces contradictory national impulses. In the realm of economics, Abe’s cabinet includes supporters of a return to old-style pork-barrel spending and export-led growth policies. But it also includes reformers who favor deregulation and open markets to force Japan to compete more efficiently in the global economy. In foreign policy, the cabinet encompasses pragmatic realists who want to expand Japan’s security role in close coordination with the United States as well as revisionist nationalists who hanker for a face-off with China and express provocatively unrepentant views about Japan’s wartime record.

It is unclear how Abe will resolve these conflicting pulls, especially under the pressure of forthcoming upper-house elections. It isn’t even clear just where he personally stands on such matters.

Americans may dream of a pragmatically conservative Japan that restores its economic health through freer trade and offers a reliable security partner for a policy of engaging China while constraining its aggressive expansionism in East Asia. But they could get a nightmare instead—a fortress Japan, isolated within Asia by its nationalism, protecting an economy beset by public debt and an aging populace.

The choices facing Japan today must be set against the backdrop of two decades of stagnation and often-faltering reform efforts. The Japan of the late 1980s, its system praised in business-management books and its arrival as the new global superpower trumpeted on the cover of weekly newsmagazines, seems like a distant memory. The early 1990s punctured the confidence, even arrogance, that had swelled the heads of Japanese policy makers. The state-directed and export-led economy ground to a halt amid the wreckage of the asset bubbles created by the 1985 revaluation of the yen. Persistent stagnation also upended a political system of single-party rule and bureaucrat-led governance. The crisis occurred in tandem with the end of the Cold War, which called into question postwar Japan’s foreign-policy anchor, its security alliance with the United States.

In 1993, Japan entered a period of political change, beginning with the emergence of its first, short-lived non-LDP government. Less than a year later, the LDP returned to power, first in coalition with the Socialist Party and then with the Buddhist New Komei Party. Then, in the late 1990s, the opposition coalesced into the newly formed, center-left Democratic Party of Japan (DPJ). This represented a serious challenge to the LDP, although its decline was obscured by the unusual, five-year rule (2001–2006) of the personally popular Prime Minister Junichiro Koizumi, based on a policy of economic reform coupled with an assault on his own party’s axis with bureaucrats and agricultural-industry lobbies.

The LDP returned to its old ways, however, under a trio of short-lived premiers, starting with Koizumi’s chosen successor, Abe. Ironically, Abe brought back into the party those who had been ousted for opposing Koizumi’s pet project, the breakup and privatization of Japan’s public-banking and insurance oligopoly run under the aegis of the postal system. With the LDP disavowing Koizumi’s reformism, the door was opened for the DPJ, whose 2009 victory was hailed as the birth of a new era of two-party politics in Japan.

The DPJ came to power with an ambitious reform agenda. It advocated breaking up mandarin rule—“people not concrete”—in the allocation of government funds, pursuing global competition and looking for sources of growth in new industries. After the massive earthquake and tsunami of March 11, 2011, the DPJ government also embraced an anti-nuclear-power agenda and assailed the “nuclear village”—the cozy ties among bureaucrats, construction firms and regional power monopolies that the LDP had fostered. But the party’s inconsistency and incompetence led to its ignominious defeat in the December 2012 elections, with little of its agenda accomplished.

The greatest legacy of the DPJ’s three-year rule was the reemergence of a long-standing debate about Japan’s foreign policy. Its intellectual roots go back to the prewar period, but it continued in subdued form during the long LDP rule and intensified with the end of the Cold War. During much of the postwar period, Japanese foreign policy was a contest between pro-American, anti-Communist conservatives and the Japanese Left, led by the Socialist Party, which rejected the U.S.-Japanese security treaty and called for a pacifist vision of “unarmed neutrality.” The doctrine of Japan’s pro-American and pro-British postwar leader Shigeru Yoshida became the consensus: Japan would focus on economic reconstruction by relying on the United States for its defense—and pay for it by giving America access to its soil for strategic bases.

The Yoshida Doctrine retains an iconic status in Japan. But below the surface, many Japanese of all political stripes were never comfortable with a strategy of reflexive dependence on the United States. On one side, Yoshida faced stiff opposition from nationalist conservatives who accepted the necessity of the alliance in the name of anti-Communism but fundamentally rejected the postwar settlement enforced by the American occupation. From the first days following the 1952 restoration of Japanese sovereignty, these nationalists sought to roll back the keystones of that American legacy: the American-authored revision of the Japanese constitution, particularly Article 9, which renounced war as an instrument of national policy; the judgments of Japanese war criminality imposed by the Tokyo war-crimes tribunal; and the educational reforms imposed by the occupation and enforced by the left-wing teachers union. Many of these nationalists had served in the wartime regime and were allowed to return to politics only by America’s Cold War decision to rally anti-Communist forces in Japan against the Left; prominent among those anti-Yoshida conservatives was Nobusuke Kishi, who served as prime minister from 1957–1960 and is beloved by Abe, his grandson.

THESE DIVISIONS have their roots in a historic Japanese debate about identity that goes back well before World War II. One camp represents what can be broadly called Asianism, a belief that Japan’s identity and strategic interests lie in Asia. Conversely, others have argued that Japan, as an offshore maritime nation ready to adapt Western technology and ideas, should ally with the Western powers. Faced with the threat of Western imperialism, and noting disdainfully the failure of China and Korea to reform themselves to meet that threat, Yukichi Fukuzawa, the intellectual leader of Japan’s nineteenth-century Meiji transformation into a modern state, famously argued in 1885: “We do not have time to wait for the enlightenment of our neighbors so that we can work together toward the development of Asia. It is better for us to leave the ranks of Asian nations and cast our lot with civilized nations of the West.”

Japan’s Asianists were themselves divided. The “greater Asianism” camp advocated imperial advance on the Asian mainland, with Japan as the self-proclaimed liberator and protector of Asia from Western colonialism. Its victory over pro-Western liberals led to the tragedy of World War II in Asia. But there was also a small camp of anti-imperial Asianists led by the prewar journalist Tanzan Ishibashi, who opposed territorial expansion while expressing sympathy for the Asian nationalism of Sun Yat-sen and others. Those debates continued among conservatives, though now reflecting postwar realities, throughout the latter decades of the U.S. partnership.

Meanwhile, the end of the Cold War cast the Japanese adrift. Most people in the country finally accepted the legitimacy and necessity of the U.S.-Japanese security treaty and a limited role for Japan’s military. In doing so, they also reinvigorated the debate among conservatives about the need for Japan to act more forcefully—and for some more independently—in its foreign and security role. Some conservatives began questioning the need to maintain Cold War levels of U.S. troop deployments, not only in Japan but also in Europe. That debate intensified after several American servicemen raped a Japanese preteen in Okinawa in 1995.

The DPJ founders represented both liberal and conservative advocates of greater self-reliance, a kind of Japanese Gaullism. Some talked of an “alliance without bases,” in which Japanese forces would be responsible for self-defense and U.S. forces would use bases and storage facilities in emergencies such as a conflict on the Korean Peninsula. Despite the Taiwan Strait crisis of 1995–1996, in which the People’s Republic of China fired a series of test missiles into the Taiwan Strait as a warning to Taiwan about pursuing an overt two-China policy, the DPJ founders optimistically hoped that the region was headed toward resolving the legacy of the Cold War, both in Korea and with China. A key part of this “new Asianism,” as I have termed it elsewhere, was the advocacy of East Asian regionalism and the creation of an East Asian community based on the model of the European Union.