The results were largely in line with the conventional opinion polls tracked by Reuters in the run-up to the election but well below some online surveys that had predicted the Sweden Democrats could become the largest party.
An anti-immigration party has won nearly 20% of votes in the Swedish election, an early exit poll suggests.
Instead, Sunday's results effectively provided no clear victor and no convenient or simple narrative to explain the shifts underway in Swedish politics.
The Sweden Democrats, which has roots in a neo-Nazi movement but has worked to soften its image, won 17.6 percent, up from 13 percent in 2014, for a third-place finish.
"[If] center-left parties can't come up with a better narrative, a better set of policies, a way of making this work, then they are screwed".
Neither of the leading alliances has reached the threshold of 175 seats, which is needed to govern.
The Social Democrats won 28.4 per cent of votes, down 2.6 points from the 2014 elections and their worst score in a century.
The Moderates - the biggest party in the Alliance - are not interested in co-operating with the Social Democrats and any such deal would also raise questions about democratic accountability.
"Of course I was hoping for more", Tobias Andersson, the head of the Sweden Democrats' youth wing and a candidate for parliament, said late Sunday night, "but still I consider us to be winners in this election, considering we are the party that gained the most new voters". "So we really have to solve it now".
However, although these two blocs continue to be discussed in relation to the 2018 election, the arrangements are now far looser than they once were, in part because the rise of the Sweden Democrats has had an important impact on the Swedish party system.
What’s next for LGBT community after Supreme Court decriminalises gay sex
Indian members and supporters of the lesbian, gay, bisexual, transgender (LGBT) community celebrate the Supreme Court decision to strike down a colonial-era ban on gay sex, in Mumbai on September 6, 2018 .
Supporters attend the Social Democratic Party's election night party in Stockholm, Sweden, Sunday.
Voters in Sweden appear to be split in an unpredictable general election that may turn into one of the most thrilling races in the Scandinavian country's history for decades amid heated debate on immigration. Before the recent refugee crisis, which brought almost 200,000 asylum-seekers to Sweden in 2015 and 2016 alone (the highest per capita number of refugees in Europe), Sweden's other parties had largely agreed on a liberal immigration policy.
Mainstream politicians have so far ruled out co-operation with the Sweden Democrats.
"The biggest factor is that the Sweden Democrats are a very different party with a very different background to, let's say, the Progress Party in Norway, and even to the Danish People's Party". And this wasn't just campaign speak.
None of the seven parties has been willing to negotiate with the Sweden Democrats, which first entered parliament in 2006 with 5.7 percent of votes.
Mainstream politicians have so far rebuffed him. "[The Social Democrats] had been seen as too polite, too open, and too much embracing of globalization and migration".
"The most likely situation will be that the Alliance will form a coalition together and try to seek cross-bloc support".
Swedish government has traditionally been run by a Social Democrat-led centre-left coalition or a centre-right coalition including a number of parties.
Vote: Jimmie Akesson, leader of Sweden Democrats party, seals his ballot at a polling station in Stockholm yesterday.
After an initial exit poll from SVT showed the Sweden Democrats with the second-largest amount of votes, the center-right Moderate party surpassed the nationalist group, with 19.8 percent of voter support.
The Sweden Democrats' breakthrough was hailed by Europe's leading populist, Italian Deputy Prime Minister and Minister of the Interior Matteo Salvini, who has been shaking things up across the European Union since his ascent to power as leader of one half of the Mediterranean country's new populist coalition government. They'll "have to face the facts", Andersson said.