In a televised address late on Wednesday, Mr Harper said: "We will not be intimidated. Canada will never be intimidated.
"In fact, this will lead us to strengthen our resolve and redouble our efforts... to take all necessary steps to identify and counter threats and keep Canada safe."
Mr Harper stressed that the perpetrators "will have no safe haven" in Canada, but admitted the attacks showed that the country was "not immune to terrorist attacks".
Mr Harper had been meeting MPs at the time of the shooting, but was safely evacuated.
The shooting began at the war memorial
Police quickly moved in to seal off the area around the parliament building
Reporters, politicians and staff barricaded themselves into chambers in the parliament building
The attack began on Wednesday morning, as two soldiers guarding the memorial came under fire from a man carrying a rifle.
One soldier, Cpl Nathan Cirillo, died of his injuries. Three other people were treated in hospital and released by evening.
Minutes after the attack at the memorial, dozens of shots were fired inside the parliament building.
Sergeant-at-Arms Kevin Vickers' brother: "We're very, very proud of him"
"MPs and [Parliament] Hill staff owe their safety, even lives, to Sergeant at Arms Kevin Vickers who shot attacker just outside the MPs' caucus rooms," New Democrat MP Craig Scott tweeted.
Zehaf-Bibeau, 32, had apparently been designated a high-risk traveller and had his passport taken away because of suspected jihadist sympathies.
A Twitter user who posted an image purporting to show Zehaf-Bibeau said it had come from an IS-linked account.
Extremism in Canada
Canadian Momin Khawaja involved in al-Qaeda-inspired plot to strike British targets with fertiliser bombs in 2004; jailed in 2009 in the first conviction under Canada's 2008 anti-terror law
So-called "Toronto 18" planned in 2006 to detonate truck bombs around Toronto and take hostages including the prime minister in al-Qaeda-inspired plot; 11 convicted of terror-related offences
Hiva Alizadeh jailed in September 2014 for attempting to organise a militant jihadist cell in Ottawa
Tunisian Chiheb Esseghaier and Palestinian Raed Jaser charged in 2013 over a plot to derail a Toronto-New York train, with alleged support from an Iran-based al-Qaeda cell; both awaiting trial
A wide area in central Ottawa was then sealed off amid reports that other gunmen may have been at large. However, on Thursday police said they were satisfied there had been only one attacker.
Some parliamentary staff, politicians and journalists were forced to remain barricaded inside the building for several hours.
Parliament Hill is still closed to the public, but MPs are expected to return to work at 10:00 (14:00 GMT).
In a telephone conversation with Mr Harper, US President Barack Obama condemned the attacks.
Are you in Ottawa? Were you affected by the events yesterday? Share your experiences by emailing haveyoursay@bbc.co.uk.