First thing we need to do is go to the Icthian Grotto to the east of the Village of Kaupangr and defeat Lord of the Fjord who is holding the Icthian Relic that we need. Map location can be found below.
The Lord of the Fjord is a pretty simple fight, he is just an Icthian with slightly more HP and more damage than the other ones. Once you've obtained the Icthian Relic our next destination, The Southern Gate, is far to the north of our current location. You will know you're at the right spot when you can't go any further north.
Place the Icthian Relic into the door and it will summon a large wave of Undead enemies for you to defeat. Once they're all defeated the door will open and grant you access to the Halls of the Dvergr.