I’m a Norwegian client, but I’m not in the VAT registry. Do I still have to pay VAT?

Yes. All Norwegian clients pay VAT. If you are in the VAT register, however, you will get the VAT refunded by the Norwegian Tax Administration.