The security guard was to be questioned on Monday, prosecutor Lucas Eriksson told AFP.
At the end of October, six other firearms and ammunition were reported stolen from the government offices, which are protected by guards from the private security firm Securitas.
According to Swedish media, the weapons were six Glock pistols and 300 special bullets kept in a safe and to be used in case of a heightened security threat, such as a terrorist attack.
Investigators are trying to determine if there is a link between the two thefts.
Defence Minister Peter Hultqvist deemed the thefts unacceptable.
“We need to have good control over the handling of weapons in our country,” he told Swedish news agency TT.