The free graviton theory given by linearising Einstein's theory has a dual formulation in terms of a dual graviton field. The dual graviton theory has two gauge invariances giving rise to two conserved charges, while the ADM charges of the graviton theory become magnetic charges for the dual graviton theory. These charges can be ill-defined in topologically non-trivial settings and we find improvement terms that can be added to these to give gauge-invariant conserved charges. These gauge-invariant charges, which have local expressions in both the graviton and dual graviton formulation, give topological operators of the theory that should be considered as the generators of the genuine symmetries of the theory.