Maintaining consistent uncertainty estimates in localization systems is crucial as the perceived uncertainty commonly affects high-level system components, such as control ordecision processes. A method for constructing an observability-constrained magnetic field-aided inertial navigation system isproposed to address the issue of erroneous yaw observability,which leads to inconsistent estimates of yaw uncertainty. The proposed method builds upon the previously proposed observability-constrained extended Kalman filter and extends it to work with amagnetic field-based odometry-aided inertial navigation system.The proposed method is evaluated using simulation and real-world data, showing that (i) the system observability propertiesare preserved, (ii) the estimation accuracy increases, and (iii) theperceived uncertainty calculated by the EKF is more consistent with the true uncertainty of the filter estimates.