In this paper, detailed studies of the effect of Mg doping in the apatite-type oxide ion conductor La9.33Si6O26 are reported. Mg is confirmed as an ambi-site dopant, capable of substituting for both La and Si, depending on the starting composition. A large enhancement in the conductivity is observed for Si site substitution, with a reduction for substitution on the La site. Neutron powder diffraction studies show that in agreement with cation size expectations, an enlargement of the unit cell is observed on Mg substitution for Si, with a corresponding increase in the size of the tetrahedral sites. For Mg substitution on the La site, a contraction of the unit cell is observed, and the neutron diffraction results indicate that there is preferential occupancy of Mg on the La2 (1/3, 2/3, ≈0.5) site. Atomistic simulation studies show significant local structural changes affecting the oxide ion channels in both cases. Mg doping on the Si site leads to a local expansion of the channels, while doping on the La site results in a large displacement of the silicate O4 site, such that it encroaches the oxide ion channels. The observed differences in conductivities are discussed with respect to these observations.