The fine structure splitting of bright exciton states is measured for a range of thermally annealed InGaAs quantum dot (QD) samples with differing degrees of In/Ga intermixing and also for a dot-in-a-well (DWELL) structure. Magnitudes of the fine structure splitting are determined in polarization-resolved differential transmission experiments from measurements of the period of quantum beats observed in QD exciton dynamics. The splitting is found to decrease in structures with weaker strain: both for In/Ga intermixed QD's and also in dots surrounded by strain-reducing layers (DWELL's). Our findings pave the way to the achievement of entangled two photon sources based on emission from individual QD's, currently prevented since the fine structure splitting is larger than the radiative linewidth.