We present a semi-classical study of the effects of the Lorentz force on electrons during high harmonic generation in the soft and hard X-ray regions driven by near- and mid-infrared lasers with wavelengths from 0.8 to 20 μm, and at intensities below $10^{15}$ W/cm$^2$. The transverse extent of the longitudinal Lorentz drift is compared for both Gaussian focus and waveguide geometries. Both geometries exhibit a longitudinal electric field component that cancels the magnetic Lorentz drift in some regions of the focus, once each full optical cycle. We show that the Lorentz force contributes a super-Gaussian scaling which acts in addition to the dominant high harmonic flux scaling of $\lambda^{-(5\text{-}6)}$ due to quantum diffusion. We predict that the high harmonic yield will be reduced for driving wavelengths $>$ 6 μm, and that the presence of dynamic spatial mode asymmetries results in the generation of both even and odd harmonic orders. Remarkably, we show that under realistic conditions, the recollision process can be controlled and does not shut off completely even for wavelengths >10 μm and recollision energies greater than 15 keV.