With accurate cross-sections and higher signal-to-noise, X-ray spectroscopy can directly measure Milky Way gas and dust-phase metal abundances with few underlying assumptions. The X-ray energy band is sensitive to absorption by all abundant interstellar metals — carbon, oxygen, neon, silicon, magnesium, and iron — whether they are in gas or dust form. High resolution X-ray spectra from Galactic X-ray point sources can be used to directly measure metal abundances from all phases of the interstellar medium (ISM) along singular sight lines. We show our progress for measuring the depth of photoelectric absorption edges from neutral ISM metals, using all the observations of bright Galactic X-ray binaries available in the Chandra HETG archive. The cross-sections we use take into account both the absorption and scattering effects by interstellar dust grains on the iron and silicate spectral features. However, there are many open problems for reconciling X-ray absorption spectroscopy with ISM observations in other wavelengths. We will review the state of the field, lab measurements needed, and ways in which the next generation of X-ray telescopes will contribute.