A water softener uses a salt (typically sodium citrate, but could be a sodium phosphate or polyphosphate) to chelate, or chemically "bind" cations (typically divalent, like calcium or magnesium; but also trivalents like iron) in your water supply.
The chelate is essentially an insoluble divalent cation salt; this is the white residue. If it is a large amount, it just means your water must be particularly hard. It'* the same stuff you see in a cloudy pool when the chemical balance is "off."
There is no practical way to avoid this residue with the quantities of softened water needed for household use like washing; the particles are so small no household-practical filter will strain them, and the kinds of ion-exchange membranes (or other ion-exchange scheme) needed would be industrial in size. I suppose you could build a still and evaporate a few hundred gallons of water a day
This residue is the tradeoff in needing softened water, as hard water doesn't lather soap well (due to divalent cation-lipid interactions), and certain trivalent metals (e.g., iron) can stain clothes.
(Side note: divalent cation-lipid interactions are why Milk of Magnesia makes a good laxative :P )