If it's 1mm washer, just fit it there and forget, imho. Even with 2mm it will work just as well, but I personally would be too anal to let that slide on my bike.
No guarantee that your wheel dish is that accurate - anything within 1mm is generally acceptable.
If you add 1mm to the ds axle, and...