What is the £40 charge for? Processing the change, change of risk profile of the new address, producing new documentation, updating information on their systems?
Were the costs for changing the policy mid-term not available to you in the policy documents and if so, do this charge fall in line with what they are proposing?
Aside from this, the distance from the new to old property has nothing to do with it. The amount of work involved is the same even if you were moving next door.