If you think that by ignoring a Party Wall Notice means that your neighbour can not go ahead with their proposed works, then unfortunately you are wrong. The Party Wall Notice that you first received will allow you 14 days to respond. Once the 14 days have elapsed and you have not responded back then another letter will be sent out to you. This letter is called a 10 day letter and it allows you a further 10 days to respond. If by this point you still have not responded as you think that by ignoring these letters you are stopping your neighbour with their planned works then you are incorrect.
If after the 10 day letter you still have not responded then your neighbour can appoint a surveyor to act on your behalf whether you like it or not. So it is best to respond to the Party Wall Notice and not ignore it. If you do have any issues with the proposed works its always best to speak to your neighbour directly and avoid escalating matters.