Can a Party Wall Agreement be refused?

Can a Party Wall Agreement be refused?