auth [success=done default=ignore] pam_access.so accessfile=/etc/security/access-local.conf
Seems to be the answer. Editing this as I learn more.
From http://linux-pam.org/Linux-PAM-html/sag-configuration-file.html
success=done
done equivalent to ok with the side effect of terminating the module stack and PAM immediately returning to the application.
default=ignore
ignore when used with a stack of modules, the module's return status will not contribute to the return code the application obtains.
from - https://serverfault.com/questions/134471/success-n-control-syntax-in-pam-conf-pam-d-files#134488
success=1 skips a line.
If someone else wants to write a better answer that explains what's happening - I'll happily accept that.
Thanks to Hostfission for pointing me at the right part.