DNS Resolution Fails for ping and curl, but not dig I'm running DNSMasq as a local DNS server, so I can resolve *.local.pcfdev.io (as discussed here Using PCF Dev Offline with Mac OS X). Everything worked when I first set things up.

A couple of days later, after a few restarts of my MacBook, whilst offline I can no longer resolve things like api.local.pcfdev.io using curl or ping. However, dig does the right thing.

$ dig api.local.pcfdev.io

; &lt;&lt;&gt;&gt; DiG 9.8.3-P1 &lt;&lt;&gt;&gt; api.local.pcfdev.io
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 46877
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;api.local.pcfdev.io.       IN      A

;; ANSWER SECTION:
api.local.pcfdev.io.    0       IN      A       192.168.11.11

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Sep  6 10:17:44 2016
;; MSG SIZE  rcvd: 53

$ curl api.local.pcfdev.io
curl: (6) Could not resolve host: api.local.pcfdev.io


I've tried adding -AlwaysAppendSearchDomains as an argument to /usr/sbin/mDNSResponder in /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist and restarted the mDNSResponder with launchctl, but to no avail.



UPDATE 1

There is definitely something listening on the right local IP:

$ nslookup api.local.pcfdev.io
Server:     127.0.0.1
Address:        127.0.0.1#53

Name:   api.local.pcfdev.io
Address: 192.168.11.11

$ ping api.local.pcfdev.io
ping: cannot resolve api.local.pcfdev.io: Unknown host

$ telnet 192.168.11.11 80
Trying 192.168.11.11...
Connected to 192.168.11.11.
Escape character is '^]'.

HTTP/1.1 400 Bad Request

Connection closed by foreign host.




UPDATE 2

After trying the suggestion below of removing all DNS servers from Network Preferences except 127.0.0.1, I can't resolve anything. I managed to get some debug logging out of mDNSResponder:

mDNSResponder[91]:  74: DNSServiceCreateConnection START PID[32612](ping)
mDNSResponder[91]:  74: Error socket 75 created 00000000 00000001
mDNSResponder[91]:  74: DNSServiceQueryRecord(15000, 0, api.local.pcfdev.io., Addr) START PID[32612]()
mDNSResponder[91]:  74: Error socket 75 closed  00000000 00000001 (0)
mDNSResponder[91]:  74: DNSServiceQueryRecord(api.local.pcfdev.io., Addr) ADD    0 api.local.pcfdev.io. Addr
mDNSResponder[91]:  74: Cancel 00000000 00000001
mDNSResponder[91]:  74: DNSServiceQueryRecord(api.local.pcfdev.io., Addr) STOP PID[32612]()
mDNSResponder[91]:  74: DNSServiceCreateConnection STOP PID[32612](ping)


I did also observe that as explained in the proposed answer, nslookup and dig don't cause anything to be logged by mDNSResponder, but other tools (ping, curl) do.

So it seems like for whatever reason either dnsmasq isn't working (I can establish a TCP connection to 127.0.0.1:53) or mDNSResponder isn't using it.



UPDATE 3

etc/resolve.conf ceases to exist when my wifi adapter is active, but I'm not connected to a network. Could this be why CLI tools don't use the local dnsmasq server?
Had this same issue. I think the local DNS cache had bad data from my previous testing. It was quickly fixed by:

sudo killall -HUP mDNSResponder