en en en en en en en en en en en hbs_lat en fr en en it en en en en en en es en en en en en en en en fr en en en pt en en en en en en en en en en de es en en en en de de en en az en en
When testing for the correct behavior of the OWASP ModSecurity Core Rule Set, a popular Web Application Firewall rule set, I needed to send empty Acccept- and User-Agent headers. This is relatively simple on the command line with curl:
Pulling this off with libcurl (pycurl in my case), was way more difficult though. It is also not really documented properly, hence this little blog post.
The tricky bit is that the Accept- and the User-Agent header are already set by default. So when you add an instruction to create an empty header, it will just add a 2nd User-Agent header (which will then be concatenated with the first one by the webserver. At least on the one I tested.)
So what you need to do is removing the existing header (suffux “:”) and then adding it anew as an empty header (suffix “;”). The following worked for me on pycurl 7.43:
import pycurl c = pycurl.Curl() c.setopt(pycurl.URL, "https://example.com") c.setopt(pycurl.HTTPHEADER, ["User-Agent:"] + ["User-Agent;"]) c.perform()
Here is a dump of the network traffic documenting the request (against localhost in cleartext). This proves that this is indeed what we wanted to get.
0x0030: b7a9 3516 4745 5420 2f20 4854 5450 2f31 ..5.GET./.HTTP/1 0x0040: 2e31 0d0a 486f 7374 3a20 6c6f 6361 6c68 .1..Host:.localh 0x0050: 6f73 740d 0a41 6363 6570 743a 202a 2f2a ost..Accept:.*/* 0x0060: 0d0a 5573 6572 2d41 6765 6e74 3a0d 0a0d ..User-Agent:... 0x0070: 0a
Christian Folini / @ChrFolini on Twitter
← Announcing Cisco ACI APIC Support for Netdisco
Phone +41 31 974 08 08
Ansible apache automation Cisco core-rules Core Rule Set CRS CRS3 DDoS Django drupal enigma enigma2017 firewall ModRewrite modsecurity NCS nervecenter netdisco nftables NMS OIN OpenSource OWASP Top10 PostgreSQL Proxmox Python 3 QoS Risks Sampling Mode security ssl SSL/TLS Swiss Cyber Experts Switzerland syslog tcpdump tls tshark typo3 ubuntu wireshark zenoss