<p>This is a work in progress answer with some content currently taken from the Google fuzzing <a href="https://github.com/google/fuzzing/blob/master/docs/intro-to-fuzzing.md#fuzzing-tools" rel="nofollow noreferrer">docs</a>.</p>
<p><strong>libFuzzer</strong></p>
<p>libFuzzer is a tool which allows you to write fuzz tests in a similar style to unit tests. For developers setting up fuzzing for the first time, the Google fuzzing team recommends it for its simplicity.</p>
<blockquote>
<p>[It] might be a good choice for testing libraries that have relatively small inputs, each input takes &lt; 10ms to run, and the library code is not expected to crash on invalid inputs. Examples: regular expression matchers, text or binary format parsers, compression, network, crypto.</p>
</blockquote>
<p>See <a href="http://llvm.org/docs/LibFuzzer.html#q-so-what-exactly-this-fuzzer-is-good-for" rel="nofollow noreferrer">http://llvm.org/docs/LibFuzzer.html#q-so-what-exactly-this-fuzzer-is-good-for</a> for more information on how libFuzzer compares to other fuzzers.</p>
<p><strong>AFL</strong></p>
<p>AFL is the tool that did the most to drive the early success of coverage-guided fuzzing. The Google fuzzing team does not recommend AFL for new projects since it is not as well-maintained as Honggfuzz or libFuzzer. That said, because of its widespread it is common to encounter existing projects which rely on it for fuzzing.</p>
<blockquote>
<p>Compared to other instrumented fuzzers, afl-fuzz is designed to be
practical: it has modest performance overhead, uses a variety of
highly effective fuzzing strategies and effort minimization tricks,
requires essentially no configuration, and seamlessly handles complex,
real-world use cases - say, common image parsing or file compression
libraries.</p>
</blockquote>
<p>See <a href="https://lcamtuf.coredump.cx/afl/" rel="nofollow noreferrer">https://lcamtuf.coredump.cx/afl/</a> for more information on how afl-fuzz compares to other fuzzers.</p>
<p><strong>Honggfuzz</strong></p>
<p>Honggfuzz is another fuzzing tool with a rich set of features available to it. The Google fuzzing team recommends it for security researchers looking for more customization options in their fuzzers, or developers on complex projects where working within libFuzzer's unit-test-like style is difficult. It allows users to specify how input files should be fed to a target program (e.g. on the command line, on stdin) or allows more complicated customizations such as feeding input to a network server directly.</p>

CC BY-SA 4.0
2020-11-11T14:29:20.760Z
alf is very much not applicable to bitcoin core.
Claris
CC BY-SA 4.0
2020-11-11T15:24:40.000Z
Why do you say that? It is in the Bitcoin Core fuzzing docs
Michael Folkson
CC BY-SA 4.0
2020-11-11T15:35:42.923Z
It’s not particularly useful for evaluating the sort of things in Bitcoin that actually matter. The consensus rules tests are partially derived from fuzzing but it isn’t ever going to find interesting bugs due to the amount of state bitcoin core has. That’s just opinion of course.
Claris
CC BY-SA 4.0
2020-11-11T18:51:28.183Z
@Anonymous I agree to an extent that there are many potential issues that are hard to find through fuzzing, but that seems true for fuzzing in general and not specific to AFL? Fuzzers have found some issues in the codebase, FWIW.
Pieter Wuille
CC BY-SA 4.0
2020-12-11T14:09:33.457Z
@Anonymous "It isn't going to find interesting bugs": that's untrue. AFL was used to find consensus bugs in other implementation of Bitcoin: https://github.com/jonasnick/bitcoinconsensus_testcases . Of course it's not a "bitcoin-core bug" but still pretty interesting bug-prone edgecases.
Antoine Poinsot
CC BY-SA 4.0
2020-11-11T13:28:22.460Z
2021-01-03T14:03:09.687Z
2021-01-03T14:03:09.687Z
Michael Folkson
<p>In the Bitcoin Core fuzzing <a href="https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md" rel="nofollow noreferrer">docs</a> there are setup instructions for three different fuzzers: <a href="https://llvm.org/docs/LibFuzzer.html" rel="nofollow noreferrer">libFuzzer</a>, <a href="https://github.com/google/afl" rel="nofollow noreferrer">american fuzzy lop</a> (afl-fuzz) and <a href="https://github.com/google/honggfuzz/blob/master/docs/USAGE.md" rel="nofollow noreferrer">Honggfuzz</a>. Presumably I should try them in this order. How do they compare? Should I use one particular fuzzer in one scenario and a different fuzzer in another scenario? When I engage in fuzz testing should I run all of them one after each other?</p>

<bitcoin-core-development><testing><bitcoincore-review><fuzz-test>
Which fuzzer should I use to fuzz test Bitcoin Core?
Michael Folkson
# Which fuzzer should I use to fuzz test Bitcoin Core?

(Asked by: username_0 on 2020-11-11)

In the Bitcoin Core fuzzing [docs](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) there are setup instructions for three different fuzzers: [libFuzzer](https://llvm.org/docs/LibFuzzer.html), [american fuzzy lop](https://github.com/google/afl) (afl-fuzz) and [Honggfuzz](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md). Presumably I should try them in this order. How do they compare? Should I use one particular fuzzer in one scenario and a different fuzzer in another scenario? When I engage in fuzz testing should I run all of them one after each other?

**Tags:** bitcoin-core-development, testing, bitcoincore-review, fuzz-test


---

## 1 answers

### Answer by username_0 on 2020-11-11. Score: 0 | Accepted answer

This is a work in progress answer with some content currently taken from the Google fuzzing [docs](https://github.com/google/fuzzing/blob/master/docs/intro-to-fuzzing.md#fuzzing-tools).

**libFuzzer**

libFuzzer is a tool which allows you to write fuzz tests in a similar style to unit tests. For developers setting up fuzzing for the first time, the Google fuzzing team recommends it for its simplicity.

> [It] might be a good choice for testing libraries that have relatively small inputs, each input takes < 10ms to run, and the library code is not expected to crash on invalid inputs. Examples: regular expression matchers, text or binary format parsers, compression, network, crypto.

See <http://llvm.org/docs/LibFuzzer.html#q-so-what-exactly-this-fuzzer-is-good-for> for more information on how libFuzzer compares to other fuzzers.

**AFL**

AFL is the tool that did the most to drive the early success of coverage-guided fuzzing. The Google fuzzing team does not recommend AFL for new projects since it is not as well-maintained as Honggfuzz or libFuzzer. That said, because of its widespread it is common to encounter existing projects which rely on it for fuzzing.

> Compared to other instrumented fuzzers, afl-fuzz is designed to be
> practical: it has modest performance overhead, uses a variety of
> highly effective fuzzing strategies and effort minimization tricks,
> requires essentially no configuration, and seamlessly handles complex,
> real-world use cases - say, common image parsing or file compression
> libraries.

See <https://lcamtuf.coredump.cx/afl/> for more information on how afl-fuzz compares to other fuzzers.

**Honggfuzz**

Honggfuzz is another fuzzing tool with a rich set of features available to it. The Google fuzzing team recommends it for security researchers looking for more customization options in their fuzzers, or developers on complex projects where working within libFuzzer's unit-test-like style is difficult. It allows users to specify how input files should be fed to a target program (e.g. on the command line, on stdin) or allows more complicated customizations such as feeding input to a network server directly.

> **Comments:**
> **username_1**: alf is very much not applicable to bitcoin core.  
> **username_0**: Why do you say that? It is in the Bitcoin Core fuzzing docs  
> **username_1**: It’s not particularly useful for evaluating the sort of things in Bitcoin that actually matter. The consensus rules tests are partially derived from fuzzing but it isn’t ever going to find interesting bugs due to the amount of state bitcoin core has. That’s just opinion of course.  
> **username_2**: @Anonymous I agree to an extent that there are many potential issues that are hard to find through fuzzing, but that seems true for fuzzing in general and not specific to AFL? Fuzzers have found some issues in the codebase, FWIW.  
> **username_3**: @Anonymous "It isn't going to find interesting bugs": that's untrue. AFL was used to find consensus bugs in other implementation of Bitcoin: https://github.com/jonasnick/bitcoinconsensus_testcases . Of course it's not a "bitcoin-core bug" but still pretty interesting bug-prone edgecases.  

---

