[null, {"document_url": "https://polysync.io/blog/byo-standard-an-explorers-guide-to-compiler-plugins", "unformatted_src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382e22b6a288d9b79546a/1525908200689/linkedin.png", "src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382e22b6a288d9b79546a/1525908200689/linkedin.png", "formatted_filename": "linkedin", "alt_text": "linkedin.png"}, {"document_url": "https://polysync.io/blog/byo-standard-an-explorers-guide-to-compiler-plugins", "unformatted_src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382d1562fa780cd18c616/1525908187733/twitter.png", "src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382d1562fa780cd18c616/1525908187733/twitter.png", "formatted_filename": "twitter", "alt_text": "twitter.png"}, {"document_url": "https://polysync.io/blog/byo-standard-an-explorers-guide-to-compiler-plugins", "unformatted_src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382be03ce6449ddb19cb6/1525908172985/reddit.png", "src": "https://static1.squarespace.com/static/5a60ec649f8dce866f011db6/t/5af382be03ce6449ddb19cb6/1525908172985/reddit.png", "formatted_filename": "reddit", "alt_text": "reddit.png"}]
← On-Ramp

BYO Standard: An Explorer's Guide to Compiler Plugins

Shea Newton

For PolySync, a runtime panic means the potential for an unsafe situation on the road, and with that in mind, we’ve explored ways to restrict that potential. Of course, we aren’t the only ones thinking about ways to improve the quality of code at compile time by enforcing the right rules for the job. Active projects like rust-clippy are working to do that too by providing lints to supplement the rustc defaults.

In this post we’ll explore how to enforce a rule by prohibiting a practice we’ve formed an opinion about, the indexing of a vector or an array. Preventing indexing represents the potential for a couple of wins:

Lets get started.

Choosing Your Lint Level

Lint levels allow you to configure the severity of a rule violation with the help of rustc.

One size doesn’t fit all and these decisions should be tailored to your project’s specific requirements.

A quick warning about “forbid”: it should be used sparingly. Forbidding everything can prove troublesome, for instance, forbidding calls to unwrap. While it may be feasible to disallow it in your own code, it’s not uncommon for macros exposed by external crates to hide calls to unwrap. Rather than finding another crate, or rewriting it internally to avoid all calls to unwrap, it might be acceptable to implement warning or deny policies that require an explicit allow.

Exploring the AST

There are at least two options for exploring a Rust program's AST. The -Z ast-json flag passed to rustc helps with getting a feel for the general look and structure of the AST. Using it looks like this:

This post uses print statements as they provide context about when the data we’re looking for is accessible (i.e. is EarlyLintPass or LateLintPass right for me?). Because the syntax::ast types implement Debug, they also provided type information that can be a little more parsable.

First, make sure you're using the nightly toolchain:

Now setup a clean linting crate, here named “customlints”:

Next, we’ll do some setup in the customlints/src/lib.rs file.

Now, we can implement rustc::lint::EarlyLintPass and rustc::lint::LateLintPass for some preliminary examination.

In order to incorporate this exploratory plugin, we can create another crate, a [[bin]] this time:

Then point to our linter in that project's Cargo.toml. (Note the optional = true , we’ll revisit that later.):

Next, we can fill in the main function:

Time to build. Note that the dependency on the customlints crate is optional, so it needs to be enabled in the build command:

Looking closely, we can see what looks like our indexing operation in our output. Here's one:

And later, another:

Now, we can modify our print statements in order to unpack the hir::Expr a bit more:

To minimize noise, we can comment out our late pass output:

After building our [[bin]] project again, there's some information that looks promising, the following Index type that contains x and 10 :

Enforcing a Lint

Using the compiler plugin docs and the docs for syntax::ast let's lint the Index type. First, we'll need a few modifications. We’ll begin by declaring the lint with a Deny qualification. This means a program that indexes will fail to compile, but if necessary, can be allowed with #![allow(indexing_lint)].

Then, we'll populate that empty lint_array in the impl LintPass for EarlyPass that we mentioned earlier.

Finally, we can define when our lint occurs (any occurance of the Index type) and the report it provides. Replace that EarlyPass print statement as follows (feel free to remove the references to LatePass, we're only going to use EarlyPass from here on out):

After building our [[bin]] project again, tada!

We've implemented a lint.

Fine Tuning

Now, in an attempt to eliminate unintended side effects, we need to stress the edges (or even just look for them). For instance, maybe since let _b = &x[..] won't cause a runtime panic we decide to allow its behavior. Let’s add that to our [[bin]] project and build it.

Sure enough, we're denying behavior we've decided we don't want to. By putting our print statement back we can take another look at some debugging output (informed by the syntax::ast::ExprKind).

That provides the following:

But what do we do with that? One option is experimenting with our [[bin]] project to see if we can get some more context. This can be achieved with a few more indexing operations.

Here's the main function.

Now let's look at the output and see what we can tell.

Because of the Int(10, Unsuffixed), it looks like it corresponds to let _a = x[10];:

Assuming the output occurs in the same order as the indexing operations in the source code, &x[..] probably corresponds to:

and &x[10..] to:

and &x[..10] to:

and &x[10..100] to:

The first two values in that Range type correspond to Some if there is a position defined and None if there isn't. We still need to disallow cases where there is potential for out-of-bounds access (all but the last case), so the next step ends up looking like the following:

And that did it! All that’s left are the errors for the behavior we decided to deny.

Continuing the Work

This lint introduces the potential for some false positives. If it’s possible to prove at compile time that an index is in range, then our error may be overstepping its utility. Taking this lint to the next level likely means allowing indexing in those cases. It may even be feasible to implement a MIR processing lint that triggers on all reachable panics. If that existed, a lint like ours could be phased out entirely.

How a Stable Product Uses Unstable Lints

Nightly Rust should be backward compatible with the last stable release. That means a stable project's build success can depend on nightly toolchain linting without having to worry about something unrelated breaking. Revisiting the setup we did for our [[bin]] example, we see that it allows us to opt out of linting on a stable toolchain.

First, the customlints dependency needs be optional in your Cargo.toml:

Then, any project that's subject to your custom lints can use cfg_attr to ensure the linting plugin isn't enabled unless the customlints feature is specified:

Using that set up, a nightly build can be invoked with:

and a stable build with:


It can be helpful to start this kind of exploratory effort by looking at other projects that also implement linting compiler plugins. Some of the places that have a lot to offer in that area are:

- The compiler plugin docs provide a lot of good context, the basics for getting started, and some example code. Using them, you should be able to fill in any blanks left in this post's code

- rust-clippy lint implementations, if they don't already offer the lint you want, they may already be doing something similar you can pull from.

- librustc_lint implementations is another great reference for linting logic that you can pull from.

-  The My lint-writing workflow post by llogiq is a great take on a similar approach to lint-writing.

Last Word

There are other powerful applications of compiler plugins too. llogiq maintains a couple of my favorites, mutagen and overflower. Mutagen is the foundation of a mutation testing framework that enables you to evaluate whether your tests work as they should. Overflower lets you decide what to do when integer overflow occurs.

Additionally, there is the possibility that others in the community will think your lints are useful. If you think that's the case, consider running them by the rust-clippy team.

While implementing lints can be an illuminating window into the compiler and a Rust program’s AST, compile time lints are a powerful approach to restricting program behavior. Because if a program with violations doesn’t compile, it can’t crash.

Share Shea's post: