en en en pt es en en en en cv az en en en en en en en en en en en en en ca es en es ne en en en fr en en en en en en en en fr en en en en en en en en en en en sl en en en en en en en en en en en en en en en en en en en en en en en en en fr en zh nl en en en en en en en en en en en en en en it en en en en en en en pt en en en en en en en en en
This blog post is a public service announcement (or maybe a reminder) for anyone working on a programming project that they already have, or intend to release to the public, be it as something open source, a pet project, or a startup.
I’ve worked on a few successful open source projects over the last 10 years. I would consider these projects successful in that they got hundreds of stars on GitHub and each attracted multiple open source contributors. I actually shut down one of these project because reviewing pull requests alone was becoming a second unpaid job on top of my regular job, taking multiple hours out of my evenings after work, which became exhausting, but that’s a story for another post. What I want to tell you about today is something important, that I believe any developer should know, but I personally didn’t understand until I had been working on open source projects for a few years.
As part of my PhD, I developed Higgs, an experimental JIT compiler for JavaScript written using the D programming language. I developed it on GitHub, completely in the open, and wrote about my progress on this blog. Pretty soon, the project had 300 stars on GitHub, a handful of open source contributors, and I was receiving some nice feedback. It made me happy to have so many people taking notice of my research work. As part of Higgs, I had written my own x86 machine code generator, which enabled it to do machine code pirouettes LLVM couldn’t. I did all my development on Linux, but had done my best to keep the code as portable as possible, and so, that code would work fine on MacOS as well, I had assumed. Unfortunately, I was wrong.
About a year into its development, Higgs had enough of a small community that it made sense to have create a chat room to exchange with other contributors and users. About a dozen people joined over the next two months. One day, someone I had been exchanging with on the chat room for two weeks reached out to me to signal a strange bug. They couldn’t get the tests to pass and were getting a segmentation fault. I was puzzled. They asked me if Higgs had MacOS support. I explained that I’d never tested it on MacOS myself, but I couldn’t see any reason why it wouldn’t work. I told this person that the problem was surely on their end. Higgs had been open source for over a year. It was a pretty niche project, but I knew for a fact that at least 40-60 people must have tried it, and at least 50% of these people must have been running MacOS. I assumed that surely, if Higgs didn’t run on MacOS at all, someone would have opened a GitHub issue by now. Again, I was wrong.
The problem, it turned out, was that MacOS had more strict requirements for keeping the stack pointer aligned. This wasn’t difficult to fix. The more important lesson, that I didn’t understand until that point, is that you can’t count on the people trying your project to quickly and reliably signal bugs to you. Most of the time, if it doesn’t work, they won’t report the problem. There are a few reasons why this might be:
They assume that someone else has already reported the problem, and there would be no point in saying anything. The bigger your project, the more likely people are to assume that someone else has already reported the issue.
They think the fault might be on their end, they may be confused and feel too embarrassed to reach out for help. Nobody wants to look stupid or get told to RTFM, and so, they choose silence.
They are just trying your project out of curiosity, and are in no way invested. They will find an alternative to your project, or go do something else.
It’s a horrifying thought, but it could be that for every one person who opens an issue on GitHub, 100 or more people have already tried your project, run into that same bug, and simply moved on. So, what can you do? You can encourage people to report bugs. I visibly write in my GitHub README that reporting bugs is encouraged and welcome: “Please tell me if something is wrong, you’re helping me make this project better.” Another obvious thing that you can do is to have robust automated testing. Some continuous integration services can automatically test on both Linux and Mac.
More broadly, if you want your project to be successful, I think it’s important to try and put yourself in the user’s shoes. Every once in a while, try installing your software from scratch, along with all the dependencies. Ideally, you want your installation process to be as simple and frictionless as possible. If your software requires the user to perform 20 steps to get it running, you’ll be losing potential users even before they have a chance to figure out if it works on their system or not. In general, I find that writing software with a minimalist design philosophy, minimizing external dependencies as much as is reasonable to do so, will help you avoid bugs, and streamline your installation process.
“Given enough eyeballs, all bugs are shallow.” — Raymond: The Cathedral and the Bazaar. 1999.
Had the same issue with bug reports and changed a fail message long ago here:
There’s a culture of assuming compilers/runtimes can’t be fixed and working around them (see C’s inline assembly). I can definitely relate to the, “I tried your project but encountered a bug but didn’t report it,” problem.
It seems like a very smart thing to do to have an error message that directs people to a link to report the issue, ideally with friendly language (the users reporting bugs are doing you a favor).
RNGesus (Peter) (@pmarreck) permalink
This is basically the Bystander Effect as applied to software:
If this effect is in fact, in effect, here, then the number of visible stars on your project might only amplify it, ironically!
I was looking for minimalist-ish systemd-free Linux distros for my RPi recently. I tried Void, it worked well, booted reeeally fast, but I couldn’t get 3d acceleration to work so I just moved on, tried Alpine instead. Turns out it has the same issue but this one I’m reporting (unless somebody beat me to it): the project runs a mailing list and an IRC channel while Void is on Github which means all the things I ‘love’: Microsoft, javascript, and Google’s reCAPTCHA. Nope. Absolutely nope.
I remember a talk by a (or is it ‘the’) developer of i3 (a window manager) where he mentioned that their bugtracker allows anonymous access exactly not to kill motivation to report.
Agreed. I wish more developers and/or maintainers would realize this.
I’ve had a vaguely similar insight awhile ago, perhaps back in the days when I’d learn [and internalize the consequences of] the Bystander Effect.
Being conscious of this, and enthusiastic about Open Source, I’d start reporting issues, as light as mere “friction points” during build/install, as part of my “trying the thing out”.
Imagine my disappointment when authors would dismiss the reports under various excuses, not recognizing that my intention is to help them, and at times even visibly skimming through the report without a second thought.
Certainly, not all authors have been like that. But please, don’t be those authors, and don’t allow your peers to be — if you care, of course.
New top story on Hacker News: They Might Never Tell You It’s Broken – Latest news
They Might Never Tell You It’s Broken – Hacker News Robot
Michael Tsai - Blog - They Might Never Tell You It’s Broken
Code that Doesn’t Rot | Pointers Gone Wild
Leave a Reply Cancel reply
Enter your comment here...
Fill in your details below or click an icon to log in:
Email (required) (Address never made public)
You are commenting using your WordPress.com account. ( Log Out /  Change )
You are commenting using your Twitter account. ( Log Out /  Change )
You are commenting using your Facebook account. ( Log Out /  Change )
Notify me of new comments via email.
Notify me of new posts via email.
« It’s Been Done Before
Balancing Fun, Stress and Profit »
Typed vs Untyped Virtual Machines
Minimalism in Programming Language Design
Democracy, Dictatorships and Access to Information
Code that Doesn’t Rot
The Internet Changed My Life
What’s Could You Use Tesla Bot For?
NoiseCraft: a Browser-Based Visual Programming Language for Sound & Music
Art Artificial Intelligence Artificial Life Assembly Biology Compilers Creativity D Language Electronics Future Game Development Genetic Algorithms Grad studies Graphics Higgs JavaScript Microcontrollers Microprocessors Music Programming Programming Languages Psychology Rant Robotics Tachyon Technology Uncategorized work x86 Zeta
Archives Select Month June 2022  (1) May 2022  (1) March 2022  (1) February 2022  (2) January 2022  (2) December 2021  (1) June 2021  (1) February 2021  (2) December 2020  (1) September 2020  (1) January 2020  (1) November 2019  (2) October 2019  (2) June 2019  (1) January 2019  (1) December 2018  (1) May 2018  (2) February 2018  (1) January 2018  (1) December 2017  (1) October 2017  (1) June 2017  (1) April 2017  (1) January 2017  (1) December 2016  (1) November 2016  (2) September 2016  (2) May 2016  (1) March 2016  (1) February 2016  (2) January 2016  (1) November 2015  (4) October 2015  (1) September 2015  (3) August 2015  (2) July 2015  (3) June 2015  (1) May 2015  (3) April 2015  (2) March 2015  (1) February 2015  (1) December 2014  (1) November 2014  (2) October 2014  (2) September 2014  (3) July 2014  (3) May 2014  (1) April 2014  (1) March 2014  (1) February 2014  (1) January 2014  (3) December 2013  (3) October 2013  (2) September 2013  (1) August 2013  (2) July 2013  (3) June 2013  (2) May 2013  (3) March 2013  (2) February 2013  (1) January 2013  (3) December 2012  (4) November 2012  (1) October 2012  (1) September 2012  (3) July 2012  (4) June 2012  (2) May 2012  (2) April 2012  (4) March 2012  (2) February 2012  (3) January 2012  (6) December 2011  (2) November 2011  (3) October 2011  (8) September 2011  (4) August 2011  (3)