Software Dev Can't Be Automated – It’s a Creative Process with an Unknown Goal
Ever read an article and think, &quot;This author really should have stated their thesis and context better&quot;? Yeah, that&#x27;s this article for me. I read it a few times but it always came off as confused to me. Before getting to my long and perhaps rude breakdown (I tried to eliminate the rude bits), let&#x27;s ask: What does it mean to automate software development? I don&#x27;t think the author specifies this idea well enough so I&#x27;ll offer my own ideas.<p>There is an idea of 100% automation, with the computer spontaneously generating a program that both meets the users requirements and is delivered to them when they need it. This is unlikely without some degree of human intervention to at least state, &quot;I need to...&quot;. If this is what the author means, then sure, they&#x27;re correct. This is such a fundamentally absurd idea it would be impossible for me to even conceive of an argument well-grounded in physics and computational theory that could support it.<p>However, here&#x27;s my take on automation in the domain of software development that doesn&#x27;t sound totally absurd and is probably closer to what most people mean when they describe it: A system in which a user can provide a minimal specification and get a complex execution as a result. I offer SQL query planners as an example of this. The user needs to enter &quot;SELECT * FROM foo;&quot; and the planner does the rest. The user doesn&#x27;t <i>need</i> to do anything else (optimizations notwithstanding). From the perspective of an intelligent data analyst but &quot;non-programmer&quot; (in their perspective at least), the program is automatically developed to meet their needs from this point forward. It is a reasonable response to say that the SQL query is a program, but it is such a minimal specification of a program that the specifier can be totally unaware of whether the database is on their machine, in their local network, on the other side of the planet or on Mars, if it&#x27;s a single database or sharded for performance, or any number of other considerations. The development process has been automated beyond this level of abstraction: The user merely needs to know the relational model of their data and the syntax of SQL and they can accomplish grand things.<p>Now, the rest of this gets a bit nitpicky and at some point I started to feel like a real asshole so I didn&#x27;t finish the paragraph by paragraph rebuttal (aside: paragraphs should be longer than tweets), also it got too long for HN and on submission it told me so. Keep the above idea of automation (mine) in mind, because if you keep the straw man version in mind then the author is, of course, correct. But it&#x27;s also a kind of useless way of being correct when the notion you&#x27;re responding to is so fundamentally absurd.<p>&gt; As much as people would like to automate software development, they can’t because it’s a creative and collaborate process.<p>Which people and to what extent do they mean to automate it? You&#x27;ve posed that they exist, explain who they are or at least what they actually mean or provide their words so that we can understand the context of your article.<p>&gt; It’s uncommon to think of developers as creative individuals but more technical people writing code. The common belief is developers take rules and automate them.<p>Is it really? At least on this forum it&#x27;s not. I mean, I do get this from managers at times. I had one who said, only loosely paraphrased, &quot;Software is easy, you just have to type. Why is this taking you so long?&quot; Fortunately they seem to be a dwindling group in my circle.<p>&gt; Developers can only create software to do what you tell it.<p>This would also be true of a program synthesis tool.<p>&gt; You can’t automate creativity and where the final software is unknown.<p>I mean, I guess it depends on what we mean by &quot;creativity&quot;. What is creativity? Per Wikipedia we get this sense of the word: Creativity is a phenomenon whereby something somehow new and somehow valuable is formed.<p>Huh, a phenomenon whereby something new and valuable is formed. I&#x27;m pretty sure program synthesis is a thing. Now, you could argue that providing the requirements and letting the computer generate a program from them is in fact an act of programming. I think that&#x27;s reasonable, but it would be a significant amount of automation. To be able to go from a mere statement about what a program <i>should</i> to a program that <i>does</i> without any more developer interaction would seem to be automating software development to me.<p>In the section titled <i>Why can’t you automate software development</i> the author concludes with:<p>&gt; You can’t automate the creation of software because the requirements are not known.<p>The rest of the section has no relation to the heading. I reread it several times trying to figure out how those sentences and bulleted items related to it, but could not. Maybe someone else can tell me how. The only relevant piece is the sentence I quoted. To which I&#x27;ll say: Umm, no.<p>The requirements may not be known in full, and they may not be known at the start. However, this statement is presented as a universal truth, when it is not. I know the requirements for my software at work, but perhaps that&#x27;s just because my customers know the requirements for the software. They know the requirements because they know their domain. They know the systems they have to interact with. They know the tasks that have to be done and consequently know the tasks and features they need us to implement. I&#x27;ll admit, this may be a rare situation, but it&#x27;s not unique.<p>All that said, if the requirements are not known for your system, then certainly you can&#x27;t automatically create it because you don&#x27;t know what to create. You can&#x27;t ask a computer to synthesize a program that you can&#x27;t even describe. That is a reasonable statement.<p>&gt; You can’t automate the creation of software because it needs to work in a way that makes it easy for the people to use the software.<p>The former does not follow from the latter.<p>&gt; Software is a tool to aid people working, to help them to their jobs. The goal isn’t to make software; the goal is to make software that makes work easier.<p>I can agree with this in a general sense. Making software is rarely the objective itself, it&#x27;s true. Making software that accomplishes some meaningful work is the more typical goal (especially in businesses where there is a dollar amount associated with the work). Nothing to do with automation, though.<p>&gt; You don’t know how software needs to work until users use it because many of the rules in a company will conflict.<p>Two things here:<p>First, finally you express your domain. Business process software, which is perhaps one of the software domains closest to being automated. BPMN has been around for a while, and not really caught on as much as Appian and others may like, but if you go with my notion of a minimal specification, this is a domain ripe for automation.<p>Second, if you don&#x27;t know how software needs to work until it&#x27;s being used then you don&#x27;t know what you&#x27;re making. We&#x27;re back to my earlier statement, you can&#x27;t synthesize what you can&#x27;t describe. Short of a stochastic method which synthesizes many programs, at least, for users to test out. Genetic programming BPMN could be entertaining to observe as chaos spreads through the organization and BPMN diagrams are mated and mutated.<p>&gt; Actions and software can have unintended consequences that are only visible when they happen and then you decide to change the software.<p>&gt; Businesses evolve, changing and adapting to improve the way they work. Software goes on a similar journey when you create it. What seems a great idea on paper or as requirements might not work as software.<p>Program synthesis does not bar evolution.<p>&gt; Finding out the final requirements and how software should work is the journey, it’s the creative process that involves many iterations. You can’t automate creativity because it can’t be broken down into steps and rules.<p>Again, depends on your notion of automation. If it&#x27;s the spontaneous creation of just the right program, sure. But if it&#x27;s about having a minimal specification that can then be used to synthesize a more complex program, then you can iterate on that specification itself. Without needing to get into the lower level systems (again, optimization notwithstanding).<p>&gt; You can’t accurately plan how long it will take to create software because you cannot say exactly how the software needs to work or how many iterations it will take to get it right.<p>Not related to automation or lack thereof. See Swift and the exploding compiler times.<p>&gt; Software developers take the requirements and create what they think the users need. The users then use it and give feedback. There is subjective and there is interpretation and assumptions that lead to the wrong software created.<p>No reason that a computer can&#x27;t take the requirements and create what it thinks the user needs (per the mechanisms provided to it to produce a working solution). No reason that a user can&#x27;t give feedback to a computer.<p>&gt; The wrong software is progress because it gives us some definitive to build on and update. This is a step towards creating the correct software.<p>Good news! Automation is almost certain to make the wrong software at times, also giving us something to build on and update.