Critique of Lazy Sequences in Clojure
TXR Lisp also fails this test:<p><pre><code>  1&gt; (len
       (with-stream (s (open-file &quot;&#x2F;usr&#x2F;share&#x2F;dict&#x2F;words&quot;))
         (get-lines s)))
  ** error reading #&lt;file-stream &#x2F;usr&#x2F;share&#x2F;dict&#x2F;words b7ad7270&gt;: file closed
  ** during evaluation of form (len (let ((s (open-file &quot;&#x2F;usr&#x2F;share&#x2F;dict&#x2F;words&quot;)))
                                      (unwind-protect
                                        (get-lines s)
                                        (close-stream s))))
  ** ... an expansion of (len (with-stream
                                (s (open-file &quot;&#x2F;usr&#x2F;share&#x2F;dict&#x2F;words&quot;))
                                (get-lines s)))
  ** which is located at expr-1:1
</code></pre>
The built-in solution is that when you create a lazy list which reads lines from a stream, that lazy list takes care of closing the stream when it is done.<p>If the lazy list isn&#x27;t processed to the end, then the stream semantically leaks; it has to be cleaned up by the garbage collector when the lazy list becomes unreachable.<p>We can see with strace that the stream is closed:<p><pre><code>  $ strace txr -p &#x27;(flow &quot;&#x2F;usr&#x2F;share&#x2F;dict&#x2F;words&quot; open-file get-lines len)&#x27;
  [...]read(3, &quot;d\nwrapper\nwrapper&#x27;s\nwrappers\nwra&quot;..., 4096) = 4096
  read(3, &quot;zigzags\nzilch\nzilch&#x27;s\nzillion\nzi&quot;..., 4096) = 826
  read(3, &quot;&quot;, 4096)                       = 0
  close(3)                                = 0
  fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
  write(1, &quot;102305\n&quot;, 7102305
  )                 = 7
  exit_group(0)                           = ?
  +++ exited with 0 +++
</code></pre>
It is possible to address the error issue with reference counting. Suppose that we define a stream with a reference count, such that it has to be closed that many times before the underlying file descriptor is closed.<p>I programmed a proof of concept of this today. (I ran into a small issue in the language run-time that I fixed; the close-stream function calls the underlying method and then caches the result, preventing the solution from working.)<p><pre><code>  (defstruct refcount-close stream-wrap
    stream
    (count 1)

    (:method close (me throw-on-error-p)
      (put-line `close called on @me`)
      (when (plusp me.count)
        (if (zerop (dec me.count))
          (close-stream me.stream throw-on-error-p)))))

  (flow
    (with-stream (s (make-struct-delegate-stream
                      (new refcount-close
                           count 2
                           stream (open-file &quot;&#x2F;usr&#x2F;share&#x2F;dict&#x2F;words&quot;))))
      (get-lines s))
    len
    prinl)
</code></pre>
With my small fix in stream.c (already merged, going into Version 292), the output is:<p><pre><code>  $ .&#x2F;txr lazy2.tl
  close called on #S(refcount-close stream #&lt;file-stream &#x2F;usr&#x2F;share&#x2F;dict&#x2F;words b7aecee0&gt; count 2)
  close called on #S(refcount-close stream #&lt;file-stream &#x2F;usr&#x2F;share&#x2F;dict&#x2F;words b7aecee0&gt; count 1)
  102305
</code></pre>
One close comes from the with-stream macro, the other from the lazy list hitting EOF when its length is being calculated.<p>Without the fix, I don&#x27;t get the second call; the code works, but the descriptor isn&#x27;t closed:<p><pre><code>  $ txr lazy2.tl
  close called on #S(refcount-close stream #&lt;file-stream &#x2F;usr&#x2F;share&#x2F;dict&#x2F;words b7b70f10&gt; count 2)
  102305
</code></pre>
In the former we see the call to close in strace; in the latter we don&#x27;t.