<p dir="auto">Currently we use the Rust parts of <a href="https://docs.rs/dprint/0.1.0-alpha/dprint/" rel="nofollow">dprint</a> to power <code class="notranslate">deno fmt</code>, but we utilise <a href="https://prettier.io/" rel="nofollow">prettier</a> for formatting as part of our toolchain.</p>
<p dir="auto">Is suspect this means we would need to create a standalone minimalistic binary that wraps dprint to do the file system bindings.</p>
<p dir="auto">cc/ <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/dsherret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/dsherret">@dsherret</a></p>
<p dir="auto">I'd like to implement a <code class="notranslate">Dir</code> class, and functions <code class="notranslate">opendir</code>/<code class="notranslate">opendirSync</code> that construct it. Most of this looks to be easy to implement, but there are some design choices it'd be helpful to get feedback on first. (Rather than implementing things one way, then needing to go back and redo it.)</p>
<p dir="auto">As a reminder, here are some things currently in master:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/** A FileInfo describes a file and is returned by `stat`, `lstat`,
 * `statSync`, `lstatSync`. A list of FileInfo is returned by `readdir`,
 * `readdirSync`. */
export interface FileInfo {
  size: number;
  modified: number | null;
  accessed: number | null;
  created: number | null;
  name: string | null; // not populated by stat/lstat, only by readdir
  isFile(): boolean;
  isDirectory(): boolean;
  isSymlink(): boolean;
  // rest are Linux/Mac only
  dev: number | null;
  ino: number | null;
  mode: number | null;
  nlink: number | null;
  uid: number | null;
  gid: number | null;
  rdev: number | null;
  blksize: number | null;
  blocks: number | null;
}

export class File
  implements
    Reader,
    SyncReader,
    Writer,
    SyncWriter,
    Seeker,
    SyncSeeker,
    Closer {
  readonly rid: number;
  constructor(rid: number);
  write(p: Uint8Array): Promise&lt;number&gt;;
  writeSync(p: Uint8Array): number;
  read(p: Uint8Array): Promise&lt;number | EOF&gt;;
  readSync(p: Uint8Array): number | EOF;
  seek(offset: number, whence: SeekMode): Promise&lt;number&gt;;
  seekSync(offset: number, whence: SeekMode): number;
  close(): void;
}"><pre class="notranslate"><code class="notranslate">/** A FileInfo describes a file and is returned by `stat`, `lstat`,
 * `statSync`, `lstatSync`. A list of FileInfo is returned by `readdir`,
 * `readdirSync`. */
export interface FileInfo {
  size: number;
  modified: number | null;
  accessed: number | null;
  created: number | null;
  name: string | null; // not populated by stat/lstat, only by readdir
  isFile(): boolean;
  isDirectory(): boolean;
  isSymlink(): boolean;
  // rest are Linux/Mac only
  dev: number | null;
  ino: number | null;
  mode: number | null;
  nlink: number | null;
  uid: number | null;
  gid: number | null;
  rdev: number | null;
  blksize: number | null;
  blocks: number | null;
}

export class File
  implements
    Reader,
    SyncReader,
    Writer,
    SyncWriter,
    Seeker,
    SyncSeeker,
    Closer {
  readonly rid: number;
  constructor(rid: number);
  write(p: Uint8Array): Promise&lt;number&gt;;
  writeSync(p: Uint8Array): number;
  read(p: Uint8Array): Promise&lt;number | EOF&gt;;
  readSync(p: Uint8Array): number | EOF;
  seek(offset: number, whence: SeekMode): Promise&lt;number&gt;;
  seekSync(offset: number, whence: SeekMode): number;
  close(): void;
}
</code></pre></div>
<p dir="auto">PRs I have queued in the <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="566419880" data-permission-text="Title is private" data-url="https://github.com/denoland/deno/issues/4017" data-hovercard-type="issue" data-hovercard-url="/denoland/deno/issues/4017/hovercard" href="https://github.com/denoland/deno/issues/4017">#4017</a> series will propose these changes:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="export interface FileInfo {
  ...
  type: FileType | null;
  // rest are Linux/Mac only
  ...
  mode: number | null; // masked to 0o7777
}

export enum FileType {
  TYPE_UNKNOWN = 14 &lt;&lt; 12,
  TYPE_REGULAR = 8 &lt;&lt; 12,
  TYPE_DIRECTORY = 4 &lt;&lt; 12,
  TYPE_SYMLINK = 10 &lt;&lt; 12,
  TYPE_SOCKET = 12 &lt;&lt; 12,
  TYPE_FIFO = 1 &lt;&lt; 12,
  TYPE_CHARDEV = 2 &lt;&lt; 12,
  TYPE_BLKDEV = 6 &lt;&lt; 12
}

export class File {
  ...
  sync(): void;
  datasync(): void;
  truncate(len?: number): Promise&lt;void&gt;;
  truncateSync(len?: number): void;
  chmod(mode: number): Promise&lt;void&gt;;
  chmodSync(mode: number): void;
  utime(atime: number | Date, mtime: number | Date): Promise&lt;void&gt;;
  utimeSync(atime: number | Date, mtime: number | Date): void;
  // chown(uid: number, gid: number): Promise&lt;void&gt;;
  // chownSync(uid: number, gid: number): void;
  stat(): Promise&lt;FileInfo&gt;;
  statSync(): FileInfo;
}"><pre class="notranslate"><code class="notranslate">export interface FileInfo {
  ...
  type: FileType | null;
  // rest are Linux/Mac only
  ...
  mode: number | null; // masked to 0o7777
}

export enum FileType {
  TYPE_UNKNOWN = 14 &lt;&lt; 12,
  TYPE_REGULAR = 8 &lt;&lt; 12,
  TYPE_DIRECTORY = 4 &lt;&lt; 12,
  TYPE_SYMLINK = 10 &lt;&lt; 12,
  TYPE_SOCKET = 12 &lt;&lt; 12,
  TYPE_FIFO = 1 &lt;&lt; 12,
  TYPE_CHARDEV = 2 &lt;&lt; 12,
  TYPE_BLKDEV = 6 &lt;&lt; 12
}

export class File {
  ...
  sync(): void;
  datasync(): void;
  truncate(len?: number): Promise&lt;void&gt;;
  truncateSync(len?: number): void;
  chmod(mode: number): Promise&lt;void&gt;;
  chmodSync(mode: number): void;
  utime(atime: number | Date, mtime: number | Date): Promise&lt;void&gt;;
  utimeSync(atime: number | Date, mtime: number | Date): void;
  // chown(uid: number, gid: number): Promise&lt;void&gt;;
  // chownSync(uid: number, gid: number): void;
  stat(): Promise&lt;FileInfo&gt;;
  statSync(): FileInfo;
}
</code></pre></div>
<p dir="auto">The <code class="notranslate">chown</code> functions are commented because I haven't implemented them yet, and am not sure whether the machinery is there to let me do so.</p>
<p dir="auto">For <code class="notranslate">opendir</code> and friends, I was thinking something like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="export class Dir
  implements
    Closer {
  readonly rid: number;
  constructor(rid: number);
  close(): void;
  chmod(mode: number): Promise&lt;void&gt;;
  chmodSync(mode: number): void;
  utime(atime: number | Date, mtime: number | Date): Promise&lt;void&gt;;
  utimeSync(atime: number | Date, mtime: number | Date): void;
  // chown(uid: number, gid: number): Promise&lt;void&gt;;
  // chownSync(uid: number, gid: number): void;
  stat(): Promise&lt;FileInfo&gt;;
  statSync(): FileInfo;
  chdir(): void;
  readdir(): Promise&lt;FileInfo[]&gt;;
  readdirSync(): FileInfo[];
}

function opendir(path: string): Promise&lt;Dir&gt;;
function opendirSync(path: string): Dir;"><pre class="notranslate"><code class="notranslate">export class Dir
  implements
    Closer {
  readonly rid: number;
  constructor(rid: number);
  close(): void;
  chmod(mode: number): Promise&lt;void&gt;;
  chmodSync(mode: number): void;
  utime(atime: number | Date, mtime: number | Date): Promise&lt;void&gt;;
  utimeSync(atime: number | Date, mtime: number | Date): void;
  // chown(uid: number, gid: number): Promise&lt;void&gt;;
  // chownSync(uid: number, gid: number): void;
  stat(): Promise&lt;FileInfo&gt;;
  statSync(): FileInfo;
  chdir(): void;
  readdir(): Promise&lt;FileInfo[]&gt;;
  readdirSync(): FileInfo[];
}

function opendir(path: string): Promise&lt;Dir&gt;;
function opendirSync(path: string): Dir;
</code></pre></div>
<p dir="auto">The factory functions (<code class="notranslate">opendir</code>/<code class="notranslate">opendirSync</code>) would throw an error if <code class="notranslate">path</code> didn't refer to a directory. (I'd suggest at the same time <code class="notranslate">open</code> and <code class="notranslate">openSync</code> should be made to throw an error if <code class="notranslate">path</code> <em>did</em> refer to a directory. Though perhaps not if it refers to a other things that are neither directories nor files.) We could use the existing <code class="notranslate">op_open</code> to implement all of these.</p>
<p dir="auto">If that all sounds reasonable, the main strategic questions remaining have to do with <code class="notranslate">readdir</code> and <code class="notranslate">FileInfo</code>.</p>
<ol dir="auto">
<li>
<p dir="auto">Ideally, there'd be a version of <code class="notranslate">std::fs::read_dir</code> in the Rust libs that took a <code class="notranslate">RawFd</code> as argument rather than a <code class="notranslate">Path</code>. And also a tokio-ized version of the same. But sadly there is neither. I tried poking around in <code class="notranslate">libstd/sys/unix/fs.rs</code> in the Rust sources to see how hard this would be to implement. But to get it to compile, I had to keep pulling in duplicates of more and more of the internal machinery of that file. In the end this didn't seem a promising way to go. Plus it would involve lots of calling into the <code class="notranslate">libc</code> crate with <code class="notranslate">unsafe</code> code blocks. (As the Rust std libs must.)</p>
</li>
<li>
<p dir="auto">We could work on getting Rust to include such functionality in their std libs, and even offer them an implementation, but I suspect even if they're open to merging that in the near future it would take a while before it's exposed in a version of Rust we're using.</p>
</li>
<li>
<p dir="auto">We're already using the <code class="notranslate">nix</code> package for some things on the Unix side (Linux/Mac). And they have an interface <code class="notranslate">nix::dir::Dir</code>, which can be constructed using either paths or fds. That looks good for our purposes. (Downsides: it's Unix-only, and also blocking, so we'd have to continue using <code class="notranslate">blocking_json</code> from <code class="notranslate">cli/ops/dispatch_json.rs</code> here, rather than moving to async functions from <code class="notranslate">tokio::fs</code>, as proposed in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="573332164" data-permission-text="Title is private" data-url="https://github.com/denoland/deno/issues/4188" data-hovercard-type="issue" data-hovercard-url="/denoland/deno/issues/4188/hovercard" href="https://github.com/denoland/deno/issues/4188">#4188</a>.) I think this is the most promising way to go.</p>
</li>
<li>
<p dir="auto">Some issues about the <code class="notranslate">nix::dir::Dir</code> implementation though. The iterator over directory entries that it provides is sparser than the one provided by <code class="notranslate">std::fs::read_dir</code> (and its counterpart in <code class="notranslate">tokio::fs</code>). It only provides something like this for each entry:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="file_name(): string;
ino(): number
file_type(): FileTypeEnum | null;"><pre class="notranslate"><code class="notranslate">file_name(): string;
ino(): number
file_type(): FileTypeEnum | null;
</code></pre></div>
<p dir="auto">The <code class="notranslate">file_type</code> information is only populated by <code class="notranslate">nix::dir::Dir</code> if it's available without making an extra <code class="notranslate">lstat</code> system call. (Some Linux filesystems would require that extra call. I don't know which ones.) So what's naturally available to us if we go this way would only be a sparser version of our existing <code class="notranslate">FileInfo</code>. At the same time, it would be more efficient than the <code class="notranslate">std::fs::read_dir</code> version, which goes ahead and does the extra <code class="notranslate">lstat</code> call to fill in the additional fields, which won't always be needed. Of course, we could go ahead and do the same with the results of <code class="notranslate">nix</code>'s directory iterator (using <code class="notranslate">nix::dir::Dir::openat(dirfd, filename, ...)</code>). But that seems wasteful. Why not instead just provide the minimal information, and give the caller the means to retrieve more info if they want it. (Using calls like <code class="notranslate">fstatat</code>, <code class="notranslate">openat</code> etc that are also present in <code class="notranslate">nix</code>. I'm assuming that from a <code class="notranslate">Dir</code> class, our implementation will only have a <code class="notranslate">rid</code> saved from which it can get an open dirfd. It won't have --- and there need not even anymore exist --- a valid path that could be used to reach that directory.)</p>
</li>
<li>
<p dir="auto">If the strategy just sketched seems attractive for <code class="notranslate">opendir(path).readdir()</code>, perhaps we should also consider it for <code class="notranslate">readdir(path)</code>. That is, have this not provide a full <code class="notranslate">FileInfo</code> like one gets from <code class="notranslate">stat</code>/<code class="notranslate">lstat</code>, but a sparser thing like described above.</p>
<p dir="auto">That is in fact what other languages/hosts do in their std libs.</p>
<ul dir="auto">
<li>Node's <code class="notranslate">fs.DirEnt</code> has the fields: <code class="notranslate">name: string|Buffer</code>, <code class="notranslate">isFile(): boolean</code>, <code class="notranslate">isDirectory(): boolean</code>, and so on.</li>
<li>Python's <code class="notranslate">os.DirEntry</code> has the fields <code class="notranslate">name: bytes|str</code>, <code class="notranslate">path: bytes|str</code>, <code class="notranslate">inode()</code>, <code class="notranslate">is_dir(follow_symlinks=False)</code>, <code class="notranslate">is_file(follow_symlinks=False)</code>, <code class="notranslate">is_symlink()</code>, and <code class="notranslate">stat(follow_symlinks=False)</code></li>
<li>Go's <code class="notranslate">os.Readdir</code> returns a list of <code class="notranslate">FileInfo</code> structures, same as is returned by its <code class="notranslate">os.Lstat</code> and <code class="notranslate">os.Stat</code>. But this is somewhat sparser than ours; and they also have the function <code class="notranslate">os.Readdirnames</code> that returns just a list of names.</li>
</ul>
</li>
<li>
<p dir="auto">I'm ignoring in all of this the issues raised in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="577406885" data-permission-text="Title is private" data-url="https://github.com/denoland/deno/issues/4277" data-hovercard-type="issue" data-hovercard-url="/denoland/deno/issues/4277/hovercard" href="https://github.com/denoland/deno/issues/4277">#4277</a> and <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="574053077" data-permission-text="Title is private" data-url="https://github.com/denoland/deno/issues/4218" data-hovercard-type="issue" data-hovercard-url="/denoland/deno/issues/4218/hovercard" href="https://github.com/denoland/deno/issues/4218">#4218</a>, which have to do with whether <code class="notranslate">readdir</code> provides its results all at once or as a stream. The latter seems ultimately desirable, but I don't know how to make the Deno Rust guts save the state needed to implement an iterator. In any case, I think those issues are orthogonal to these.</p>
</li>
</ol>