Answers: <p>Since <a href="https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver" rel="nofollow noreferrer"><code>ABFS</code></a> mentioned in the stacktrace as a caching filesystem, then I suppose two possible reasons and suggest solutions for them</p>
  
  <h2>Dependency mesh in <code>maven</code></h2>
  
  <p>As noted in two answers (<a href="https://stackoverflow.com/a/21118824/5720818">one</a>, <a href="https://stackoverflow.com/a/27532248/5720818">two</a>) to a <a href="https://stackoverflow.com/questions/17265002/hadoop-no-filesystem-for-scheme-file">similar question</a></p>
  
  <blockquote>
    <p>When we use maven-assembly-plugin, it merges all our JARs into one, and all META-INFO/services/org.apache.hadoop.fs.FileSystem overwrite each-other. Only one of these files remains (the last one that was added). In this case, the FileSystem list from hadoop-commons overwrites the list from hadoop-hdfs, so DistributedFileSystem was no longer declared.</p>
  </blockquote>
  
  <p>So, check your dependencies and if there are <code>JAR</code> with correct <code>org.apache.hadoop.fs.FileSystem</code> in your distibution.</p>
  
  <h2><code>PrestoDB</code> uses outdated <code>azure-site.xml</code></h2>
  
  <p>There is <a href="https://github.com/prestodb/presto/issues/11648" rel="nofollow noreferrer">known issue #11648</a>. 
  And as <a href="https://github.com/prestodb/presto/issues/11648#issuecomment-522853533" rel="nofollow noreferrer">said</a>, <code>ABFS</code>, <code>ADLS</code> and <code>ALDS gen2</code> was only introduced to <a href="https://github.com/prestosql/presto" rel="nofollow noreferrer">https://github.com/prestosql/presto</a>, not to <code>prestodb</code></p>
  
  <p>So, suggestion is to ask your question on <a href="https://prestosql.io/community.html" rel="nofollow noreferrer">Presto Community Slack</a></p>
  
  <h1>P.S. try <code>storediag</code></h1>
  
  <p>There is a <code>hadoop</code> manual on <code>Azure</code> <a href="https://github.com/apache/hadoop/blob/745a6c1e69b3699f6496a146afc48824dd735461/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md" rel="nofollow noreferrer">ABFS</a></p>
  
  <p>And among other tips <a href="https://github.com/apache/hadoop/blob/745a6c1e69b3699f6496a146afc48824dd735461/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md" rel="nofollow noreferrer">they recommend</a>:</p>
  
  <blockquote>
    <p>One useful tool for debugging connectivity is the <a href="https://github.com/steveloughran/cloudstore/releases" rel="nofollow noreferrer">cloudstore storediag utility</a>.</p>
    
    <p>This validates the classpath, the settings, then tries to work with the filesystem.</p>
  </blockquote>
  
Answers: CC BY-SA 4.0
Answers: 2020-04-21T17:56:19.153Z
Answers: Thanks @Yasen , the above suggestions helped me lot to resolve my issue. First i shifted to prestosql and used respective jar versions resolved my problem
Answers: Rocky Hai
Answers: CC BY-SA 4.0
Answers: 2020-04-20T06:05:02.087Z
Answers: 2020-04-20T06:05:02.087Z
Answers: Yasen
Body: <p>I am using <code>prestodb:0.234</code></p>
  
  <p>Dependency jars: </p>
  
  <pre><code>   azure-storage-2.0.0.jar
     hadoop-azure-3.2.1.jar
     hadoop-azure-datalake-3.2.1.jar 
     azure-data-lake-store-sdk-2.3.8.jar
  </code></pre>
  
  <p>I am getting below error while doing the querying.</p>
  
  <pre><code>com.facebook.presto.spi.PrestoException: cannot create caching file system
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration.lambda$getConfiguration$0(HiveCachingHdfsConfiguration.java:70)
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration$CachingJobConf.createFileSystem(HiveCachingHdfsConfiguration.java:94)
      at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:59)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at com.facebook.presto.hive.HdfsEnvironment.lambda$getFileSystem$0(HdfsEnvironment.java:73)
      at com.facebook.presto.hive.authentication.NoHdfsAuthentication.doAs(NoHdfsAuthentication.java:23)
      at com.facebook.presto.hive.HdfsEnvironment.getFileSystem(HdfsEnvironment.java:72)
      at com.facebook.presto.hive.HdfsEnvironment.getFileSystem(HdfsEnvironment.java:66)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:298)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:270)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:100)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:199)
      at com.facebook.presto.hive.util.ResumableTasks.safeProcessTask(ResumableTasks.java:47)
      at com.facebook.presto.hive.util.ResumableTasks.access$000(ResumableTasks.java:20)
      at com.facebook.presto.hive.util.ResumableTasks$1.run(ResumableTasks.java:35)
      at com.facebook.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
  Caused by: java.io.IOException: No FileSystem for scheme: abfs
      at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
      at org.apache.hadoop.fs.PrestoFileSystemCache.createFileSystem(PrestoFileSystemCache.java:114)
      at org.apache.hadoop.fs.PrestoFileSystemCache.getInternal(PrestoFileSystemCache.java:89)
      at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:62)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration.lambda$getConfiguration$0(HiveCachingHdfsConfiguration.java:66)
  </code></pre>
  
  <p>Any suggestions?</p>
  
Tags: <azure><java><hive>
Title: I was trying ti set up presto with azure gen2 , but facing the below issue when I am querying
UserDisplayName: Rocky Hai
ThreadText: # I was trying ti set up presto with azure gen2 , but facing the below issue when I am querying
  
  (Asked by: username_0 on 2020-04-19)
  
  I am using `prestodb:0.234`
  
  Dependency jars:
  
  ```
     azure-storage-2.0.0.jar
     hadoop-azure-3.2.1.jar
     hadoop-azure-datalake-3.2.1.jar 
     azure-data-lake-store-sdk-2.3.8.jar
  
  ```
  
  I am getting below error while doing the querying.
  
  ```
  com.facebook.presto.spi.PrestoException: cannot create caching file system
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration.lambda$getConfiguration$0(HiveCachingHdfsConfiguration.java:70)
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration$CachingJobConf.createFileSystem(HiveCachingHdfsConfiguration.java:94)
      at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:59)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at com.facebook.presto.hive.HdfsEnvironment.lambda$getFileSystem$0(HdfsEnvironment.java:73)
      at com.facebook.presto.hive.authentication.NoHdfsAuthentication.doAs(NoHdfsAuthentication.java:23)
      at com.facebook.presto.hive.HdfsEnvironment.getFileSystem(HdfsEnvironment.java:72)
      at com.facebook.presto.hive.HdfsEnvironment.getFileSystem(HdfsEnvironment.java:66)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:298)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:270)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:100)
      at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:199)
      at com.facebook.presto.hive.util.ResumableTasks.safeProcessTask(ResumableTasks.java:47)
      at com.facebook.presto.hive.util.ResumableTasks.access$000(ResumableTasks.java:20)
      at com.facebook.presto.hive.util.ResumableTasks$1.run(ResumableTasks.java:35)
      at com.facebook.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
  Caused by: java.io.IOException: No FileSystem for scheme: abfs
      at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
      at org.apache.hadoop.fs.PrestoFileSystemCache.createFileSystem(PrestoFileSystemCache.java:114)
      at org.apache.hadoop.fs.PrestoFileSystemCache.getInternal(PrestoFileSystemCache.java:89)
      at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:62)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at com.facebook.presto.hive.HiveCachingHdfsConfiguration.lambda$getConfiguration$0(HiveCachingHdfsConfiguration.java:66)
  
  ```
  
  Any suggestions?
  
  **Tags:** azure, java, hive
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-20. Score: 2 | Accepted answer
  
  Since [`ABFS`](https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver) mentioned in the stacktrace as a caching filesystem, then I suppose two possible reasons and suggest solutions for them
  
  ## Dependency mesh in `maven`
  
  As noted in two answers ([one](https://stackoverflow.com/a/21118824/5720818), [two](https://stackoverflow.com/a/27532248/5720818)) to a [similar question](https://stackoverflow.com/questions/17265002/hadoop-no-filesystem-for-scheme-file)
  
  > When we use maven-assembly-plugin, it merges all our JARs into one, and all META-INFO/services/org.apache.hadoop.fs.FileSystem overwrite each-other. Only one of these files remains (the last one that was added). In this case, the FileSystem list from hadoop-commons overwrites the list from hadoop-hdfs, so DistributedFileSystem was no longer declared.
  
  So, check your dependencies and if there are `JAR` with correct `org.apache.hadoop.fs.FileSystem` in your distibution.
  
  ## `PrestoDB` uses outdated `azure-site.xml`
  
  There is [known issue #11648](https://github.com/prestodb/presto/issues/11648).
  And as [said](https://github.com/prestodb/presto/issues/11648#issuecomment-522853533), `ABFS`, `ADLS` and `ALDS gen2` was only introduced to <https://github.com/prestosql/presto>, not to `prestodb`
  
  So, suggestion is to ask your question on [Presto Community Slack](https://prestosql.io/community.html)
  
  # P.S. try `storediag`
  
  There is a `hadoop` manual on `Azure` [ABFS](https://github.com/apache/hadoop/blob/745a6c1e69b3699f6496a146afc48824dd735461/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md)
  
  And among other tips [they recommend](https://github.com/apache/hadoop/blob/745a6c1e69b3699f6496a146afc48824dd735461/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md):
  
  > One useful tool for debugging connectivity is the [cloudstore storediag utility](https://github.com/steveloughran/cloudstore/releases).
  >
  > This validates the classpath, the settings, then tries to work with the filesystem.
  
  > **Comments:**
  > **username_0**: Thanks @username_1 , the above suggestions helped me lot to resolve my issue. First i shifted to prestosql and used respective jar versions resolved my problem  
  
  ---
  
  