<p dir="auto">Running neo4j-admin gets stuck deterministically. We can see using strace there's no system call running doing IO. All threads are stuck on Mutexes.<br>
I suspect a potential dead lock or a thread dying.</p>
<p dir="auto"><strong>Neo4j Version:</strong>  4.1.2 (also reproduced on previous two versions)</p>
<h3 dir="auto">Steps to reproduce</h3>
<p dir="auto">neo4j-admin import <br>
--database=$DBNAME <br>
--delimiter="|" <br>
--nodes=REDACTED=${NODE_DIR}/REDACTED.csv,${DATA_DIR}/REDACTED.csv \ # 44 more lines like this<br>
--relationships=REDACTED=${EDGE_DIR}/REDACTED.csv,${DATA_DIR}/REDACTED.csv \ # 75 more lines like this<br>
--skip-bad-relationships=REDACTED <br>
--skip-duplicate-nodes=REDACTED <br>
--trim-strings=true</p>
<p dir="auto">Jstack attached:</p>
<p dir="auto"><a href="https://github.com/neo4j/neo4j/files/5376298/jstack_4.1.2.txt">jstack_4.1.2.txt</a></p>
<p dir="auto">Console output attached:</p>
<p dir="auto"><a href="https://github.com/neo4j/neo4j/files/5376579/console.txt">console.txt</a></p>
<p dir="auto">The backing store is NFS which we know is not recommended by the documentation but we disabled locking by NFS so the kernel manages them locally as if the application was running locally.</p>
<ul dir="auto">
<li>Neo4j version: 3.4.0</li>
<li>Operating system: Windows 10 64 bit</li>
<li>Neo4j Desktop: 1.1.2</li>
</ul>
<p dir="auto">Using the bulk csv import through neo4j desktop generated this error. As you may also see from the timings this importing had taken &gt;20 hours. I had upgraded from neo4j 3.3.4 to attempt to get away with the slow speeds. Not sure if this error is related to the slow importing, but the full details are on <a href="https://stackoverflow.com/questions/50552882/neo4j-bulk-import-doesnt-work-for-big-datasets" rel="nofollow">stackoverflow</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="C:\Users\D\.Neo4jDesktop\neo4jDatabases\database-d6438069-dd9b-435c-bbc1-83e313cdc77f\installation-3.4.0&gt;SET DATA=D:\Neo4jdb\data

C:\Users\D\.Neo4jDesktop\neo4jDatabases\database-d6438069-dd9b-435c-bbc1-83e313cdc77f\installation-3.4.0&gt;.\bin\neo4j-admin import ^
More? --mode=csv ^
More? --database=association_graph.db ^
More? --ignore-duplicate-nodes true ^
More? --nodes %DATA%\nodes_assocs.csv ^
More? --nodes %DATA%\nodes_docs_fixed.csv ^
More? --nodes %DATA%\nodes_entities_dedup.csv ^
More? --nodes %DATA%\nodes_sents.csv ^
More? --relationships %DATA%\edges_IN_ASSOC.csv ^
More? --relationships %DATA%\edges_IN_DOC.csv ^
More? --relationships %DATA%\edges_IN_SENT.csv

WARNING: This command does not appear to be running with administrative rights.  Some commands may fail e.g. Start/Stop

Neo4j version: 3.4.0

Importing the contents of these files into D:\Neo4jdb\databases\association_graph.db:
Nodes:

.\bin\neo4j-admin import ^

--mode=csv ^
--database=association_graph.db ^
--ignore-duplicate-nodes true ^
--nodes D:\Neo4jdb\data\nodes_assoc.csv ^
--nodes D:\Neo4jdb\data\nodes_article.csv ^
--nodes D:\Neo4jdb\data\nodes_term.csv ^
--nodes D:\Neo4jdb\data\nodes_chunk.csv ^
--relationships %DATA%\edges_IN_ASSOC.csv ^
--relationships %DATA%\edges_IN_ARTICLE.csv ^
--relationships %DATA%\edges_IN_CHUNK.csv

Available resources:
  Total machine memory: 63.91 GB
  Free machine memory: 54.02 GB
  Max heap memory : 14.20 GB
  Processors: 12
  Configured max memory: 44.74 GB
  High-IO: false

Import starting 2018-05-28 18:28:00.474+0100
  Estimated number of nodes: 653.00 M
  Estimated number of node properties: 1.01 G
  Estimated number of relationships: 641.45 M
  Estimated number of relationship properties: 0.00
  Estimated disk space usage: 104.70 GB
  Estimated required memory usage: 9.02 GB

InteractiveReporterInteractions command list (end with ENTER):
  c: Print more detailed information about current stage
  i: Print more detailed information

(1/4) Node import 2018-05-28 18:28:00.642+0100
  Estimated number of nodes: 653.00 M
  Estimated disk space usage: 84.39 GB
  Estimated required memory usage: 9.02 GB
.......... .......... .......... .......... .......... 5%
.......... .......... .......... .......... .......... 10%
.......... .......... .......... .......... .......... 15%
.......... .......... .......... .......... .......... 20%
.......... .......... .......... .......... .......... 25%
.......... .......... .......... .......... .......... 30%
.......... .......... .......... .......... .......... 35%
.......... .......... .......... .......... .......... 40%
.......... .......... .......... .......... .......... 45%
.......... .......... .......... .......... .......... 50%
.......... .......... .......... .......... .......... 55%
.......... .......... .......... .......... .......... 60%
.......... .......... .......... .......... .......... 65%
.......... .......... .......... .......... .......... 70%
.......... .......... .......... .......... .......... 75%
.......... .......... .......... .......... .......... 80%
.......... .......... .......... .......... .......... 85%
..Exception in thread &quot;Thread-8&quot; java.lang.RuntimeException: java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCurso
r@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:150)
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:142)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:58)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offse
t 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        ... 1 more
........ .......... .......... .......... .......... 90%
.......... .......... .......... .......... .......... 95%
.......... .......... .......... .......... .......... 100%


IMPORT DONE in 21h 24m 620ms.
Data statistics is not available.
Peak memory usage: 5.91 GB
Import error: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
Caused by:Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        at java.lang.Thread.run(Thread.java:748)

WARNING Import failed. The store files in D:\Neo4jdb\databases\association_graph.db are left as they are, although they are likely in an unusable state. Sta
rting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        at java.lang.Thread.run(Thread.java:748)
unexpected error: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0"><pre class="notranslate"><code class="notranslate">C:\Users\D\.Neo4jDesktop\neo4jDatabases\database-d6438069-dd9b-435c-bbc1-83e313cdc77f\installation-3.4.0&gt;SET DATA=D:\Neo4jdb\data

C:\Users\D\.Neo4jDesktop\neo4jDatabases\database-d6438069-dd9b-435c-bbc1-83e313cdc77f\installation-3.4.0&gt;.\bin\neo4j-admin import ^
More? --mode=csv ^
More? --database=association_graph.db ^
More? --ignore-duplicate-nodes true ^
More? --nodes %DATA%\nodes_assocs.csv ^
More? --nodes %DATA%\nodes_docs_fixed.csv ^
More? --nodes %DATA%\nodes_entities_dedup.csv ^
More? --nodes %DATA%\nodes_sents.csv ^
More? --relationships %DATA%\edges_IN_ASSOC.csv ^
More? --relationships %DATA%\edges_IN_DOC.csv ^
More? --relationships %DATA%\edges_IN_SENT.csv

WARNING: This command does not appear to be running with administrative rights.  Some commands may fail e.g. Start/Stop

Neo4j version: 3.4.0

Importing the contents of these files into D:\Neo4jdb\databases\association_graph.db:
Nodes:

.\bin\neo4j-admin import ^

--mode=csv ^
--database=association_graph.db ^
--ignore-duplicate-nodes true ^
--nodes D:\Neo4jdb\data\nodes_assoc.csv ^
--nodes D:\Neo4jdb\data\nodes_article.csv ^
--nodes D:\Neo4jdb\data\nodes_term.csv ^
--nodes D:\Neo4jdb\data\nodes_chunk.csv ^
--relationships %DATA%\edges_IN_ASSOC.csv ^
--relationships %DATA%\edges_IN_ARTICLE.csv ^
--relationships %DATA%\edges_IN_CHUNK.csv

Available resources:
  Total machine memory: 63.91 GB
  Free machine memory: 54.02 GB
  Max heap memory : 14.20 GB
  Processors: 12
  Configured max memory: 44.74 GB
  High-IO: false

Import starting 2018-05-28 18:28:00.474+0100
  Estimated number of nodes: 653.00 M
  Estimated number of node properties: 1.01 G
  Estimated number of relationships: 641.45 M
  Estimated number of relationship properties: 0.00
  Estimated disk space usage: 104.70 GB
  Estimated required memory usage: 9.02 GB

InteractiveReporterInteractions command list (end with ENTER):
  c: Print more detailed information about current stage
  i: Print more detailed information

(1/4) Node import 2018-05-28 18:28:00.642+0100
  Estimated number of nodes: 653.00 M
  Estimated disk space usage: 84.39 GB
  Estimated required memory usage: 9.02 GB
.......... .......... .......... .......... .......... 5%
.......... .......... .......... .......... .......... 10%
.......... .......... .......... .......... .......... 15%
.......... .......... .......... .......... .......... 20%
.......... .......... .......... .......... .......... 25%
.......... .......... .......... .......... .......... 30%
.......... .......... .......... .......... .......... 35%
.......... .......... .......... .......... .......... 40%
.......... .......... .......... .......... .......... 45%
.......... .......... .......... .......... .......... 50%
.......... .......... .......... .......... .......... 55%
.......... .......... .......... .......... .......... 60%
.......... .......... .......... .......... .......... 65%
.......... .......... .......... .......... .......... 70%
.......... .......... .......... .......... .......... 75%
.......... .......... .......... .......... .......... 80%
.......... .......... .......... .......... .......... 85%
..Exception in thread "Thread-8" java.lang.RuntimeException: java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCurso
r@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:150)
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:142)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:58)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offse
t 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        ... 1 more
........ .......... .......... .......... .......... 90%
.......... .......... .......... .......... .......... 95%
.......... .......... .......... .......... .......... 100%


IMPORT DONE in 21h 24m 620ms.
Data statistics is not available.
Peak memory usage: 5.91 GB
Import error: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
Caused by:Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        at java.lang.Thread.run(Thread.java:748)

WARNING Import failed. The store files in D:\Neo4jdb\databases\association_graph.db are left as they are, although they are likely in an unusable state. Sta
rting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
java.lang.IllegalStateException: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheNumberArray.checkBounds(PageCacheNumberArray.java:185)
        at org.neo4j.unsafe.impl.batchimport.cache.PageCacheByteArray.setShort(PageCacheByteArray.java:292)
        at org.neo4j.unsafe.impl.batchimport.cache.DynamicByteArray.setShort(DynamicByteArray.java:136)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.StringCollisionValues.add(StringCollisionValues.java:69)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.buildCollisionInfo(EncodingIdMapper.java:508)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:237)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        at java.lang.Thread.run(Thread.java:748)
unexpected error: Cursor org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor@4b7148ec access out of bounds, page id 11, offset 0
</code></pre></div>