Answers.Body: <p>A very large one, the world wide web with highly scaled and optimized indexing by Google.com is the most distributed and robust schema-agnostic database known today.  Without the schema-awareness Google brought to the table by applying more rigorous information science to the table, it was almost useless to those that did not know the URL of the target document in advance.</p>
  
  <p>Schema agnosticism is another way of saying that the database cannot</p>
  
  <ul>
  <li>Provide meta information to the services accessing it,</li>
  <li>Normalize the structure using simple SQL query-insert combinations</li>
  <li>Proactively optimize the keys automatically as is now possible with machine learning, or</li>
  <li>Validate insertions</li>
  </ul>
  
  <p>Without first detecting a schema from data patterns.  Moving away from structure is appealing because you can just jam data in like a librarian without a book shelf.   However, the data scientist will point out that adding entropy working alongside thermodynamic devolution into stochasm.</p>
  
  <p>The purpose of storing data is to be able to retrieve it.  Feature extraction is an opportunity to improve structure automatically during the storing structure, rather than store documents chaotically, a trend that will not lead anywhere good for the world of IT.</p>
  
  <p>Consider whether Google is successful because it organizes its data as it crawls or later as we enter key phrases.  Which is the efficient sequence?</p>
  
  <p>One more point, Wikipedia is a blog, and they know this, which is why they want peer review for everything now (after much of the information was added without peer review).  It is a good place to find lists but not verified facts.  The existence of a Wikipedia page is definitely not an indication of the value of the concept on it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-25T13:52:35.413Z
Answers.Text: I improved a bit the question. I asked about an Engine, not the data itself. The scenario is already chaotic, Google provides a search engine for the web and needs to organize the data, sure. What about the data in private silos? The amount of data is not huge like in the web but the chaos exists with or without a schema. Each time an enterprise deploys a new system it has to interface with legacy data and/or perhaps creates more data, more chaos to be queried. The agnosticism is much more about how to consume the data.
Answers.UserDisplayName: Leo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-25T13:57:51.743Z
Answers.Text: Another thing. Yes wikipedia alone is not proof of anything, but in this case I am one of the contributos of this article and have worked with several people from Academia and Industry to consolidate this concept. And someone needs wirte down the first lines anyway.
Answers.UserDisplayName: Leo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-27T12:19:12.097Z
Answers.Text: Summing up all your points. Yes, the question is how to have a smart system that can infer (with some level of error, of course) and perhaps return useful information for the end user in a way that the user will further improve the query and at the same time give feedback to the system, making the engine even better. The agnosticism lies mostly in the user query language interface. About Wikipedia you are right, I just pointed that perhaps this article, in particular, is a good one ;)
Answers.UserDisplayName: Leo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-31T08:21:09.350Z
Answers.Text: I have upvoted to balance the entropy ;)
Answers.UserDisplayName: Leo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-23T15:17:08.587Z
Answers.LastActivityDate: 2018-07-23T15:17:08.587Z
Answers.UserDisplayName: Douglas Daseeco
Answers.Body: <p>You could be interested in orthogonally <a href="https://en.wikipedia.org/wiki/Persistence_(computer_science)" rel="nofollow noreferrer">persistent</a> systems. You could look at them as  schema-agnostic database systems whose data fits <em>entirely</em> in RAM (remember also 1980s <a href="https://en.wikipedia.org/wiki/Smalltalk" rel="nofollow noreferrer">Smalltalk</a> or <a href="https://en.wikipedia.org/wiki/Lisp_machine" rel="nofollow noreferrer">Lisp Machines</a> or <a href="https://en.wikipedia.org/wiki/Prolog" rel="nofollow noreferrer">Prolog</a> ones and 1994 <a href="https://www.usenix.org/legacy/publications/compsystems/1994/sum_dearle.pdf" rel="nofollow noreferrer">GrassHopper</a> OS) or at least in <a href="https://en.wikipedia.org/wiki/Virtual_memory" rel="nofollow noreferrer">virtual memory</a>. With that approach, even <a href="https://sbcl.org/" rel="nofollow noreferrer">SBCL</a> almost fits in your wish, since it has <a href="http://www.sbcl.org/manual/index.html#Saving-a-Core-Image" rel="nofollow noreferrer"><code>save-lisp-and-die</code></a>. Look also into <a href="https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)" rel="nofollow noreferrer">frame</a> based systems and <a href="https://en.wikipedia.org/wiki/Object_database" rel="nofollow noreferrer">object databases</a>. Read also a good <a href="http://pages.cs.wisc.edu/~remzi/OSTEP/" rel="nofollow noreferrer"><em>operating systems</em> textbook</a> and see past discussions archived on <a href="https://tunes.org/" rel="nofollow noreferrer">tunes.org</a>.</p>
  
  <p>Shameful self promotion: My <a href="https://github.com/bstarynk/bismon/" rel="nofollow noreferrer">bismon system</a> (work in progress in summer 2019) <a href="http://starynkevitch.net/Basile/bismon-chariot-doc.pdf" rel="nofollow noreferrer">claims</a> to be a GPLv3+ orthogonally persistent system applied for static source code analysis of IoT software. But you might reuse most of it for other kind of orthogonal persistence (of <a href="https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)" rel="nofollow noreferrer">frame</a> based data).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T06:42:28.863Z
Answers.LastActivityDate: 2019-08-16T06:54:01.370Z
Answers.LastEditDate: 2019-08-16T06:54:01.370Z
Answers.UserDisplayName: Basile Starynkevitch
Body: <p>Has any <a href="https://en.wikipedia.org/wiki/Schema-agnostic_databases" rel="nofollow noreferrer">schema-agnostic database</a> engine been implemented?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-27T21:27:19.073Z
Comments.Text: The same article you link points to SPARQL as a query language and https://query.wikidata.org/ and http://dbpedia.org/isparql/ as example implementations. Is that what you were looking for? If not, why not?
Comments.UserDisplayName: Alpha
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-30T10:08:13.610Z
Comments.Text: @Alpha The goal is to abstract users from the data representation. The user acessing this kind of database does not need to known all the available entities that could be queried hence a query like "SELECT * FROM customers" or "SELECT * FROM clients" must give the same result. The db engine will figure out that "clients" and "customers" is referencing the same entity.
Comments.UserDisplayName: Leo
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-13T19:13:27.673Z
Comments.Text: For future reference. MarkLogic is not a Schema-agnostic database: http://www.marklogic.com/blog/schema-agnosticism-what-it-is-and-why-you-should-care/
Comments.UserDisplayName: Leo
Tags: <natural-language-processing><knowledge-representation>
Title: Has any schema-agnostic database engine been implemented?
UserDisplayName: Leo
ThreadText: # Has any schema-agnostic database engine been implemented?
  
  (Asked by: username_0 on 2017-01-23)
  
  Has any [schema-agnostic database](https://en.wikipedia.org/wiki/Schema-agnostic_databases) engine been implemented?
  
  **Tags:** natural-language-processing, knowledge-representation
  
  > **Comments:**
  > **username_1**: The same article you link points to SPARQL as a query language and https://query.wikidata.org/ and http://dbpedia.org/isparql/ as example implementations. Is that what you were looking for? If not, why not?  
  > **username_0**: @username_1 The goal is to abstract users from the data representation. The user acessing this kind of database does not need to known all the available entities that could be queried hence a query like "SELECT * FROM customers" or "SELECT * FROM clients" must give the same result. The db engine will figure out that "clients" and "customers" is referencing the same entity.  
  > **username_0**: For future reference. MarkLogic is not a Schema-agnostic database: http://www.marklogic.com/blog/schema-agnosticism-what-it-is-and-why-you-should-care/  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-07-23. Score: 1
  
  A very large one, the world wide web with highly scaled and optimized indexing by Google.com is the most distributed and robust schema-agnostic database known today. Without the schema-awareness Google brought to the table by applying more rigorous information science to the table, it was almost useless to those that did not know the URL of the target document in advance.
  
  Schema agnosticism is another way of saying that the database cannot
  
  * Provide meta information to the services accessing it,
  * Normalize the structure using simple SQL query-insert combinations
  * Proactively optimize the keys automatically as is now possible with machine learning, or
  * Validate insertions
  
  Without first detecting a schema from data patterns. Moving away from structure is appealing because you can just jam data in like a librarian without a book shelf. However, the data scientist will point out that adding entropy working alongside thermodynamic devolution into stochasm.
  
  The purpose of storing data is to be able to retrieve it. Feature extraction is an opportunity to improve structure automatically during the storing structure, rather than store documents chaotically, a trend that will not lead anywhere good for the world of IT.
  
  Consider whether Google is successful because it organizes its data as it crawls or later as we enter key phrases. Which is the efficient sequence?
  
  One more point, Wikipedia is a blog, and they know this, which is why they want peer review for everything now (after much of the information was added without peer review). It is a good place to find lists but not verified facts. The existence of a Wikipedia page is definitely not an indication of the value of the concept on it.
  
  > **Comments:**
  > **username_0**: I improved a bit the question. I asked about an Engine, not the data itself. The scenario is already chaotic, Google provides a search engine for the web and needs to organize the data, sure. What about the data in private silos? The amount of data is not huge like in the web but the chaos exists with or without a schema. Each time an enterprise deploys a new system it has to interface with legacy data and/or perhaps creates more data, more chaos to be queried. The agnosticism is much more about how to consume the data.  
  > **username_0**: Another thing. Yes wikipedia alone is not proof of anything, but in this case I am one of the contributos of this article and have worked with several people from Academia and Industry to consolidate this concept. And someone needs wirte down the first lines anyway.  
  > **username_0**: Summing up all your points. Yes, the question is how to have a smart system that can infer (with some level of error, of course) and perhaps return useful information for the end user in a way that the user will further improve the query and at the same time give feedback to the system, making the engine even better. The agnosticism lies mostly in the user query language interface. About Wikipedia you are right, I just pointed that perhaps this article, in particular, is a good one ;)  
  > **username_0**: I have upvoted to balance the entropy ;)  
  
  ---
  
  ### Answer by username_3 on 2019-08-16. Score: 0
  
  You could be interested in orthogonally [persistent](https://en.wikipedia.org/wiki/Persistence_(computer_science)) systems. You could look at them as schema-agnostic database systems whose data fits *entirely* in RAM (remember also 1980s [Smalltalk](https://en.wikipedia.org/wiki/Smalltalk) or [Lisp Machines](https://en.wikipedia.org/wiki/Lisp_machine) or [Prolog](https://en.wikipedia.org/wiki/Prolog) ones and 1994 [GrassHopper](https://www.usenix.org/legacy/publications/compsystems/1994/sum_dearle.pdf) OS) or at least in [virtual memory](https://en.wikipedia.org/wiki/Virtual_memory). With that approach, even [SBCL](https://sbcl.org/) almost fits in your wish, since it has [`save-lisp-and-die`](http://www.sbcl.org/manual/index.html#Saving-a-Core-Image). Look also into [frame](https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)) based systems and [object databases](https://en.wikipedia.org/wiki/Object_database). Read also a good [*operating systems* textbook](http://pages.cs.wisc.edu/~remzi/OSTEP/) and see past discussions archived on [tunes.org](https://tunes.org/).
  
  Shameful self promotion: My [bismon system](https://github.com/bstarynk/bismon/) (work in progress in summer 2019) [claims](http://username_3.net/username_3/bismon-chariot-doc.pdf) to be a GPLv3+ orthogonally persistent system applied for static source code analysis of IoT software. But you might reuse most of it for other kind of orthogonal persistence (of [frame](https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)) based data).
  
  
  ---
  
  