Launch HN: Vellum (YC W23) – Dev Platform for LLM Apps
First: Congratulations! I&#x27;ve just sent an early access request.
Second - I&#x27;ve got questions!<p>Here is my use case. We have hundreds of clients who each have dozens of videos on our platform. Videos are grouped in collections. For each video we have structured content (think of it like a complex mashup of transcript + other info).
I would like to be able to send the API a group of 13 documents, grouped into a collection called &quot;User123_collection2&quot; or whatever. And then run natural language queries against that through an LLM.<p>A. Do I understand correctly that you allow to send and organize documents programmatically? Are there any length restrictions?<p>B. In your demo, Vellum simply grabs the top 3 most relevant snippets (and those seem to be relatively short snippets). Can this be customized (longer snippets and more of them?)<p>C. Can I get the &quot;sources&quot; cited with the answer? 
Let&#x27;s say I run a query on these snippets such as the ones in your demo. (An &quot;end user&quot; kind of question). Similar to how Bing&#x27;s chatbot will give the links to the pages used to build the answer, I&#x27;d like the response I get to tell me it comes from Document 7 in collection &quot;User123_collection2&quot;. Even better if I can get the response to tell me where in the document (otherwise I&#x27;d have to split my documents into smaller pieces when uploading).<p>D. Do you offer any guarantees of privacy of, not only the data, but also the prompts? I think that these prompts might be one of the valuable &quot;trade secrets&quot; for startups who want to add LLM features. If the prompt is leaked publicly or to another Vellum customer&#x2F;stakeholder then it makes the feature replicable.<p>E. How much latency does this additional layer tack onto typical response times? If I was going to make an API request to OpenAI, how much longer does my app wait for the response?<p>I&#x27;m impressed by where you&#x27;re going. It&#x27;s a brilliant idea and I hope we get to be a customer, rather than build the whole LangChain&#x2F;GPT-Index idea we were going to run with up until I read your post :)