Answers.Body: <p>I worked at a start-up doing this exact task for two years. There were several developers involved in it, and it took several years; by the end (with a lot of manual input, eg categorising ingredients in an ontology, converting measurements, and mapping ingredients to shopping items from a supermarket inventory) we had something that worked reasonably well.</p>
  
  <p>You rightly recognise that it is not trivial... The first step would be to identify the list of ingredients from the recipe (assuming you have scraped it from a website), and turn it into a list; beware that sometimes there might be more than one ingredient in a line. Then you need to analyse the entry: we had a grammar that distinguished between quantities, attributes (<em>fresh</em>), processing instructions (<em>peeled</em>) and a number of other things. The main item would of course be the actual ingredient. Then we would look this up in a supermarket inventory and map it to the closest item (but I don't think this is part of your remit).</p>
  
  <p>We tried different ways of analysing the recipe lines, and any grammar/parser is fine, as long as it can deal with partial analyses, as you cannot always rely on lines being complete and well-formed, and there is always scope for additional comments you want to ignore ("I only use the freshest lemons directly imported from Tuscany", "get your fishmonger to take the bones out"). In the simplest case you could have a list of items (<em>lemon</em>) with inflections etc, that you try to match in your recipe lines. If you don't care whether it's big lemons or not, don't bother using a full-blown grammar. And even then, you could just look at a list of adjectives preceding an ingredient to identify any modifiers. We used a grammar based on recursive transition networks (RTNs, where the parser was complex but the grammars easy to handle) and one that was a simple phrase structure one (where the parser was simpler, but the grammar was much more complex). Both worked fine.</p>
  
  <p>As far as I am aware there is not really much written about this kind of work. The main advice I can give you is to look at as many recipes as possible to get an idea for what you will encounter, and cut corners wherever possible -- it's a big job to do it perfectly, and you can get very far by using simple methods.</p>
  
  <p>We also transferred it into different languages (Polish too, which I assume you're working with?). We didn't find that inflections were that big an issue. You just have a list of all inflected forms linking to the lemma. As you have a limited domain, that is a perfectly feasible approach. Just have a master list of ingredients, with all alternate forms (<em>courgette</em> and <em>zucchini</em> are the same thing, for example).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-12T18:01:17.957Z
Answers.LastActivityDate: 2018-12-12T18:01:17.957Z
Answers.UserDisplayName: Oliver Mason
Body: <p>I'm working on a home tool that will help create a shopping list from a list of recipes chosen for a coming week.</p>
  
  <p>This boils down to:</p>
  
  <ol>
  <li>Extracting ingredients and their quantities from recipes.</li>
  <li>Grouping similar ingredients together.</li>
  <li>Summing up quantities for similar ingredients.</li>
  <li>Naming groups of similar products in a shopping list.</li>
  </ol>
  
  <p>The tasks seem non-trivial for a few reasons.</p>
  
  <ul>
  <li><p><strong>Similar ingredients are described differently</strong>, depending on the recipe book/portal, e.g.:</p>
  
  <blockquote>
    <ul>
    <li>5 lemons</li>
    <li>5 lemons (to be squeezed)</li>
    <li>5 fresh lemons</li>
    <li>5 big yellow lemons</li>
    </ul>
  </blockquote></li>
  <li><p><strong>Recipes lists alternatives for ingredients</strong> (e.g., <em>"3 lemons or 5 limes"</em>), leaving decision up to a user. </p></li>
  <li><p><strong>Recipes involve some information about product-preprocessing</strong>. For instance, one has to buy lemons instead of lemon juice when the recipe says:</p>
  
  <blockquote>
    <ul>
    <li>100ml lemon juice</li>
    <li>100ml freshly squeezed lemon juice</li>
    </ul>
  </blockquote></li>
  <li><p><strong>My language has a complex inflection</strong>. For instance, there can be multiple plural forms of a noun and the form of an adjective must be agreed with a form of noun. Adapting NLP algorithms designed for English language might be not straightforward and require some lemmatizing/stemming but not for single words, but whole phrases.</p></li>
  <li><p><strong>Naming products group is hard</strong>. Once fresh lemons and big yellow lemons are group together and their quantities summed up, one need to decide how to name this group in a shopping list, e.g.: <em>"10 lemons"</em> or <em>"10 fresh lemons"</em>.</p></li>
  </ul>
  
  <p>Is there any research paper that would cover those challenges?</p>
  
  <p>Especially applied in the same domain? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-12T14:41:31.033Z
Comments.Text: PS. There are other challenges like different ways to quantity quantities (volume-based: ml, spoon, bunch, etc. and weigth-based: kg, g, etc.), but let's leave them out for now.
Comments.UserDisplayName: dzieciou
Tags: <natural-language-processing><applications>
Title: Grouping products and naming groups
UserDisplayName: dzieciou
ThreadText: # Grouping products and naming groups
  
  (Asked by: username_0 on 2018-12-12)
  
  I'm working on a home tool that will help create a shopping list from a list of recipes chosen for a coming week.
  
  This boils down to:
  
  1. Extracting ingredients and their quantities from recipes.
  2. Grouping similar ingredients together.
  3. Summing up quantities for similar ingredients.
  4. Naming groups of similar products in a shopping list.
  
  The tasks seem non-trivial for a few reasons.
  
  * **Similar ingredients are described differently**, depending on the recipe book/portal, e.g.:
  
    > + 5 lemons
    > + 5 lemons (to be squeezed)
    > + 5 fresh lemons
    > + 5 big yellow lemons
  * **Recipes lists alternatives for ingredients** (e.g., *"3 lemons or 5 limes"*), leaving decision up to a user.
  * **Recipes involve some information about product-preprocessing**. For instance, one has to buy lemons instead of lemon juice when the recipe says:
  
    > + 100ml lemon juice
    > + 100ml freshly squeezed lemon juice
  * **My language has a complex inflection**. For instance, there can be multiple plural forms of a noun and the form of an adjective must be agreed with a form of noun. Adapting NLP algorithms designed for English language might be not straightforward and require some lemmatizing/stemming but not for single words, but whole phrases.
  * **Naming products group is hard**. Once fresh lemons and big yellow lemons are group together and their quantities summed up, one need to decide how to name this group in a shopping list, e.g.: *"10 lemons"* or *"10 fresh lemons"*.
  
  Is there any research paper that would cover those challenges?
  
  Especially applied in the same domain?
  
  **Tags:** natural-language-processing, applications
  > **Comments:**
  > **username_0**: PS. There are other challenges like different ways to quantity quantities (volume-based: ml, spoon, bunch, etc. and weigth-based: kg, g, etc.), but let's leave them out for now.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-12-12. Score: 1 | Accepted answer
  
  I worked at a start-up doing this exact task for two years. There were several developers involved in it, and it took several years; by the end (with a lot of manual input, eg categorising ingredients in an ontology, converting measurements, and mapping ingredients to shopping items from a supermarket inventory) we had something that worked reasonably well.
  
  You rightly recognise that it is not trivial... The first step would be to identify the list of ingredients from the recipe (assuming you have scraped it from a website), and turn it into a list; beware that sometimes there might be more than one ingredient in a line. Then you need to analyse the entry: we had a grammar that distinguished between quantities, attributes (*fresh*), processing instructions (*peeled*) and a number of other things. The main item would of course be the actual ingredient. Then we would look this up in a supermarket inventory and map it to the closest item (but I don't think this is part of your remit).
  
  We tried different ways of analysing the recipe lines, and any grammar/parser is fine, as long as it can deal with partial analyses, as you cannot always rely on lines being complete and well-formed, and there is always scope for additional comments you want to ignore ("I only use the freshest lemons directly imported from Tuscany", "get your fishmonger to take the bones out"). In the simplest case you could have a list of items (*lemon*) with inflections etc, that you try to match in your recipe lines. If you don't care whether it's big lemons or not, don't bother using a full-blown grammar. And even then, you could just look at a list of adjectives preceding an ingredient to identify any modifiers. We used a grammar based on recursive transition networks (RTNs, where the parser was complex but the grammars easy to handle) and one that was a simple phrase structure one (where the parser was simpler, but the grammar was much more complex). Both worked fine.
  
  As far as I am aware there is not really much written about this kind of work. The main advice I can give you is to look at as many recipes as possible to get an idea for what you will encounter, and cut corners wherever possible -- it's a big job to do it perfectly, and you can get very far by using simple methods.
  
  We also transferred it into different languages (Polish too, which I assume you're working with?). We didn't find that inflections were that big an issue. You just have a list of all inflected forms linking to the lemma. As you have a limited domain, that is a perfectly feasible approach. Just have a master list of ingredients, with all alternate forms (*courgette* and *zucchini* are the same thing, for example).
  
  
  ---
  
  