Stack Exchange
How can I update my unnesting python fucntion to include products without options also?
I am trying to extract size and color from options in a dataframe and it is working partially with below code-
def unnest(df, col, col2,reset_index=False):
 col_flat = pd.DataFrame([[i, x]
                  for i, y in df[col].apply(list).iteritems()
                      for x in y ], columns=['I', col]
                       )
 col_flat = col_flat.set_index('I')
 df = df.drop(col, 1)
 df = df.merge(col_flat, left_index=True, right_index=True)

 if reset_index:
     df = df.reset_index(drop=True)
 merchant_product_code = (df['Product code'] + '-' + df[col])
 df['item_group_id'] = df['Product code']
 df['Product code'] = merchant_product_code
 return df

But there are some products with blank options( value is []) and they are removed when I apply the unnesting.
How can I ignore this on the products without options values?
PFB screen shot for before and after unnesting and see the options column-


I am applying my function on a datafarme as below-
full_df=unnest(full_df,'Options','product code')

And the full_df looks like below screen shot-

I have converted the full_df to dictionary for pasting here the sample-
{'Product code': {4: 'BBFTDR-6-12m', 5: 'BBFTPBG-3-6m'}, 'Category': {4: 'Apparel', 5: 'Apparel'}, 'List price': {4: '28', 5: '28'}, 'Price': {4: '28', 5: '28'}, 'Weight': {4: '0', 5: '0'}, 'Quantity': {4: '1', 5: '1'}, 'Min quantity': {4: '0', 5: '0'}, 'Shipping freight': {4: '0', 5: '0'}, 'Date added': {4: '7/9/17 0:00', 5: '7/9/17 0:00'}, 'Downloadable': {4: 'N', 5: 'N'}, 'Files': {4: nan, 5: nan}, 'Ship downloadable': {4: 'N', 5: 'N'}, 'Inventory tracking': {4: 'O', 5: 'O'}, 'Free shipping': {4: 'N', 5: 'N'}, 'Feature comparison': {4: 'N', 5: 'N'}, 'Zero price action': {4: 'P', 5: 'P'}, 'Thumbnail': {4: nan, 5: nan}, 'Detailed image_x': {4: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Deer.jpg', 5: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Bear_Gray.jpg'}, 'Product name': {4: 'Bamboo Baby Footie with Fold Over Mitten - Deer', 5: 'Bamboo Baby Footie with Fold Over Mitten - Polar Bear Grey'}, 'Description': {4: '<p class="MsoNormal" style="border: 0px; margin: 0px; padding: 6px 0px; color: #333333;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Bebe Bamboo basic range. Simple yet stylish. This romper has an in built fold over mittens.&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">- Extremely soft, delicate and smooth.</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">- Promote comfortable and restful sleep.&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Composition: 70% Viscose from Bamboo, 30% Cotton</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;">(Oeko Tex Standard 100)</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-size: small; font-family: \'comic sans ms\', sans-serif;">Lead &amp; Phthalate Free</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-size: small;"><br /></span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Washing instructions:</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Machine or hand wash cold</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Wash and iron inside out</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Tumble dry low</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Do not bleach</span></p><p>&nbsp;<img src="http://bebebamboo.com/images/Footie%20washing%20symbol.jpg" alt="" width="152" height="43" /></p><p>&nbsp;</p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Available size: 0-3m, 3-6m, 6-12m</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Wear Snug Fit, Not Flame Resistant&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;"><br /></span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Footie measurement (cm)</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;"><img src="http://bebebamboo.com/images/Footie%20size.jpg" alt="" width="578" height="149" /></span></p>', 5: '<p class="MsoNormal" style="border: 0px; margin: 0px; padding: 6px 0px; color: #333333;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Bebe Bamboo basic range. Simple yet stylish. This romper has an in built fold over mittens.&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">- Extremely soft, delicate and smooth.</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">- Promote comfortable and restful sleep.&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;" lang="EN-US">&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Composition: 70% Viscose from Bamboo, 30% Cotton</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="border: 0px; font-size: small; margin: 0px; padding: 0px; font-family: \'comic sans ms\', sans-serif;">(Oeko Tex Standard 100)</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-size: small; font-family: \'comic sans ms\', sans-serif;">Lead &amp; Phthalate Free</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-size: small;"><br /></span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Washing instructions:</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Machine or hand wash cold</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Wash and iron inside out</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Tumble dry low</span></p><p><span style="font-size: small;">*&nbsp;</span><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Do not bleach</span></p><p>&nbsp;<img src="http://bebebamboo.com/images/Footie%20washing%20symbol.jpg" alt="" width="152" height="43" /></p><p>&nbsp;</p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Available size: 0-3m, 3-6m, 6-12m</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Wear Snug Fit, Not Flame Resistant&nbsp;</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;"><br /></span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;">Footie measurement (cm)</span></p><p class="MsoNormal" style="border: 0px; font-size: 13px; margin: 0px; padding: 6px 0px; color: #333333; font-family: Arial, Tahoma, Helvetica, sans-serif;"><span style="font-family: \'comic sans ms\', sans-serif; font-size: small;"><img src="http://bebebamboo.com/images/Footie%20size.jpg" alt="" width="578" height="149" /></span></p>'}, 'Meta keywords': {4: nan, 5: nan}, 'Meta description': {4: nan, 5: nan}, 'Search words': {4: nan, 5: nan}, 'Page title': {4: nan, 5: nan}, 'Taxes': {4: nan, 5: nan}, 'Features': {4: nan, 5: nan}, 'Secondary categories': {4: 'Apparel/Footie with Foldover Mittens', 5: 'Apparel/Footie with Foldover Mittens'}, 'SEO name': {4: 'bamboo-baby-footie-with-fold-over-mitten-deer', 5: 'bamboo-baby-footie-with-fold-over-mitten-deer-clone'}, 'Short description': {4: nan, 5: nan}, 'Localizations': {4: nan, 5: nan}, 'Status': {4: 'A', 5: 'A'}, 'Product URL': {4: 'http://bebebamboo.com/bamboo-baby-footie-with-fold-over-mitten-deer.html', 5: 'http://bebebamboo.com/bamboo-baby-footie-with-fold-over-mitten-deer-clone.html'}, 'Image URL': {4: 'http://bebebamboo.com/images/thumbnails/1/420/420/Romper_Deer.jpg', 5: 'http://bebebamboo.com/images/thumbnails/1/420/420/Romper_Bear_Gray.jpg'}, 'Sale price': {4: '28', 5: '28'}, 'Product id': {4: '361', 5: '362'}, 'Out of stock actions': {4: 'N', 5: 'N'}, 'Items in box': {4: 'min:0;max:0', 5: 'min:0;max:0'}, 'Box size': {4: 'length:0;width:0;height:0', 5: 'length:0;width:0;height:0'}, 'Unnamed: 39': {4: nan, 5: nan}, 'Unnamed: 40': {4: nan, 5: nan}, 'Unnamed: 41': {4: nan, 5: nan}, 'Unnamed: 42': {4: nan, 5: nan}, 'Unnamed: 43': {4: nan, 5: nan}, 'Unnamed: 44': {4: nan, 5: nan}, 'Unnamed: 45': {4: nan, 5: nan}, 'Unnamed: 46': {4: nan, 5: nan}, 'Unnamed: 47': {4: nan, 5: nan}, 'Unnamed: 48': {4: nan, 5: nan}, 'Unnamed: 49': {4: nan, 5: nan}, 'Unnamed: 50': {4: nan, 5: nan}, 'Unnamed: 51': {4: nan, 5: nan}, 'Unnamed: 52': {4: nan, 5: nan}, 'Unnamed: 53': {4: nan, 5: nan}, 'Unnamed: 54': {4: nan, 5: nan}, 'Unnamed: 55': {4: nan, 5: nan}, 'Unnamed: 56': {4: nan, 5: nan}, 'Unnamed: 57': {4: nan, 5: nan}, 'Unnamed: 58': {4: nan, 5: nan}, 'Unnamed: 59': {4: nan, 5: nan}, 'Unnamed: 60': {4: nan, 5: nan}, 'Unnamed: 61': {4: nan, 5: nan}, 'Unnamed: 62': {4: nan, 5: nan}, 'Unnamed: 63': {4: nan, 5: nan}, 'Unnamed: 64': {4: nan, 5: nan}, 'Unnamed: 65': {4: nan, 5: nan}, 'Unnamed: 66': {4: nan, 5: nan}, 'Unnamed: 67': {4: nan, 5: nan}, 'Unnamed: 68': {4: nan, 5: nan}, 'Unnamed: 69': {4: nan, 5: nan}, 'Unnamed: 70': {4: nan, 5: nan}, 'Unnamed: 71': {4: nan, 5: nan}, 'Unnamed: 72': {4: nan, 5: nan}, 'Unnamed: 73': {4: nan, 5: nan}, 'Unnamed: 74': {4: nan, 5: nan}, 'Unnamed: 75': {4: nan, 5: nan}, 'Unnamed: 76': {4: nan, 5: nan}, 'Unnamed: 77': {4: nan, 5: nan}, 'Unnamed: 78': {4: nan, 5: nan}, 'Unnamed: 79': {4: nan, 5: nan}, 'Unnamed: 80': {4: nan, 5: nan}, 'Unnamed: 81': {4: nan, 5: nan}, 'Unnamed: 82': {4: nan, 5: nan}, 'Unnamed: 83': {4: nan, 5: nan}, 'Unnamed: 84': {4: nan, 5: nan}, 'Unnamed: 85': {4: nan, 5: nan}, 'Unnamed: 86': {4: nan, 5: nan}, 'Unnamed: 87': {4: nan, 5: nan}, 'Unnamed: 88': {4: nan, 5: nan}, 'Unnamed: 89': {4: nan, 5: nan}, 'Unnamed: 90': {4: nan, 5: nan}, 'Unnamed: 91': {4: nan, 5: nan}, 'Unnamed: 92': {4: nan, 5: nan}, 'Unnamed: 93': {4: nan, 5: nan}, 'Unnamed: 94': {4: nan, 5: nan}, 'Unnamed: 95': {4: nan, 5: nan}, 'Unnamed: 96': {4: nan, 5: nan}, 'Unnamed: 97': {4: nan, 5: nan}, 'Unnamed: 98': {4: nan, 5: nan}, 'Unnamed: 99': {4: nan, 5: nan}, 'Unnamed: 100': {4: nan, 5: nan}, 'Unnamed: 101': {4: nan, 5: nan}, 'Unnamed: 102': {4: nan, 5: nan}, 'Unnamed: 103': {4: nan, 5: nan}, 'Unnamed: 104': {4: nan, 5: nan}, 'Unnamed: 105': {4: nan, 5: nan}, 'Unnamed: 106': {4: nan, 5: nan}, 'Unnamed: 107': {4: nan, 5: nan}, 'Unnamed: 108': {4: nan, 5: nan}, 'Unnamed: 109': {4: nan, 5: nan}, 'Unnamed: 110': {4: nan, 5: nan}, 'Unnamed: 111': {4: nan, 5: nan}, 'Unnamed: 112': {4: nan, 5: nan}, 'Unnamed: 113': {4: nan, 5: nan}, 'Unnamed: 114': {4: nan, 5: nan}, 'Unnamed: 115': {4: nan, 5: nan}, 'Detailed image_y': {4: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Deer.jpg', 5: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Bear_Gray.jpg'}, 'image_urls': {4: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Deer.jpg', 5: 'https://bebebamboo.com.sg/newshop/images/backup/detailed/Romper_Bear_Gray.jpg'}, 'Options': {4: '6-12m', 5: '3-6m'}, 'item_group_id': {4: 'BBFTDR', 5: 'BBFTPBG'}}


Can you please elaborate a little more? Please let us know what is 'expected output' and 'current output' clearly.

If you see the full_df's Options column, it has some values as []. When I apply my unnset function it removes the rows which have Options as  []. So how can ignore this? I don't want to remove products which have Options valuse as [].

Okay! Can you provide some sample data?

How can I add csv here?

Just give a sample with 5 - 7 rows. Or else check this out.

I have converted the full_df into dict and added the sample , please use that.

I tried to print col_flat. It printed something like this Options I         4       6 4       - 4       1 4       2 4       m 5       3 5       - 5       6 5       m     is it correct?
