title: finding sum of values in a nested dictionary in python
body: <p>I have around around 20000 text files, numbered 5.txt,10.txt and so on..</p>
  
  <p>I am storing the filepaths of these files in a list "list2" that i have created.</p>
  
  <p>I also  have a text file "temp.txt" with a list of 500 words</p>
  
  <pre><code>vs
  mln
  money
  </code></pre>
  
  <p>and so on..</p>
  
  <p>I am storing these words in another list "list" that i have created.</p>
  
  <p>Now i create a nested dictionary d2[file][word]=frequency count of "word" in "file"</p>
  
  <p>Now,</p>
  
  <p>I need to iterate through these words for each text file as,</p>
  
  <p>i am trying to get the following output :</p>
  
  <pre><code>filename.txt- sum(d[filename][word]*log(prob))
  </code></pre>
  
  <p>Here, filename.txt is of the form 5.txt,10.txt and so on...</p>
  
  <p>"prob",which is a value that i have already  obtained</p>
  
  <p>I basically need to find the sum of the inner keys'(words) values, (which is the frequency of the word) for every outer key(file).</p>
  
  <p>Say:</p>
  
  <pre><code>d['5.txt']['the']=6
  </code></pre>
  
  <p>here "the" is my word and "5.txt" is the file.Now 6 is the number of times "the" occurs in "5.txt".</p>
  
  <p>Similarly:</p>
  
  <pre><code>d['5.txt']['as']=2.
  </code></pre>
  
  <p>I need to find the sum of the dictionary values.</p>
  
  <p>So,here for 5.txt: i need my answer to be :</p>
  
  <pre><code>6*log(prob('the'))+2*log(prob('as'))+...`(for all the words in list)
  </code></pre>
  
  <p>I need this to be done for all the files. </p>
  
  <p>My problem lies in the part where I am supposed to iterate through the nested dictionary</p>
  
  <pre><code>import collections, sys, os, re
  
  sys.stdout=open('4.txt','w')
  from collections import Counter
  from glob import glob
  
  folderpath='d:/individual-articles'
  folderpaths='d:/individual-articles/'
  counter=Counter()
  filepaths = glob(os.path.join(folderpath,'*.txt'))
  
  
  #test contains: d:/individual-articles/5.txt,d:/individual,articles/10.txt,d:/individual-articles/15.txt and so on...
  with open('test.txt', 'r') as fi:
      list2= [line.strip() for line in fi]
  
  
  #temp contains the list of words
  with open('temp.txt', 'r') as fi:
      list= [line.strip() for line in fi]
  
  
  #the dictionary that contains d2[file][word]
  d2 =defaultdict(dict)
  for fil in list2:
      with open(fil) as f:
         path, name = os.path.split(fil)
         words_c = Counter([word for line in f for word in line.split()])
         for word in list:
             d2[name][word] = words_c[word]
  
  
  
  #this portion is also for the generation of dictionary "prob",that is generated from file 2.txt can be overlooked!
  with open('2.txt', 'r+') as istream:
  for line in istream.readlines():
      try:
          k,r = line.strip().split(':')
          answer_ca[k.strip()].append(r.strip())
      except ValueError:
          print('Ignoring: malformed line: "{}"'.format(line))
  
  
  
  
  #my problem lies here
  items = d2.items()
  small_d2 = dict(next(items) for _ in range(10))
  for fil in list2:
      total=0
      for k,v in small_d2[fil].items():
          total=total+(v*answer_ca[k])
      print("Total of {} is {}".format(fil,total))
  </code></pre>