title: Performance problems when using MongoDB Aggregation Framework through C#
body: <p>I'm having performance problems when using the MongoDB Aggregation Framework through C#. An aggregation which works fast through Mongo shell takes forever when executed with C#.</p>
  
  <p>Before trying to call the framework through C#, I executed the following aggregation through Mongo shell to check that everything works:</p>
  
  <pre><code>db.runCommand(
      {
          aggregate: "actions", 
          pipeline : 
          [
              { $match : { CustomerAppId : "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName : "install"}}, 
              { $group : { _id : { CustomerAppId:"$CustomerAppId",ActionDate:"$ActionDate" }, count : { $sum : 1 } }}
          ]
      });
  </code></pre>
  
  <p>The script <strong>executed in &lt; 500ms</strong> and returns the expected around 200 results (The CustomerAppId is defined as a string in the database. It's not possible to use GUIDs with aggregation framework.).</p>
  
  <p>Then, I ported the same script to C#:</p>
  
  <pre><code>var pipeline = new BsonArray
          {
              new BsonDocument
                  {
                      {
                          "$match", 
                          new BsonDocument
                              {
                                  {"CustomerAppId", "f5357224-b1a8-4f1a-8ea2-a06a00ca597a"},
                                  {"ActionName", "install"}
                              }
                      },
                      { "$group", 
                          new BsonDocument
                              {
                                  { "_id", new BsonDocument
                                               {
                                                   {
                                                       "CustomerAppId","$CustomerAppId"
                                                   },
                                                   {
                                                       "ActionName","$ActionName"
                                                   }
                                               } 
  
                                  },
                                  {
                                      "Count", new BsonDocument
                                                   {
                                                       {
                                                           "$sum", 1
                                                       }
                                                   }
                                  }
                              } 
                    }
              }
          };
  
  
  var command = new CommandDocument
  {
      { "aggregate", "actions" },
      { "pipeline", pipeline }
  };
  </code></pre>
  
  <p>(Please let me know if there's an easier way to write the aggregation in C# :) )</p>
  
  <p>Which I'm executing like this:</p>
  
  <pre><code>var result = db.RunCommand(command);
  </code></pre>
  
  <p>The problem is that <strong>it kills the server</strong>: The CPU and mem usage go way up. When I check db.currentOp(), I can see the aggregate operation but I eventually have to kill it using db.killOp(1281546):</p>
  
  <pre><code>"opid" : 1281546,
  "active" : true,
  "secs_running" : 294,
  "op" : "query",
  "ns" : "database.actions",
  "query" : {
          "aggregate" : "actions",
          "pipeline" : [
                  {
                          "$match" : {
                                  "CustomerAppId" : "f5357224-b1a8-4f1a-8ea2-a06a00ca597a",
                                  "ActionName" : "install"
                          },
                          "$group" : {
                                  "_id" : {
                                          "CustomerAppId" : "$CustomerAppId",
                                          "ActionName" : "$ActionName"
                                  },
                                  "Count" : {
                                          "$sum" : 1
                                  }
                          }
                  }
          ]
  },
  </code></pre>
  
  <p>To me the operation looks completely fine and similar to the script I run directly from mongo shell. It feels like running the aggregation through C# causes the MongoDB to miss the index and it's doing a table scan for all the ~6 million documents in the collection. </p>
  
  <p>Any ideas? </p>
  
  <p><strong>Update: Logs</strong></p>
  
  <p>Thanks to cirrus' suggestion, I enabled the verbose logging and then used tail to get the queries. And <strong>they are different</strong>! So I think there is something wrong in my C# port. Any ideas on how to format the query correctly?</p>
  
  <p>The query when executed through <strong>shell</strong>:</p>
  
  <pre><code>Mon Oct  8 15:00:13 [conn1] run command database.$cmd { aggregate: "actions", pipeline: [ { $match: { CustomerAppId: "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName: "install" } }, { $group: { _id: { CustomerAppId: "$CustomerAppId", ActionDate: "$ActionDate" }, count: { $sum: 1.0 } } } ] }
  Mon Oct  8 15:00:13 [conn1] command database.$cmd command: { aggregate: "actions", pipeline: [ { $match: { CustomerAppId: "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName: "install" } }, { $group: { _id: { CustomerAppId: "$CustomerAppId", ActionDate: "$ActionDate" }, count: { $sum: 1.0 } } } ] } ntoreturn:1 keyUpdates:0 locks(micros) r:27944 reslen:12705 29ms
  </code></pre>
  
  <p>And the query when executed through C#:</p>
  
  <pre><code>Mon Oct  8 15:00:16 [conn8] run command database.$cmd { aggregate: "actions", pipeline: [ { $match: { CustomerAppId: "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName: "install" }, $group: { _id: { CustomerAppId: "$CustomerAppId", ActionDate: "$ActionDate" }, Count: { $sum: 1 } } } ] }
  </code></pre>
  
  <p>Second line is missing, I suppose because the query doesn't finish.</p>
  
  <p>And here are the logs again for easier comparison. Script is up, C# down:</p>
  
  <pre><code>Mon Oct  8 15:00:13 [conn1] run command database.$cmd { aggregate: "actions", pipeline: [ { $match: { CustomerAppId: "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName: "install" } }, { $group: { _id: { CustomerAppId: "$CustomerAppId", ActionDate: "$ActionDate" }, count: { $sum: 1.0 } } } ] }
  Mon Oct  8 15:00:16 [conn8] run command database.$cmd { aggregate: "actions", pipeline: [ { $match: { CustomerAppId: "f5357224-b1a8-4f1a-8ea2-a06a00ca597a", ActionName: "install" }, $group: { _id: { CustomerAppId: "$CustomerAppId", ActionDate: "$ActionDate" }, Count: { $sum: 1 } } } ] }
  </code></pre>