Autonomous vehicles are becoming more common in city transportation. Companies will begin to find a need to teach these vehicles smart city fleet coordination. Currently, simulation based modeling along with hand coded rules dictate the decision making of these autonomous vehicles. We believe that complex intelligent behavior can be learned by these agents through Reinforcement Learning. In this paper, we discuss our work for solving this system by adapting the Deep Q-Learning (DQN) model to the multi-agent setting. Our approach applies deep reinforcement learning by combining convolutional neural networks with DQN to teach agents to fulfill customer demand in an environment that is partially observable to them. We also demonstrate how to utilize transfer learning to teach agents to balance multiple objectives such as navigating to a charging station when its energy level is low. The two evaluations presented show that our solution has shown that we are successfully able to teach agents cooperation policies while balancing multiple objectives.
The main contribution of the paper seems to be the application to this problem, plus minor algorithmic/problem-setting contributions that consist in considering partial observability and to balance multiple objectives. On one hand, fleet management is an interesting and important problem. On the other hand, although the experiments are well designed and illustrative, the approach is only tested in a small 7x7 grid and 2 agents and in a 10x10 grid with 4 agents. In spirit, these simulations are similar to those in the original paper by M. Egorov. Since the main contribution is to use an existing algorithm to tackle a practical application, it would be more interesting to tweak the approach until it is able to tackle a more realistic scenario (mainly larger scale, but also more realistic dynamics with traffic models, real data, etc.).
Simulation results compare MADQN with Dijkstra's algorithm as a baseline, which offers a myopic solution where each agent picks up the closest customer. Again, since the main contribution is to solve a specific problem, it would be worthy to compare with a more extensive benchmark, including state of the art algorithms used for this problem (e.g., heuristics and metaheuristics). 
The paper is clear and well written. There are several minor typos and formatting errors (e.g., at the end of Sec. 3.3, the authors mention Figure 3, which seems to be missing, also references [Egorov, Maxim] and [Palmer, Gregory] are bad formatted). 
-- Comments and questions to the authors:
1. In the introduction, please, could you add references to what is called "traditional solutions"?
2. Regarding the partial observability, each agent knows the location of all agents, including itself, and the location of all obstacles and charging locations; but it only knows the location of customers that are in its vision range. This assumption seems reasonable if a central station broadcasts all agents' positions and customers are only allowed to stop vehicles in the street, without ever contacting the central station; otherwise if agents order vehicles in advance (e.g., by calling or using an app) the central station should be able to communicate customers locations too. On the other hand, if no communication with the central station is allowed, then positions of other agents may be also partial observable. In other words, the proposed partial observability assumption requires some further motivation. Moreover, in Sec. 4.3, it is said that agents can see around them +10 spaces away; however, experiments are run in 7x7 and 10x10 grid worlds, meaning that the agents are able to observe the grid completely.
3. The fact that partial observability helped to alleviate the credit-assignment noise caused by the missing customer penalty might be an artefact of the setting. For instance, since the reward has been designed arbitrarily, it could have been defined as giving a penalty for those missing customers that are at some distance of an agent.
4. Please, could you explain the last sentence of Sec. 4.3 that says "The drawback here is that the agents will not be able to generalize to other unseen maps that may have very different geographies." In particular, how is this sentence related to partial observability?