Building An Aws Pipeline, Data Science, Big Data, AWS, Towards Data Science.
load a large chunk of data to RAM. Let’s create a config file on our local machine named config.json. The config file will look as follows: It deals with our resource manager, YARN (Yet Another Resource Negotiator), and tells us how much memory can be allocated, for our entire application, as well