title: Hadoop Based Automation
body: <p>I am currently learning about Hadoop for my project. My Project Goal is to develop a system which can trigger the tools in the cluster based on user input.
  It is a batch processing system. The system should support both sequential and non sequential computing.
  Since the size of data is huge we are using hadoop and mapreduce. 
  All the nodes in the cluster have all the tools necessary but it will use them according based on the input file format</p>
  
  <p>For example,</p>
  
  <p>If the user gives input in the form of pdf file. The file will be loaded into hdfs. The Master node will say some information to clusters to open the tools for processing the pdf file.
  If it is .jpeg or so the master will ask the cluster nodes to open different tools to process the jpeg file. </p>
  
  <p>Here tools are some general tools which are used to process the files given by the user. Based on file format different tools should trigger. These tools are free downloadable tools which can run on standalone machines.</p>
  
  <p>As of now we are doing this manually (taking a file and process it with a tool A and taking the output as input to tool B in a single machine). But goal is to automate it using Hadoop Cluters.</p>
  
  <p>I read about workflow tool Oozie which can help us to schedule and trigger other hadoop tools(Hive, Hbase etc) but I need to trigger non Hadoop tools (for ex.tools used for processing text file, pdf files , image). Can I use oozie to trigger non hadoop tools ? Is there a better way to do this with hadoop??</p>
  
  <p>I couldn't move forward , don't know where to start or which tools I need to learn for this  ?</p>
  
  <p>Can someone please let me know If you have any idea on how can I achieve this? </p>