Google Cloud Dataflow: Dataflow programming model taking same computational time as it would take on a regular VM machine?
Apache Beam, and Dataflow are able to parallelize your computation based on the input that comes into it.
If you have a single computation to apply, and this computation takes one hour, then Beam will not be able to speed up your computation. Beam can help you if you need to apply the same computation multiple times to different elements (or data points).
You should also consider things such as overhead of running the computation in a distributed fashion (data copying, network calls, etc).
So, to be able to answer your question: How may individual "data points" (how many lines) are there in your GCS file? Is it possible to parallelize the computation over each one? How long does it take to process each one?
I am trying google cloud's dataflow service which is useful for efficient computation time. My code has the following programming model for the dataflow pipeline:

start=(p | "read" >> beam.io.ReadFromText("gcs path")) 
  end= start | "data_generation" >> beam.Pardo(PerfromFunction)

What I am doing:
PerformFunction is a regular Python function which contains a few series of functions for data-generation purpose. My problem is that when I run this function on a regular VM of n1-standard-16 on a single processor, it takes around 1 hour to complete the whole process.
Why I opted Dataflow:
I then decided to go for Dataflow where a ParDo function performs Multi-Threading of the given function and obviously to reduce the computational time from 1 hour to less than 1 hour.
The Problem:
After running a Dataflow job with the above-mentioned programming model, I came to realize that Dataflow is still taking around 1 hour to complete the entire process which is mentioned as wall-time on the GCP Dataflow UI. I then logged in to the worker machine and saw the resource utilization using the command htop and found that the machine was only utilizing one processor with 60% average usage.
Expected Results or Suggestions:
1. Can multiprocessing be done in the Dataflow worker Cluster?
2. Is my programming model very limited and wrong?
3. ParDo function does not seem to reduce the computational time as expected, What do you think I am doing wrong here?
PS- Owing to some protocols, I can not share the code. Thank you for understanding. Also please correct me if I wrongly understand dataflow at some point.