Metaphysical Contemplation and Speculation There are many parables and stories from the Buddha's life suggesting he valued above all what is practical and conducive to virtue. He discouraged metaphysical speculation and musings. 

Given without knowledge one can only speculate, does Buddhism suggest one should delay certain questions until one can empirically get a definite answer?

For example, the jhanas are not immediately accessible, and thinking about them might lead to speculation. What is speculation in Buddhism? Should one always restrict their attention and cognition to what is immediately accessible?

I guess I could also add: How does one know when one is speculating?
OP: Given without knowledge one can only speculate, does Buddhism suggest one should delay certain questions until one can empirically
  get a definite answer?
  
  OP: For example, the jhanas are not immediately accessible, and thinking about them might lead to speculation. What is speculation in
  Buddhism? Should one always restrict their attention and cognition to
  what is immediately accessible?


The Buddha certainly valued pragmatism and discouraged metaphysical speculation.

Does this mean that ALL speculation is discouraged?

The answer to this is NO. Only speculation that is not useful towards the ending of suffering is discouraged.

For e.g. when did samsara first begin? How did the universe originate? What's the range of the Buddha's abilities? How do the consequences of karma work precisely? You can find some of these in Acintita Sutta and also the Parable of the Poisoned Arrow. Speculation on topics like these were strongly discouraged.

As for topics that are useful towards the ending of suffering, speculation is not discouraged. Without investigation, contemplation, reflection and speculation, it's impossible for a learner to learn and understand new things. Correcting your mistaken speculations is a great learning method.

Over time, as you learn and understand more aspects of the Dhamma and gain more insight, your speculations would be replaced by wisdom. It's like putting together a jigsaw puzzle of which you do not know how the final picture looks like. But it gets clearer over time.

Also, in the suttas, asking difficult questions based on speculative thinking was not just allowed by the Buddha but also used by him as a tool to convey teachings.

For example, from SN 12.46:


  As he was sitting there he said to the Blessed One: "What now, Master
  Gotama: Is the one who acts the same one who experiences [the results
  of the act]?"
  
  [The Buddha:] "[To say,] 'The one who acts is the same one who
  experiences,' is one extreme."
  
  [The brahman:] "Then, Master Gotama, is the one who acts someone other
  than the one who experiences?"
  
  [The Buddha:] "[To say,] 'The one who acts is someone other than the
  one who experiences,' is the second extreme. Avoiding both of these
  extremes, the Tathagata teaches the Dhamma by means of the middle:
  (explains dependent origination)
  
  When this was said, the brahman said to the Blessed One: "Magnificent,
  Master Gotama! Magnificent! Just as if he were to place upright what
  was overturned, to reveal what was hidden, to point out the way to one
  who was lost, or to carry a lamp into the dark so that those with eyes
  could see forms, in the same way has Master Gotama — through many
  lines of reasoning — made the Dhamma clear. I go to Master Gotama for
  refuge, to the Dhamma, &amp; to the community of monks. May Master Gotama
  remember me as a lay follower who has gone for refuge from this day
  forward, for life."


And there are many more such examples in the suttas. In the Attakari Sutta, we find someone speculating that there is no self-doer and he was corrected by the Buddha.

Of course, one also needs to practise the teachings (through the Noble Eightfold Path) and not just contemplate it. But, investigation, contemplation, reflection and speculation forms part of the journey of learning the Dhamma, in order to foster Right View, which is the forerunner of the path.


  OP: I guess I could also add: How does one know when one is speculating?


If you have doubt (vicikiccha) in your mind, then you're most likely speculating about the Dhamma. The explanation of doubt in Buddhism from this Wikipedia page is:


  The reality of vicikicchā is not the same as what we mean by doubt in
  conventional language. Vicikicchā is not doubt about someone's name or
  about the weather. Vicikicchā is doubt about realities, about nāma and
  rūpa, about cause and result, about the four noble Truths, about the
  “Dependent Origination”.


Another sign of mistaken speculation is that it may lead you towards the three poisons of lust/ greed, aversion and delusion. A good example of this is the Vesali Sutta - in which some monks who misunderstood the Dhamma committed suicide.