Answers.Body: <p>Hoeffding’s inequality assumes any single hypothesis function remains the same across all samples, so for finite hypothesis set case all parameters of any hypothesis are known and fixed. Apparently for the infinite hypothesis set case parameters are unknown as a family of functions, union bound trick fails with Hoeffding’s inequality, so VC dimension is needed to effectively reduce to a finite hypothesis set.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-10T23:28:19.137Z
Answers.LastActivityDate: 2024-11-10T23:28:19.137Z
Answers.UserDisplayName: cinch
Body: <p>In machine learning, we use a hypothesis set <span class="math-container">$\mathcal{H}$</span> and Hoeffding
  Inequality for generalization theory.</p>
  <p>I have the following question:</p>
  <p>Case 1: When we have a finite and fixed hypothesis set <span class="math-container">$\mathcal{H}$</span>, do we have the parameter of hypotheses in the set known or unknown?</p>
  <p>Case 2: When we have an infinite hypothesis set <span class="math-container">$\mathcal{H}$</span>, do we have the parameter of hypotheses in the set known or unknown?</p>
  
Tags: <machine-learning><generalization>
Title: Unclear points about the nature of hypothesis sets in Hoeffding Inequality-based generalization theory
UserDisplayName: DSPinfinity
ThreadText: # Unclear points about the nature of hypothesis sets in Hoeffding Inequality-based generalization theory
  
  (Asked by: username_0 on 2024-11-10)
  
  In machine learning, we use a hypothesis set $\mathcal{H}$ and Hoeffding
  Inequality for generalization theory.
  
  I have the following question:
  
  Case 1: When we have a finite and fixed hypothesis set $\mathcal{H}$, do we have the parameter of hypotheses in the set known or unknown?
  
  Case 2: When we have an infinite hypothesis set $\mathcal{H}$, do we have the parameter of hypotheses in the set known or unknown?
  
  **Tags:** machine-learning, generalization
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-10. Score: 0
  
  Hoeffding’s inequality assumes any single hypothesis function remains the same across all samples, so for finite hypothesis set case all parameters of any hypothesis are known and fixed. Apparently for the infinite hypothesis set case parameters are unknown as a family of functions, union bound trick fails with Hoeffding’s inequality, so VC dimension is needed to effectively reduce to a finite hypothesis set.
  
  
  ---
  
  