Answers.Body: <p>If future breakthroughs of foundational models require even more compute-intensive techniques such as training multi-modal models with fine-grained temporal reasoning or personalized agents, access could indeed become more exclusive, creating a tiered AI ecosystem, at least in the LLM NLP related AI areas which could be a stepping stone of AGI,</p>
  <p>On the other hand, open-source projects like LLaMA, Falcon, and Bloom are democratizing access to high-quality LLMs. While these models might not always match the state-of-the-art of foundational models, they are increasingly competitive. And innovations like low-rank adaptation (LoRA), parameter-efficient fine-tuning (PEFT), and memory-reducing quantization enable users to fine-tune large models with significantly less compute. This allows individuals and smaller organizations to adapt pre-trained models for novel or domain specific use cases.</p>
  <p>Finally initiatives like distributed training such as <a href="https://en.wikipedia.org/wiki/EleutherAI" rel="nofollow noreferrer">EleutherAI</a> and <a href="https://en.wikipedia.org/wiki/Federated_learning" rel="nofollow noreferrer">federated learning</a> are exploring ways to decentralize the development of AI models, reducing dependence on centralized compute-rich entities.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-08T04:16:16.987Z
Answers.LastActivityDate: 2024-12-08T04:22:41.923Z
Answers.LastEditDate: 2024-12-08T04:22:41.923Z
Answers.UserDisplayName: cinch
Body: <p>I was looking at the progress of the more popular LLMs in the last few years and wondering whether in the near future what will happen is that through the use of semi-exhaustive methods only those with enough compute power will be able to generate useful information for non-trivial, novel problems and the rest of us will get the scraps.</p>
  <p>Some of this information may be afterwards integrated within the more traditional llms but this will be inadequate for solving particularly novel problems.</p>
  <p>Since I haven't really been following AI's progress, I am unsure of the likelihood of this happening in the foreseeable future.</p>
  <p>What do you think?</p>
  
Tags: <machine-learning><natural-language-processing>
Title: Where is machine learning leading to?
UserDisplayName: GEP
ThreadText: # Where is machine learning leading to?
  
  (Asked by: username_0 on 2024-12-07)
  
  I was looking at the progress of the more popular LLMs in the last few years and wondering whether in the near future what will happen is that through the use of semi-exhaustive methods only those with enough compute power will be able to generate useful information for non-trivial, novel problems and the rest of us will get the scraps.
  
  Some of this information may be afterwards integrated within the more traditional llms but this will be inadequate for solving particularly novel problems.
  
  Since I haven't really been following AI's progress, I am unsure of the likelihood of this happening in the foreseeable future.
  
  What do you think?
  
  **Tags:** machine-learning, natural-language-processing
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-12-08. Score: 1 | Accepted answer
  
  If future breakthroughs of foundational models require even more compute-intensive techniques such as training multi-modal models with fine-grained temporal reasoning or personalized agents, access could indeed become more exclusive, creating a tiered AI ecosystem, at least in the LLM NLP related AI areas which could be a stepping stone of AGI,
  
  On the other hand, open-source projects like LLaMA, Falcon, and Bloom are democratizing access to high-quality LLMs. While these models might not always match the state-of-the-art of foundational models, they are increasingly competitive. And innovations like low-rank adaptation (LoRA), parameter-efficient fine-tuning (PEFT), and memory-reducing quantization enable users to fine-tune large models with significantly less compute. This allows individuals and smaller organizations to adapt pre-trained models for novel or domain specific use cases.
  
  Finally initiatives like distributed training such as [EleutherAI](https://en.wikipedia.org/wiki/EleutherAI) and [federated learning](https://en.wikipedia.org/wiki/Federated_learning) are exploring ways to decentralize the development of AI models, reducing dependence on centralized compute-rich entities.
  
  
  ---
  
  