human
- How to evaluate -> leaderboard
    - https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard based on evaluation on benchmarks
    - https://lmsys.org/blog/2023-05-25-leaderboard/ crowdsourcing human evaluation based on ELo rating
    - https://tatsu-lab.github.io/alpaca_eval/ win rate vs the baseline Davinci003 evaluated by GPT4/Claude

 - Open source models
	-Llama and variations:
		* Llama, sizes range from 7b->65b, performance on benchmarks between GPT3 and GPT-3.5.
				not open to commercial use.
				ref: LLaMA: Open and Efficient Foundation Language Models 
		* Alpaca: fine-tuning of Llama using self-instruct, on par with text-davinci-003,
				https://crfm.stanford.edu/2023/03/13/alpaca.html
				ref: Self-Instruct: Aligning Language Models with Self-Generated Instructions
		* Vicuna: fine-tuning of Llama using  70K user-shared conversations gathered from ShareGPT.com
		* OpenLlama, OpenAlpaca. No OpenVicuna so far
		(do not appear in leaderboard, OpenLlama claim metric around those of Llama;
		no evaluation of OpenAlpaca, expect < Alpaca due to smaller dataset)
		in practice performance seems much worse (tokenizer bug??)
		ref https://github.com/openlm-research/open_llama
		      https://github.com/yxuansu/OpenAlpaca
	-  FastChat-T5: similar to Vicuna - fine-tuning  of FLAN-T5 on ShareGPT data 
		ref: https://github.com/lm-sys/FastChat#FastChat-T5
Write as single text the above elements
gpt
--- Reasoning ---
1. Is the question multiple-choice question?
No
2. Is it implied multiple-choice question?
No
3. If two experts were independently asked the question, will the final answer be exactly the same?
No

--- Answer ---
open