| text,label,label_name |
| Give me a clear definition of coral reefs.,0,QA |
| Summarize this 10-page document in under 100 words.,1,Summary |
| Explain the time complexity of this algorithm that parse a CSV.,2,Coding |
| Invent a fantasy world with magic based on music.,5,Creative |
| Extract the steps and ingredients from this recipe for traditional cuisine.,8,Extraction |
| Write a high-level summary of this report on space telescopes.,1,Summary |
| Why is the sky blue?,0,QA |
| Create a dialogue between two characters arguing about photography.,5,Creative |
| Write unit tests for this API endpoint.,2,Coding |
| Pull out the total from this contract.,8,Extraction |
| Render this quote into Arabic and explain any nuances.,6,Translation |
| "If the probability of rain is 31%, what is the chance it does not rain?",3,Reasoning |
| Summarize this article about renewable energy in one line.,1,Summary |
| Review this entire transcript and give the setup steps.,9,Long-context |
| Extract the key figures revenue from this legal document.,8,Extraction |
| Explain Mars exploration in plain terms.,0,QA |
| "Call the API to get data on skyscrapers, then summarize the response and save it to disk.",4,Agentic |
| Convert this English text to Korean and keep the tone casual.,6,Translation |
| Draft a whimsical flash-fiction piece about blockchain.,5,Creative |
| "Use the file tool to open notes.txt, then the search tool for volcanoes, then combine both.",4,Agentic |
| "Use the file tool to open notes.txt, then the search tool for oceanography, then combine both.",4,Agentic |
| Compose a haiku about the Kalahari desert.,5,Creative |
| "Here is the full dataset README for space telescopes (1,000-line). List the action items.",9,Long-context |
| "Which of these topics is this text about: skyscrapers, bees and pollination, the human brain?",7,Classification |
| "Coordinate a multi-agent system: one agent searches electric vehicles, another drafts a plan, and I review the result.",4,Agentic |
| Summarize this article about basketball in two sentences.,1,Summary |
| Explain the solar system in plain terms.,0,QA |
| Briefly summarize the plot of this story about the Great Barrier Reef.,1,Summary |
| Translate 'How are you today' into Portuguese.,6,Translation |
| How do you say the human genome in Greek?,6,Translation |
| What is the capital of Cairo?,0,QA |
| Why is the ocean salty?,0,QA |
| Create a Dockerfile for a Rust project.,2,Coding |
| List every key stats and numbers mentioned in this legal document on Renaissance art.,8,Extraction |
| What is 44 + 29?,10,Fast/simple |
| A train departs at 3:00 traveling at 67 mph; how far in 3 hours?,3,Reasoning |
| What is 7 + 5?,10,Fast/simple |
| Translate 'Where is the train station' into Swahili.,6,Translation |
| "Which of these topics is this text about: coral reefs, the stock market, the Silk Road?",7,Classification |
| What is the difference between the human brain and space telescopes?,0,QA |
| What is the difference between Renaissance art and the moon landing?,0,QA |
| What are the main causes of the immune system?,0,QA |
| What is 29 + 24?,10,Fast/simple |
| Give me a TL;DR of the following passage about the ocean floor.,1,Summary |
| Give me a clear definition of the industrial revolution.,0,QA |
| Briefly summarize the plot of this story about ancient Egypt.,1,Summary |
| Create a dialogue between two characters arguing about basketball.,5,Creative |
| Draft a whimsical flash-fiction piece about coral reefs.,5,Creative |
| Mary buys 4 apples at $2 each and a loaf for $4. Total cost?,3,Reasoning |
| What is 26 + 29?,10,Fast/simple |
| How do you say oceanography in Hindi?,6,Translation |
| "List every people, places, organizations mentioned in this product review on skyscrapers.",8,Extraction |
| A train departs at 7:00 traveling at 90 mph; how far in 2 hours?,3,Reasoning |
| Give me a clear definition of artificial intelligence.,0,QA |
| Render this quote into Italian and explain any nuances.,6,Translation |
| Explain climate change in beginner-friendly terms.,0,QA |
| Write a docstring and type hints for a Rust function that processes electric vehicles.,2,Coding |
| "Coordinate a multi-agent system: one agent searches the Roman Empire, another drafts a plan, and I review the result.",4,Agentic |
| Compose a haiku about the Silk Road.,5,Creative |
| Render this quote into Hindi and explain any nuances.,6,Translation |
| Use your registry of tools to deploy a model and report back in a JSON summary.,4,Agentic |
| "Coordinate a multi-agent system: one agent searches the French Revolution, another drafts a plan, and I review the result.",4,Agentic |
| Give me an executive recap of renewable energy.,1,Summary |
| Explain the internet of things in beginner-friendly terms.,0,QA |
| "Which of these topics is this text about: the Roman Empire, Renaissance art, oceanography?",7,Classification |
| What time is it in Sao Paulo right now?,10,Fast/simple |
| "List every people, places, organizations mentioned in this order confirmation on artificial intelligence.",8,Extraction |
| "There are 3 boxes, one holds a prize. If I open 3 wrong boxes, how many remain?",3,Reasoning |
| "Which of these topics is this text about: the Kalahari desert, photography, the internet of things?",7,Classification |
| "Using this long codebase README about electric vehicles, give the setup steps.",9,Long-context |
| Extract the steps and ingredients from this recipe for the moon landing.,8,Extraction |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: coral reefs.",4,Agentic |
| "Which of these numbers is the odd one out and why: [19, 5, 51, 70]?",3,Reasoning |
| Is this statement a fact or an opinion: 'skyscrapers is largely misunderstood.'?,7,Classification |
| Which topic does this paragraph best belong to? skyscrapers mentions recipes.,7,Classification |
| List every named entities and keywords mentioned in this order confirmation on the Kalahari desert.,8,Extraction |
| Is this statement a fact or an opinion: 'modern architecture is largely misunderstood.'?,7,Classification |
| Summarize this entire year of meeting notes covering the industrial revolution.,9,Long-context |
| Translate this recipe into Dutch.,6,Translation |
| Compose a haiku about the French Revolution.,5,Creative |
| Write a JavaScript script to monitor a log file.,2,Coding |
| Translate this paragraph into Portuguese: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Create a dialogue between two characters arguing about the Kalahari desert.,5,Creative |
| Write a short executive summary of this report on volcanoes.,1,Summary |
| Write a letter to my future self about climate change.,5,Creative |
| Convert this pseudocode into working Go code.,2,Coding |
| Translate these product instructions into Mandarin.,6,Translation |
| "Search the web for ancient Greek philosophy, then book a dinner and send me a summary.",4,Agentic |
| Give me a TL;DR of the following passage about the stock market.,1,Summary |
| Write a JavaScript script to rename all files.,2,Coding |
| Give me an executive recap of the solar system.,1,Summary |
| Draft a whimsical flash-fiction piece about renewable energy.,5,Creative |
| "Which of these topics is this text about: the human brain, the moon landing, basketball?",7,Classification |
| What is 24 + 49?,10,Fast/simple |
| Create a dialogue between two characters arguing about traditional cuisine.,5,Creative |
| "Here is a 1,000-line legal document. Analyze it and explain the compliance requirements.",9,Long-context |
| Write a letter to my future self about the Kalahari desert.,5,Creative |
| What is a hash collision?,0,QA |
| Who is Charles Darwin known for?,0,QA |
| "List every people, places, organizations mentioned in this contract on climate change.",8,Extraction |
| Write a docstring and type hints for a Go function that processes ancient Greek philosophy.,2,Coding |
| "Using this long user manual about ancient Greek philosophy, summarize the main conclusions.",9,Long-context |
| "Referencing this long legal document, list all renewal dates.",9,Long-context |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Reykjavik.",4,Agentic |
| What is the difference between traditional cuisine and the French Revolution?,0,QA |
| What is the difference between the internet of things and the moon landing?,0,QA |
| "Use the file tool to open config.json, then the search tool for the industrial revolution, then combine both.",4,Agentic |
| Create a dialogue between two characters arguing about wildlife conservation.,5,Creative |
| What is the difference between wildlife conservation and ancient Egypt?,0,QA |
| Write a SQL query to get the top 1 products by revenue.,2,Coding |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Paris'.,4,Agentic |
| Condense this chapter into the main takeaways about the Great Barrier Reef.,1,Summary |
| What is the difference between photography and ancient Greek philosophy?,0,QA |
| "If the probability of rain is 36%, what is the chance it does not rain?",3,Reasoning |
| "If the probability of rain is 52%, what is the chance it does not rain?",3,Reasoning |
| What is 30 + 1?,10,Fast/simple |
| What is the difference between deep-sea creatures and the Kalahari desert?,0,QA |
| Draft a whimsical flash-fiction piece about Mars exploration.,5,Creative |
| "Which of these topics is this text about: blockchain, skyscrapers, Renaissance art?",7,Classification |
| Summarize this entire year of meeting notes covering artificial intelligence.,9,Long-context |
| Give me the Korean equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| What is the difference between the industrial revolution and Renaissance art?,0,QA |
| Write a poem about the ocean and adventure.,5,Creative |
| How do you say coral reefs in Japanese?,6,Translation |
| "If the probability of rain is 40%, what is the chance it does not rain?",3,Reasoning |
| What is 6 + 47?,10,Fast/simple |
| Draft a whimsical flash-fiction piece about the immune system.,5,Creative |
| "Provide a brief summary of the customer interview, focusing on the moon landing.",1,Summary |
| What is 21 + 48?,10,Fast/simple |
| What are the main causes of ancient Greek philosophy?,0,QA |
| "Which of these topics is this text about: the human genome, the Roman Empire, renewable energy?",7,Classification |
| A train departs at 7:00 traveling at 82 mph; how far in 2 hours?,3,Reasoning |
| Give me a clear definition of Mars exploration.,0,QA |
| Draft a melancholic flash-fiction piece about medieval castles.,5,Creative |
| "Which of these topics is this text about: the Roman Empire, medieval castles, the jazz era?",7,Classification |
| What is 27 + 30?,10,Fast/simple |
| "Which of these numbers is the odd one out and why: [78, 26, 14, 39]?",3,Reasoning |
| "Call the API to get data on deep-sea creatures, then summarize the response and save it to disk.",4,Agentic |
| Translate 'Where is the train station' into French.,6,Translation |
| Read the whole 300-page regulatory filing which covers the human genome and summarize the main conclusions.,9,Long-context |
| Which topic does this paragraph best belong to? the French Revolution mentions players.,7,Classification |
| "Tag this comment with one of: urgent, normal, low priority.",7,Classification |
| "Which of these numbers is the odd one out and why: [54, 59, 5, 43]?",3,Reasoning |
| Give me a TL;DR of the following passage about climate change.,1,Summary |
| "Which of these topics is this text about: climate change, the Kalahari desert, culinary history?",7,Classification |
| Debug this IndexError that occurs when running my script.,2,Coding |
| How do you say climate change in Swahili?,6,Translation |
| Mary buys 5 apples at $2 each and a loaf for $5. Total cost?,3,Reasoning |
| When was the wheel invented?,0,QA |
| Explain the time complexity of this algorithm that merge two sorted arrays.,2,Coding |
| "Which of these numbers is the odd one out and why: [19, 41, 12, 27]?",3,Reasoning |
| Make a concise summary of the key points in this transcript about electric vehicles.,1,Summary |
| Extract the main keyword phrases and topics from this passage on the human brain.,8,Extraction |
| Mary buys 5 apples at $3 each and a loaf for $5. Total cost?,3,Reasoning |
| How do you say bees and pollination in Greek?,6,Translation |
| Extract all product names present in the following medical note.,8,Extraction |
| Create a dialogue between two characters arguing about culinary history.,5,Creative |
| Which topic does this paragraph best belong to? coral reefs mentions flights.,7,Classification |
| Render this quote into German and explain any nuances.,6,Translation |
| Write a letter to my future self about renewable energy.,5,Creative |
| Improve the performance of my Python program that find duplicates.,2,Coding |
| "Which of these topics is this text about: medieval castles, the Silk Road, the Great Barrier Reef?",7,Classification |
| Translate this paragraph into Japanese: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Translate this paragraph into Spanish: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Translate this paragraph into German: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| "Which of these topics is this text about: modern architecture, the moon landing, the Silk Road?",7,Classification |
| "Provide a brief summary of the product review, focusing on the immune system.",1,Summary |
| Which topic does this paragraph best belong to? photography mentions recipes.,7,Classification |
| "Read this 1,000-line regulatory filing and answer the question about cost.",9,Long-context |
| Write a letter to my future self about the industrial revolution.,5,Creative |
| Explain Renaissance art in technical terms.,0,QA |
| "Search the web for the Kalahari desert, then book a hotel and send me a summary.",4,Agentic |
| "If 5 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| What is the difference between volcanoes and coral reefs?,0,QA |
| Design a schema for an app that tracks coral reefs.,2,Coding |
| Write a docstring and type hints for a Rust function that processes culinary history.,2,Coding |
| Draft a melancholic flash-fiction piece about space telescopes.,5,Creative |
| "If the probability of rain is 84%, what is the chance it does not rain?",3,Reasoning |
| Give me the Swahili equivalent of this idiom: 'spill the beans'.,6,Translation |
| "Given this lengthy source, extract the section that discusses costs.",9,Long-context |
| Localize this ad copy about basketball into Hindi.,6,Translation |
| What is the difference between electric vehicles and skyscrapers?,0,QA |
| What is 49 + 19?,10,Fast/simple |
| "There are 7 boxes, one holds a prize. If I open 2 wrong boxes, how many remain?",3,Reasoning |
| How do you say the human brain in Korean?,6,Translation |
| "Read this 1,000-line contract and summarize the key provisions.",9,Long-context |
| Summarize this article about ancient Greek philosophy in two sentences.,1,Summary |
| How does photosynthesis actually work?,0,QA |
| Extract the steps and ingredients from this recipe for the Kalahari desert.,8,Extraction |
| Give me a one-sentence recap of renewable energy.,1,Summary |
| "Coordinate a multi-agent system: one agent searches modern architecture, another drafts a plan, and I review the result.",4,Agentic |
| "Which of these numbers is the odd one out and why: [10, 4, 27, 23]?",3,Reasoning |
| Write a poem about the ocean and loneliness.,5,Creative |
| "Which of these topics is this text about: the industrial revolution, the human genome, the stock market?",7,Classification |
| How do you say ancient Egypt in Spanish?,6,Translation |
| When was the printing press invented?,0,QA |
| What is 39 + 20?,10,Fast/simple |
| Read the whole 50-page contract which covers electric vehicles and identify contradictions.,9,Long-context |
| Which topic does this paragraph best belong to? the stock market mentions flights.,7,Classification |
| Extract all person names and geographic locations from the text about traditional cuisine.,8,Extraction |
| Extract the key entities from this report about ancient Egypt.,8,Extraction |
| What is 27 + 10?,10,Fast/simple |
| Make a concise summary of the key points in this transcript about volcanoes.,1,Summary |
| What is 23 + 49?,10,Fast/simple |
| Translate 'How are you today' into Hindi.,6,Translation |
| "Extract named entities (people, places, organizations) from this text about volcanoes.",8,Extraction |
| Find and extract ingredients and geographic locations from this text.,8,Extraction |
| How many continents are on Earth?,0,QA |
| "If the probability of rain is 80%, what is the chance it does not rain?",3,Reasoning |
| Analyze this full 200-page report on the Roman Empire and summarize the conclusions.,9,Long-context |
| Create a dialogue between two characters arguing about deep-sea creatures.,5,Creative |
| Draft a funny flash-fiction piece about the human genome.,5,Creative |
| What is 22 + 34?,10,Fast/simple |
| Briefly summarize the plot of this story about the immune system.,1,Summary |
| Summarize this entire year of meeting notes covering the Kalahari desert.,9,Long-context |
| "Argue whether a self-driving car should protect the passenger or a pedestrian, using ethical reasoning.",3,Reasoning |
| "Search the web for climate change, then book a dinner and send me a summary.",4,Agentic |
| Condense this chapter into the main takeaways about traditional cuisine.,1,Summary |
| "Given this 25-page regulatory filing about ancient Egypt, give the setup steps.",9,Long-context |
| "Which of these numbers is the odd one out and why: [4, 33, 79, 64]?",3,Reasoning |
| Is this statement a fact or an opinion: 'the moon landing is largely misunderstood.'?,7,Classification |
| Translate 'Where is the train station' into Korean.,6,Translation |
| Scan this invoice and extract every legal terms.,8,Extraction |
| Translate 'How are you today' into Mandarin.,6,Translation |
| Extract the key figures date from this invoice.,8,Extraction |
| Find and extract product names and person names from this text.,8,Extraction |
| Draft a funny flash-fiction piece about oceanography.,5,Creative |
| Give me the Spanish equivalent of this idiom: 'bite the bullet'.,6,Translation |
| Condense this chapter into the main takeaways about the internet of things.,1,Summary |
| "List every people, places, organizations mentioned in this report on wildlife conservation.",8,Extraction |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the French Revolution.",4,Agentic |
| "Which of these topics is this text about: artificial intelligence, volcanoes, medieval castles?",7,Classification |
| Determine whether this tweet about the Great Barrier Reef is positive or negative.,7,Classification |
| "Which of these numbers is the odd one out and why: [75, 10, 77, 49]?",3,Reasoning |
| Compose a haiku about photography.,5,Creative |
| Interpret this legal clause into plain Spanish.,6,Translation |
| "Given this 120-page legal document about the stock market, answer the question about cost.",9,Long-context |
| "Call the API to get data on bees and pollination, then summarize the response and save it to disk.",4,Agentic |
| Summarize the key findings from this research paper on basketball.,1,Summary |
| "Which of these topics is this text about: bees and pollination, the solar system, Mars exploration?",7,Classification |
| What are the main causes of photography?,0,QA |
| A train departs at 11:00 traveling at 72 mph; how far in 2 hours?,3,Reasoning |
| Localize this ad copy about the Kalahari desert into Russian.,6,Translation |
| What is the difference between quantum computing and artificial intelligence?,0,QA |
| Explain the solar system in beginner-friendly terms.,0,QA |
| What is 49 + 23?,10,Fast/simple |
| Summarize this article about the Silk Road in two sentences.,1,Summary |
| Extract the key entities from this meeting minutes about modern architecture.,8,Extraction |
| "There are 6 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| Add error handling and logging to this worker.,2,Coding |
| "Search the web for medieval castles, then book a hotel and send me a summary.",4,Agentic |
| Give me a one-sentence recap of modern architecture.,1,Summary |
| How do you say climate change in Spanish?,6,Translation |
| What is 6 + 40?,10,Fast/simple |
| List all phone numbers and email addresses found in this document.,8,Extraction |
| Extract all emails and phone numbers and geographic locations from the text about space telescopes.,8,Extraction |
| "Extract named entities (people, places, organizations) from this text about coral reefs.",8,Extraction |
| Which topic does this paragraph best belong to? Mars exploration mentions players.,7,Classification |
| Summarize this article about oceanography in three bullet points.,1,Summary |
| Extract the key entities from this insurance policy about space telescopes.,8,Extraction |
| Write a one-paragraph summary of this report on the Roman Empire.,1,Summary |
| "There are 9 boxes, one holds a prize. If I open 2 wrong boxes, how many remain?",3,Reasoning |
| Which topic does this paragraph best belong to? renewable energy mentions budgets.,7,Classification |
| Explain the industrial revolution in simple terms.,0,QA |
| "Call the API to get data on wildlife conservation, then summarize the response and save it to disk.",4,Agentic |
| Summarize this article about bees and pollination in two sentences.,1,Summary |
| Compose a haiku about the moon landing.,5,Creative |
| "Call the API to get data on Mars exploration, then summarize the response and save it to disk.",4,Agentic |
| Write a SQL query to get the top 10 customers by revenue.,2,Coding |
| Translate this recipe into Swahili.,6,Translation |
| Read the whole 25-page dataset README which covers renewable energy and identify contradictions.,9,Long-context |
| What is 9 + 7?,10,Fast/simple |
| Here is the full research paper for the internet of things (500-page). Extract the key clauses.,9,Long-context |
| Write a SQL query to get the top 4 products by revenue.,2,Coding |
| Write a Rust script to monitor a log file.,2,Coding |
| Draft a whimsical flash-fiction piece about bees and pollination.,5,Creative |
| "Which of these topics is this text about: basketball, photography, Mars exploration?",7,Classification |
| "Extract named entities (people, places, organizations) from this text about artificial intelligence.",8,Extraction |
| Here is the full thesis for the Kalahari desert (50-page). Extract the key clauses.,9,Long-context |
| Write a Java script to scrape a webpage.,2,Coding |
| "Which of these topics is this text about: climate change, electric vehicles, coral reefs?",7,Classification |
| "If the probability of rain is 79%, what is the chance it does not rain?",3,Reasoning |
| Give me the French equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| Design a schema for an app that tracks Renaissance art.,2,Coding |
| "List every people, places, organizations mentioned in this insurance policy on photography.",8,Extraction |
| Which topic does this paragraph best belong to? culinary history mentions flights.,7,Classification |
| Interpret this legal clause into plain Japanese.,6,Translation |
| Briefly summarize the plot of this story about Mars exploration.,1,Summary |
| Mary buys 4 apples at $2 each and a loaf for $5. Total cost?,3,Reasoning |
| Write a function that merge two sorted arrays and returns a generator.,2,Coding |
| Read the whole 50-page user manual which covers the solar system and summarize the key provisions.,9,Long-context |
| Give me a one-sentence recap of coral reefs.,1,Summary |
| Find and extract ingredients and prices from this text.,8,Extraction |
| Summarize the key findings from this research paper on the jazz era.,1,Summary |
| Briefly summarize the plot of this story about quantum computing.,1,Summary |
| "Use the file tool to open config.json, then the search tool for traditional cuisine, then combine both.",4,Agentic |
| "Using this long research paper about ancient Greek philosophy, compare the revisions.",9,Long-context |
| Give me the German equivalent of this idiom: 'spill the beans'.,6,Translation |
| Extract all prices and person names from the text about volcanoes.,8,Extraction |
| "Given this lengthy source, extract the section that discusses risks.",9,Long-context |
| Give me a TL;DR of the following passage about the industrial revolution.,1,Summary |
| Summarize this article about climate change in three bullet points.,1,Summary |
| Improve the performance of my TypeScript program that restructure a nested JSON.,2,Coding |
| Here is the full codebase README for the internet of things (120-page). Summarize it.,9,Long-context |
| Which topic does this paragraph best belong to? the Great Barrier Reef mentions players.,7,Classification |
| Process this 300-page research paper on electric vehicles; then identify contradictions.,9,Long-context |
| What is 35 + 7?,10,Fast/simple |
| "Provide a brief summary of the meeting notes, focusing on the Great Barrier Reef.",1,Summary |
| What is 29 + 21?,10,Fast/simple |
| How do you say oceanography in Portuguese?,6,Translation |
| Here is a 25-page contract. Analyze it and compare the revisions.,9,Long-context |
| Is this statement a fact or an opinion: 'the stock market is largely misunderstood.'?,7,Classification |
| Extract the key figures timestamp from this product review.,8,Extraction |
| Briefly summarize the plot of this story about basketball.,1,Summary |
| Which topic does this paragraph best belong to? the stock market mentions recipes.,7,Classification |
| Extract all ingredients present in the following contract.,8,Extraction |
| Use your registry of tools to update a database row and report back in a JSON summary.,4,Agentic |
| "Search the web for the French Revolution, then book a hotel and send me a summary.",4,Agentic |
| "Search the web for traditional cuisine, then book a hotel and send me a summary.",4,Agentic |
| "Pull the customer name, order id, and total from this order confirmation.",8,Extraction |
| Translate 'I would like a coffee' into French.,6,Translation |
| "Which of these topics is this text about: coral reefs, the internet of things, electric vehicles?",7,Classification |
| "Use the file tool to open config.json, then the search tool for skyscrapers, then combine both.",4,Agentic |
| "Extract named entities (people, places, organizations) from this text about the immune system.",8,Extraction |
| Review this entire thesis and return a structured overview.,9,Long-context |
| What is the difference between Mars exploration and the French Revolution?,0,QA |
| Condense this chapter into the main takeaways about climate change.,1,Summary |
| Read the whole 10-page codebase README which covers culinary history and compare the revisions.,9,Long-context |
| Write a letter to my future self about culinary history.,5,Creative |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the moon landing.",4,Agentic |
| "Given this 1,000-line regulatory filing about climate change, give the setup steps.",9,Long-context |
| Pull the key stats and numbers out of this report about basketball.,8,Extraction |
| What is 9 + 11?,10,Fast/simple |
| Is this statement a fact or an opinion: 'medieval castles is largely misunderstood.'?,7,Classification |
| "Given this 300-page transcript about renewable energy, summarize the key provisions.",9,Long-context |
| How do you say modern architecture in Korean?,6,Translation |
| "Which of these topics is this text about: oceanography, space telescopes, the jazz era?",7,Classification |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on photography.",4,Agentic |
| Is this statement a fact or an opinion: 'photography is largely misunderstood.'?,7,Classification |
| "Which of these numbers is the odd one out and why: [39, 1, 50, 58]?",3,Reasoning |
| Write a Java script to back up a database.,2,Coding |
| Create a dialogue between two characters arguing about the moon landing.,5,Creative |
| "Which of these topics is this text about: the ocean floor, the Kalahari desert, volcanoes?",7,Classification |
| "Which of these numbers is the odd one out and why: [31, 70, 41, 29]?",3,Reasoning |
| "Provide a brief summary of the customer interview, focusing on the stock market.",1,Summary |
| Convert this pseudocode into working Java code.,2,Coding |
| Which topic does this paragraph best belong to? the moon landing mentions recipes.,7,Classification |
| Explain volcanoes in simple terms.,0,QA |
| "Search the web for Mars exploration, then book a dinner and send me a summary.",4,Agentic |
| A train departs at 4:00 traveling at 52 mph; how far in 6 hours?,3,Reasoning |
| Translate this paragraph into Arabic: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Give me the Hindi equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| Write a letter to my future self about the human brain.,5,Creative |
| Give me a 3-bullet recap of ancient Greek philosophy.,1,Summary |
| "Return a JSON object with dates, amounts, and identifiers parsed from this meeting minutes.",8,Extraction |
| What is 1 + 36?,10,Fast/simple |
| Review this entire research paper and summarize the main conclusions.,9,Long-context |
| Read these 15 documents about the stock market and synthesize a single answer.,9,Long-context |
| Give me a clear definition of electric vehicles.,0,QA |
| Analyze this full 200-page report on coral reefs and summarize the conclusions.,9,Long-context |
| Give me the Italian equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| What is the difference between the stock market and space telescopes?,0,QA |
| Summarize this 50-page document in under 80 words.,1,Summary |
| How do I implement a linked list in Python?,2,Coding |
| What is 46 + 21?,10,Fast/simple |
| What is 32 + 15?,10,Fast/simple |
| Review this entire research paper and extract the risks.,9,Long-context |
| Is this statement a fact or an opinion: 'ancient Egypt is largely misunderstood.'?,7,Classification |
| Extract the steps and ingredients from this recipe for the immune system.,8,Extraction |
| "Here is the full research paper for ancient Egypt (1,000-line). Summarize it.",9,Long-context |
| "Provide a brief summary of the customer interview, focusing on the Kalahari desert.",1,Summary |
| Draft a melancholic flash-fiction piece about the internet of things.,5,Creative |
| Write a product description for a foldable drone.,5,Creative |
| "Pull the name, date, amount out of this legal document about ancient Egypt.",8,Extraction |
| What is 43 + 38?,10,Fast/simple |
| Compose a haiku about culinary history.,5,Creative |
| Make a concise summary of the key points in this transcript about the stock market.,1,Summary |
| What is 41 + 29?,10,Fast/simple |
| "Search the web for climate change, then book a flight and send me a summary.",4,Agentic |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: electric vehicles.",4,Agentic |
| Translate the following business email into Hindi.,6,Translation |
| "Process this 1,000-line regulatory filing on deep-sea creatures; then list every deadline.",9,Long-context |
| "Using this long thesis about the French Revolution, compare the revisions.",9,Long-context |
| Is this statement a fact or an opinion: 'the solar system is largely misunderstood.'?,7,Classification |
| "Call the API to get data on space telescopes, then summarize the response and save it to disk.",4,Agentic |
| Write a letter to my future self about blockchain.,5,Creative |
| Localize this ad copy about climate change into Swahili.,6,Translation |
| Write a song lyric about digital nostalgia.,5,Creative |
| Which topic does this paragraph best belong to? wildlife conservation mentions recipes.,7,Classification |
| Find and extract emails and phone numbers and prices from this text.,8,Extraction |
| Write a SQL query to get the top 3 customers by revenue.,2,Coding |
| "Provide a brief summary of the meeting notes, focusing on quantum computing.",1,Summary |
| "Call the API to get data on ancient Greek philosophy, then summarize the response and save it to disk.",4,Agentic |
| "If the probability of rain is 53%, what is the chance it does not rain?",3,Reasoning |
| "Which of these topics is this text about: climate change, traditional cuisine, wildlife conservation?",7,Classification |
| "Use the file tool to open data.csv, then the search tool for oceanography, then combine both.",4,Agentic |
| Is this statement a fact or an opinion: 'the internet of things is largely misunderstood.'?,7,Classification |
| Improve the performance of my TypeScript program that normalize a phone number.,2,Coding |
| Convert this English text to French and keep the tone casual.,6,Translation |
| "Process this 1,000-line user manual on the Great Barrier Reef; then identify contradictions.",9,Long-context |
| What is the difference between the stock market and quantum computing?,0,QA |
| "Which of these numbers is the odd one out and why: [6, 77, 4, 25]?",3,Reasoning |
| Read these 15 documents about blockchain and synthesize a single answer.,9,Long-context |
| What are the main causes of the Roman Empire?,0,QA |
| Extract the key entities from this legal document about ancient Egypt.,8,Extraction |
| Give me a TL;DR of the following passage about the Silk Road.,1,Summary |
| "Based on the full codebase README about the industrial revolution, identify contradictions.",9,Long-context |
| Summarize the key findings from this research paper on coral reefs.,1,Summary |
| Create a dialogue between two characters arguing about climate change.,5,Creative |
| Translate this paragraph into Mandarin: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Analyze this full 200-page report on basketball and summarize the conclusions.,9,Long-context |
| A train departs at 11:00 traveling at 74 mph; how far in 3 hours?,3,Reasoning |
| Which topic does this paragraph best belong to? the industrial revolution mentions flights.,7,Classification |
| Return a JSON object with dates and locations parsed from this legal document.,8,Extraction |
| What is the difference between traditional cuisine and the human brain?,0,QA |
| "Coordinate a multi-agent system: one agent searches ancient Egypt, another drafts a plan, and I review the result.",4,Agentic |
| Flip a coin for me.,10,Fast/simple |
| Translate this recipe into Korean.,6,Translation |
| "Which of these numbers is the odd one out and why: [38, 12, 34, 17]?",3,Reasoning |
| Read these 15 documents about the solar system and synthesize a single answer.,9,Long-context |
| Translate these product instructions into Korean.,6,Translation |
| "Pull the people, places, organizations out of this meeting minutes about the solar system.",8,Extraction |
| "Which of these numbers is the odd one out and why: [34, 69, 53, 31]?",3,Reasoning |
| What is the difference between medieval castles and the solar system?,0,QA |
| What is entropy?,0,QA |
| Extract all product names present in the following contract.,8,Extraction |
| "Which of these numbers is the odd one out and why: [37, 3, 63, 22]?",3,Reasoning |
| Explain medieval castles in plain terms.,0,QA |
| Use your registry of tools to update a database row and report back in a CSV.,4,Agentic |
| Translate this recipe into Russian.,6,Translation |
| What are the main causes of the moon landing?,0,QA |
| Give me a TL;DR of the following passage about ancient Greek philosophy.,1,Summary |
| Give me a clear definition of the moon landing.,0,QA |
| Convert this English text to Greek and keep the tone neutral.,6,Translation |
| Design a schema for an app that tracks the Silk Road.,2,Coding |
| What is 10 + 11?,10,Fast/simple |
| "Using this long thesis about the stock market, answer the question about cost.",9,Long-context |
| What is 32 + 20?,10,Fast/simple |
| "There are 2 boxes, one holds a prize. If I open 5 wrong boxes, how many remain?",3,Reasoning |
| Localize this ad copy about skyscrapers into Arabic.,6,Translation |
| Write a TypeScript script to rename all files.,2,Coding |
| Say hello.,10,Fast/simple |
| Give me a clear definition of climate change.,0,QA |
| What are the main causes of renewable energy?,0,QA |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Tokyo.",4,Agentic |
| Compose a haiku about blockchain.,5,Creative |
| Briefly summarize the plot of this story about the Silk Road.,1,Summary |
| Give me a one-word answer: wait.,10,Fast/simple |
| Draft a melancholic flash-fiction piece about the stock market.,5,Creative |
| "Pull the people, places, organizations out of this receipt about coral reefs.",8,Extraction |
| What are the main causes of deep-sea creatures?,0,QA |
| "Using the tools available, plan a data-pipeline workflow to achieve this goal.",4,Agentic |
| Mary buys 2 apples at $2 each and a loaf for $2. Total cost?,3,Reasoning |
| "Search the web for artificial intelligence, then book a hotel and send me a summary.",4,Agentic |
| Compose a haiku about the stock market.,5,Creative |
| "There are 2 boxes, one holds a prize. If I open 8 wrong boxes, how many remain?",3,Reasoning |
| What is the difference between skyscrapers and the moon landing?,0,QA |
| "Search the web for the Roman Empire, then book a flight and send me a summary.",4,Agentic |
| "Search the web for the solar system, then book a flight and send me a summary.",4,Agentic |
| How do you say climate change in Arabic?,6,Translation |
| "Coordinate a multi-agent system: one agent searches oceanography, another drafts a plan, and I review the result.",4,Agentic |
| Write a docstring and type hints for a TypeScript function that processes the ocean floor.,2,Coding |
| Process this 10-page user manual on volcanoes; then return a structured overview.,9,Long-context |
| Draft a whimsical flash-fiction piece about space telescopes.,5,Creative |
| "Which of these topics is this text about: climate change, the Kalahari desert, coral reefs?",7,Classification |
| Here is the full legal document for electric vehicles (50-page). List the action items.,9,Long-context |
| Write a one-paragraph summary of this report on Mars exploration.,1,Summary |
| Here is a 25-page research paper. Analyze it and return a structured overview.,9,Long-context |
| "Coordinate a multi-agent system: one agent searches Mars exploration, another drafts a plan, and I review the result.",4,Agentic |
| Write a one-paragraph summary of this report on quantum computing.,1,Summary |
| Summarize this article about the Kalahari desert in one line.,1,Summary |
| Design a schema for an app that tracks the French Revolution.,2,Coding |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the Great Barrier Reef.",4,Agentic |
| Localize this ad copy about the industrial revolution into Korean.,6,Translation |
| "There are 4 boxes, one holds a prize. If I open 8 wrong boxes, how many remain?",3,Reasoning |
| What is 46 + 32?,10,Fast/simple |
| "Process this 1,000-line research paper on the moon landing; then return a structured overview.",9,Long-context |
| Write a function that restructure a nested JSON and returns a list.,2,Coding |
| Improve the performance of my Go program that restructure a nested JSON.,2,Coding |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on space telescopes.",4,Agentic |
| "There are 7 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| Write a letter to my future self about the French Revolution.,5,Creative |
| What is 20 + 30?,10,Fast/simple |
| Give me an executive recap of modern architecture.,1,Summary |
| Return a JSON object with key stats and numbers parsed from this legal document.,8,Extraction |
| Build a an image resizer with proper error handling.,2,Coding |
| "Which of these topics is this text about: the Kalahari desert, volcanoes, artificial intelligence?",7,Classification |
| Which topic does this paragraph best belong to? the Kalahari desert mentions algorithms.,7,Classification |
| "Which of these numbers is the odd one out and why: [26, 9, 79, 14]?",3,Reasoning |
| Write a short executive summary of this report on Renaissance art.,1,Summary |
| What is the difference between bees and pollination and the human brain?,0,QA |
| Mary buys 2 apples at $3 each and a loaf for $2. Total cost?,3,Reasoning |
| Extract the key entities from this legal document about the Great Barrier Reef.,8,Extraction |
| What is 20 + 42?,10,Fast/simple |
| What is the difference between renewable energy and the moon landing?,0,QA |
| Explain electric vehicles in technical terms.,0,QA |
| What is 38 + 38?,10,Fast/simple |
| "If the probability of rain is 64%, what is the chance it does not rain?",3,Reasoning |
| Localize this ad copy about medieval castles into Arabic.,6,Translation |
| "Extract structured JSON with fields name, date, amount from this snippet.",8,Extraction |
| Determine whether this tweet about modern architecture is positive or negative.,7,Classification |
| Give me the Arabic equivalent of this idiom: 'bite the bullet'.,6,Translation |
| Process this 120-page user manual on bees and pollination; then identify contradictions.,9,Long-context |
| "Provide a brief summary of the meeting notes, focusing on the internet of things.",1,Summary |
| Determine whether this tweet about bees and pollination is positive or negative.,7,Classification |
| Translate these product instructions into Hindi.,6,Translation |
| Compose a haiku about the ocean floor.,5,Creative |
| Process this 10-page codebase README on artificial intelligence; then explain the compliance requirements.,9,Long-context |
| What is 20 + 11?,10,Fast/simple |
| "Search the web for traditional cuisine, then book a flight and send me a summary.",4,Agentic |
| Translate 'How are you today' into Arabic.,6,Translation |
| Which topic does this paragraph best belong to? the jazz era mentions flights.,7,Classification |
| Here is a 25-page regulatory filing. Analyze it and summarize the key provisions.,9,Long-context |
| "Use the file tool to open config.json, then the search tool for renewable energy, then combine both.",4,Agentic |
| Write a poem about the ocean and beginning again.,5,Creative |
| Extract the steps and ingredients from this recipe for ancient Egypt.,8,Extraction |
| Briefly summarize the plot of this story about deep-sea creatures.,1,Summary |
| Summarize this article about the internet of things in two sentences.,1,Summary |
| Draft a whimsical flash-fiction piece about basketball.,5,Creative |
| Is this statement a fact or an opinion: 'artificial intelligence is largely misunderstood.'?,7,Classification |
| What day of the week is tomorrow?,10,Fast/simple |
| "Search the web for volcanoes, then book a hotel and send me a summary.",4,Agentic |
| "Search the web for artificial intelligence, then book a dinner and send me a summary.",4,Agentic |
| Improve the performance of my Go program that find duplicates.,2,Coding |
| What year is it today?,10,Fast/simple |
| What is 13 + 20?,10,Fast/simple |
| Give me a clear definition of ancient Greek philosophy.,0,QA |
| What is the difference between photography and basketball?,0,QA |
| Translate 'I would like a coffee' into Swahili.,6,Translation |
| "Using this long codebase README about skyscrapers, answer the question about cost.",9,Long-context |
| Explain the immune system in plain terms.,0,QA |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on bees and pollination.",4,Agentic |
| What is 47 + 34?,10,Fast/simple |
| "Which of these numbers is the odd one out and why: [12, 69, 34, 52]?",3,Reasoning |
| What does LOL mean?,10,Fast/simple |
| Extract all legal terms present in the following order confirmation.,8,Extraction |
| Convert this English text to Portuguese and keep the tone casual.,6,Translation |
| Here is the full report for modern architecture (10-page). List the action items.,9,Long-context |
| Condense this chapter into the main takeaways about photography.,1,Summary |
| A train departs at 7:00 traveling at 63 mph; how far in 3 hours?,3,Reasoning |
| What is 48 + 29?,10,Fast/simple |
| How do I implement a binary search in Python?,2,Coding |
| Explain the internet of things in simple terms.,0,QA |
| What is 1 + 29?,10,Fast/simple |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on oceanography.",4,Agentic |
| Give me a 3-bullet recap of climate change.,1,Summary |
| What is 36 + 8?,10,Fast/simple |
| What is 43 + 36?,10,Fast/simple |
| Create a dialogue between two characters arguing about the solar system.,5,Creative |
| Find and extract person names and emails and phone numbers from this text.,8,Extraction |
| Extract the key entities from this insurance policy about skyscrapers.,8,Extraction |
| "Search the web for renewable energy, then book a hotel and send me a summary.",4,Agentic |
| Localize this ad copy about the moon landing into French.,6,Translation |
| Mary buys 3 apples at $3 each and a loaf for $3. Total cost?,3,Reasoning |
| Give me a clear definition of basketball.,0,QA |
| What is the difference between the Silk Road and volcanoes?,0,QA |
| Read this 120-page user manual and extract the recommendations.,9,Long-context |
| "Which of these topics is this text about: Mars exploration, machine learning, space telescopes?",7,Classification |
| Write a letter to my future self about the Silk Road.,5,Creative |
| Build a a login form with proper error handling.,2,Coding |
| "If the probability of rain is 68%, what is the chance it does not rain?",3,Reasoning |
| Mary buys 3 apples at $3 each and a loaf for $2. Total cost?,3,Reasoning |
| Give me a one-sentence recap of basketball.,1,Summary |
| "Which of these numbers is the odd one out and why: [46, 6, 70, 7]?",3,Reasoning |
| Extract the key entities from this receipt about blockchain.,8,Extraction |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the Great Barrier Reef.",4,Agentic |
| Write a short story about a time-traveling detective.,5,Creative |
| What is 33 + 46?,10,Fast/simple |
| "What comes next in the sequence: 2, 6, 12, 20, 42?",3,Reasoning |
| Read these 15 documents about oceanography and synthesize a single answer.,9,Long-context |
| Give me a 3-bullet recap of the industrial revolution.,1,Summary |
| "Read the whole 1,000-line codebase README which covers medieval castles and return a structured overview.",9,Long-context |
| How many legs does a spider have?,0,QA |
| Summarize this 50-page document in under 50 words.,1,Summary |
| What's the weather like in Sao Paulo today?,10,Fast/simple |
| Extract all emails and phone numbers present in the following news article.,8,Extraction |
| "Which of these topics is this text about: ancient Greek philosophy, the human brain, medieval castles?",7,Classification |
| "Coordinate a multi-agent system: one agent searches coral reefs, another drafts a plan, and I review the result.",4,Agentic |
| Write a JavaScript script to batch-resize images.,2,Coding |
| What are the main causes of the internet of things?,0,QA |
| What is 44 + 14?,10,Fast/simple |
| How do you say the industrial revolution in Arabic?,6,Translation |
| Draft a melancholic flash-fiction piece about the Kalahari desert.,5,Creative |
| What is the difference between skyscrapers and climate change?,0,QA |
| Convert this English text to Spanish and keep the tone neutral.,6,Translation |
| Summarize this entire year of meeting notes covering deep-sea creatures.,9,Long-context |
| Mary buys 3 apples at $1 each and a loaf for $4. Total cost?,3,Reasoning |
| What is 14 + 31?,10,Fast/simple |
| Read this 500-page legal document and list every deadline.,9,Long-context |
| What's the weather like in Cairo today?,10,Fast/simple |
| Improve the performance of my Java program that flatten a nested list.,2,Coding |
| "Call the API to get data on photography, then summarize the response and save it to disk.",4,Agentic |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: deep-sea creatures.",4,Agentic |
| Translate the following business email into Portuguese.,6,Translation |
| Translate these product instructions into Portuguese.,6,Translation |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Sydney.",4,Agentic |
| Design a schema for an app that tracks the solar system.,2,Coding |
| Give me the Swahili equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| Give me a one-sentence recap of the solar system.,1,Summary |
| Write a letter to my future self about the jazz era.,5,Creative |
| Compose a haiku about the industrial revolution.,5,Creative |
| What time is it in Tokyo right now?,10,Fast/simple |
| Read the whole 500-page report which covers medieval castles and list every deadline.,9,Long-context |
| What is 17 + 11?,10,Fast/simple |
| What is 30 + 21?,10,Fast/simple |
| Process this 500-page regulatory filing on ancient Greek philosophy; then identify contradictions.,9,Long-context |
| "Which of these topics is this text about: medieval castles, the human genome, oceanography?",7,Classification |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: space telescopes.",4,Agentic |
| Give me a clear definition of the Silk Road.,0,QA |
| Condense this long email into 1 sentence(s).,1,Summary |
| Mary buys 2 apples at $2 each and a loaf for $4. Total cost?,3,Reasoning |
| Scan this receipt and extract every key figures.,8,Extraction |
| Create a dialogue between two characters arguing about bees and pollination.,5,Creative |
| Draft a melancholic flash-fiction piece about blockchain.,5,Creative |
| Review this entire dataset README and summarize the main conclusions.,9,Long-context |
| Determine whether this tweet about culinary history is positive or negative.,7,Classification |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the immune system.",4,Agentic |
| "Process this 1,000-line user manual on oceanography; then compare the revisions.",9,Long-context |
| Create a dialogue between two characters arguing about the Great Barrier Reef.,5,Creative |
| Convert this pseudocode into working TypeScript code.,2,Coding |
| Find the memory leak in this Python service.,2,Coding |
| Extract the key figures date from this receipt.,8,Extraction |
| "Which of these numbers is the odd one out and why: [45, 79, 65, 14]?",3,Reasoning |
| A train departs at 11:00 traveling at 45 mph; how far in 2 hours?,3,Reasoning |
| Explain the jazz era in plain terms.,0,QA |
| Extract all emails and phone numbers and product names from the text about Mars exploration.,8,Extraction |
| What is 48 + 7?,10,Fast/simple |
| Analyze this full 200-page report on skyscrapers and summarize the conclusions.,9,Long-context |
| Read the whole 500-page research paper which covers wildlife conservation and extract the risks.,9,Long-context |
| Here is a 120-page dataset README. Analyze it and return a structured overview.,9,Long-context |
| Compose a haiku about the jazz era.,5,Creative |
| What is the difference between volcanoes and the internet of things?,0,QA |
| Which topic does this paragraph best belong to? coral reefs mentions recipes.,7,Classification |
| Give me a TL;DR of the following passage about ancient Egypt.,1,Summary |
| Process this 25-page contract on deep-sea creatures; then give the setup steps.,9,Long-context |
| A train departs at 3:00 traveling at 46 mph; how far in 5 hours?,3,Reasoning |
| Which topic does this paragraph best belong to? the stock market mentions players.,7,Classification |
| Here is a 300-page contract. Analyze it and answer the question about cost.,9,Long-context |
| "Use the file tool to open notes.txt, then the search tool for the ocean floor, then combine both.",4,Agentic |
| "Return a JSON object with people, places, organizations parsed from this order confirmation.",8,Extraction |
| A father is twice his son's age; in 4 years he'll be 1.5x. Find their ages.,3,Reasoning |
| What is the difference between deep-sea creatures and quantum computing?,0,QA |
| Write a function that sort a list of dicts and returns a list.,2,Coding |
| How do I implement a circular buffer in Python?,2,Coding |
| Translate 'I would like a coffee' into Portuguese.,6,Translation |
| Extract all organizations and person names from the text about the ocean floor.,8,Extraction |
| "Use the file tool to open notes.txt, then the search tool for coral reefs, then combine both.",4,Agentic |
| What is 6 + 31?,10,Fast/simple |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: medieval castles.",4,Agentic |
| Pull out the customer ID from this product review.,8,Extraction |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Sydney'.,4,Agentic |
| Write a letter to my future self about the solar system.,5,Creative |
| Condense this chapter into the main takeaways about the immune system.,1,Summary |
| Give me a clear definition of oceanography.,0,QA |
| Analyze this full 200-page report on volcanoes and summarize the conclusions.,9,Long-context |
| "Which of these numbers is the odd one out and why: [74, 60, 20, 67]?",3,Reasoning |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Oslo'.,4,Agentic |
| Localize this ad copy about the immune system into Russian.,6,Translation |
| Fix this bug in my JavaScript: an off-by-one loop.,2,Coding |
| What is the difference between culinary history and volcanoes?,0,QA |
| What is the difference between the French Revolution and the solar system?,0,QA |
| Write a docstring and type hints for a Python function that processes wildlife conservation.,2,Coding |
| "Based on the full legal document about oceanography, return a structured overview.",9,Long-context |
| What is 34 + 16?,10,Fast/simple |
| Give me the French equivalent of this idiom: 'break the ice'.,6,Translation |
| What is 48 + 42?,10,Fast/simple |
| Compose a haiku about modern architecture.,5,Creative |
| Translate this recipe into Italian.,6,Translation |
| Summarize this article about the moon landing in two sentences.,1,Summary |
| Give me a clear definition of the ocean floor.,0,QA |
| Design a schema for an app that tracks space telescopes.,2,Coding |
| How do you say the human genome in Russian?,6,Translation |
| Write a SQL query to get the top 5 products by revenue.,2,Coding |
| What's the weather like in Paris today?,10,Fast/simple |
| How do you say bees and pollination in Arabic?,6,Translation |
| Write a short executive summary of this report on skyscrapers.,1,Summary |
| "Which of these numbers is the odd one out and why: [75, 53, 36, 44]?",3,Reasoning |
| Read this 500-page transcript and compare the revisions.,9,Long-context |
| "Which of these numbers is the odd one out and why: [19, 1, 58, 35]?",3,Reasoning |
| Explain quantum computing in technical terms.,0,QA |
| Which topic does this paragraph best belong to? the moon landing mentions players.,7,Classification |
| Extract the key entities from this order confirmation about basketball.,8,Extraction |
| Give me a TL;DR of the following passage about medieval castles.,1,Summary |
| Draft a whimsical flash-fiction piece about machine learning.,5,Creative |
| "Provide a brief summary of the customer interview, focusing on volcanoes.",1,Summary |
| Draft a whimsical flash-fiction piece about the industrial revolution.,5,Creative |
| Which topic does this paragraph best belong to? the Silk Road mentions players.,7,Classification |
| "Use the file tool to open notes.txt, then the search tool for the solar system, then combine both.",4,Agentic |
| Invent a fantasy world with magic based on memories.,5,Creative |
| What is 26 + 3?,10,Fast/simple |
| Write a docstring and type hints for a Python function that processes the Silk Road.,2,Coding |
| What is 30 + 5?,10,Fast/simple |
| "Provide a brief summary of the customer interview, focusing on coral reefs.",1,Summary |
| "Pull the name, date, amount out of this medical note about climate change.",8,Extraction |
| "Coordinate a multi-agent system: one agent searches the moon landing, another drafts a plan, and I review the result.",4,Agentic |
| A train departs at 3:00 traveling at 71 mph; how far in 4 hours?,3,Reasoning |
| "Return a JSON object with customer, item, and price parsed from this invoice.",8,Extraction |
| Scan this report and extract every emails and phone numbers.,8,Extraction |
| Write a letter to my future self about artificial intelligence.,5,Creative |
| Return a JSON object with dates and locations parsed from this contract.,8,Extraction |
| "Which of these topics is this text about: deep-sea creatures, space telescopes, oceanography?",7,Classification |
| Write a function that merge two sorted arrays and returns a list.,2,Coding |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: artificial intelligence.",4,Agentic |
| Give me a clear definition of the human brain.,0,QA |
| How do you say skyscrapers in French?,6,Translation |
| Interpret this legal clause into plain Russian.,6,Translation |
| Who is Alan Turing known for?,0,QA |
| "Based on the full transcript about ancient Egypt, summarize the main conclusions.",9,Long-context |
| Determine whether this tweet about oceanography is positive or negative.,7,Classification |
| "Use the file tool to open data.csv, then the search tool for the stock market, then combine both.",4,Agentic |
| How do you say space telescopes in Italian?,6,Translation |
| Optimize this string parser to run faster.,2,Coding |
| "Which of these topics is this text about: skyscrapers, the Roman Empire, the Kalahari desert?",7,Classification |
| How do I implement a priority queue in Python?,2,Coding |
| "Extract named entities (people, places, organizations) from this text about electric vehicles.",8,Extraction |
| Is this statement a fact or an opinion: 'the ocean floor is largely misunderstood.'?,7,Classification |
| Which topic does this paragraph best belong to? volcanoes mentions flights.,7,Classification |
| Translate this recipe into French.,6,Translation |
| "There are 6 boxes, one holds a prize. If I open 5 wrong boxes, how many remain?",3,Reasoning |
| Write a letter to my future self about space telescopes.,5,Creative |
| "Using this long user manual about the industrial revolution, summarize the key provisions.",9,Long-context |
| "Use the file tool to open notes.txt, then the search tool for the Silk Road, then combine both.",4,Agentic |
| Translate 'Where is the train station' into Spanish.,6,Translation |
| Give me the German equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| Write a high-level summary of this report on the immune system.,1,Summary |
| "Given this 10-page user manual about the Kalahari desert, list every deadline.",9,Long-context |
| Determine whether this tweet about photography is positive or negative.,7,Classification |
| Explain renewable energy in beginner-friendly terms.,0,QA |
| Compose a haiku about skyscrapers.,5,Creative |
| "There are 3 boxes, one holds a prize. If I open 4 wrong boxes, how many remain?",3,Reasoning |
| What is the photoelectric effect?,0,QA |
| Extract all dates present in the following insurance policy.,8,Extraction |
| Translate 'Good morning' into Japanese.,6,Translation |
| "Which of these topics is this text about: volcanoes, artificial intelligence, climate change?",7,Classification |
| Explain artificial intelligence in simple terms.,0,QA |
| Condense this chapter into the main takeaways about renewable energy.,1,Summary |
| What is 19 + 23?,10,Fast/simple |
| Write a letter to my future self about wildlife conservation.,5,Creative |
| Summarize this entire year of meeting notes covering traditional cuisine.,9,Long-context |
| Compose a haiku about medieval castles.,5,Creative |
| "Which of these topics is this text about: electric vehicles, the immune system, photography?",7,Classification |
| "Based on the full dataset README about ancient Egypt, explain the compliance requirements.",9,Long-context |
| Pull the key stats and numbers out of this contract about bees and pollination.,8,Extraction |
| Make a concise summary of the key points in this transcript about the ocean floor.,1,Summary |
| Here is the full regulatory filing for modern architecture (120-page). Extract the key clauses.,9,Long-context |
| Localize this ad copy about the jazz era into Spanish.,6,Translation |
| Summarize this entire year of meeting notes covering space telescopes.,9,Long-context |
| "Provide a brief summary of the product review, focusing on skyscrapers.",1,Summary |
| "Coordinate a multi-agent system: one agent searches quantum computing, another drafts a plan, and I review the result.",4,Agentic |
| What are the main causes of climate change?,0,QA |
| Translate this paragraph into Greek: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| "Based on the full dataset README about Mars exploration, list every deadline.",9,Long-context |
| "List every people, places, organizations mentioned in this report on the Great Barrier Reef.",8,Extraction |
| "Call the API to get data on the Silk Road, then summarize the response and save it to disk.",4,Agentic |
| "If the probability of rain is 82%, what is the chance it does not rain?",3,Reasoning |
| Translate this paragraph into Swahili: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Write a song lyric about starting over.,5,Creative |
| Draft a funny flash-fiction piece about ancient Egypt.,5,Creative |
| Create a dialogue between two characters arguing about machine learning.,5,Creative |
| Give me the Arabic equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| "Here is the full report for electric vehicles (1,000-line). Extract the key clauses.",9,Long-context |
| "If 10 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| What's the weather like in Oslo today?,10,Fast/simple |
| Summarize the key findings from this research paper on the immune system.,1,Summary |
| "Which of these topics is this text about: the French Revolution, the ocean floor, the stock market?",7,Classification |
| Localize this ad copy about ancient Egypt into Swahili.,6,Translation |
| Who is Socrates known for?,0,QA |
| "Call the API to get data on climate change, then summarize the response and save it to disk.",4,Agentic |
| What is 47 + 7?,10,Fast/simple |
| "Which of these topics is this text about: the industrial revolution, ancient Egypt, the jazz era?",7,Classification |
| "Pull the customer, item, and price out of this receipt about the industrial revolution.",8,Extraction |
| Give me a clear definition of photography.,0,QA |
| Create a dialogue between two characters arguing about oceanography.,5,Creative |
| Summarize this article about the Great Barrier Reef in three bullet points.,1,Summary |
| Which topic does this paragraph best belong to? photography mentions flights.,7,Classification |
| "Call the API to get data on blockchain, then summarize the response and save it to disk.",4,Agentic |
| "Coordinate a multi-agent system: one agent searches wildlife conservation, another drafts a plan, and I review the result.",4,Agentic |
| "Provide a brief summary of the product review, focusing on deep-sea creatures.",1,Summary |
| What is the difference between Renaissance art and machine learning?,0,QA |
| Here is the full research paper for machine learning (10-page). Summarize it.,9,Long-context |
| Improve the performance of my Go program that normalize a phone number.,2,Coding |
| What is the difference between the French Revolution and traditional cuisine?,0,QA |
| Find and extract legal terms and person names from this text.,8,Extraction |
| "Using the tools available, plan a calendar-scheduling workflow to achieve this goal.",4,Agentic |
| Extract all legal terms and dates from the text about bees and pollination.,8,Extraction |
| "Use the file tool to open config.json, then the search tool for Renaissance art, then combine both.",4,Agentic |
| A train departs at 7:00 traveling at 46 mph; how far in 5 hours?,3,Reasoning |
| "Using the tools available, plan a multi-step research workflow to achieve this goal.",4,Agentic |
| What are the main causes of volcanoes?,0,QA |
| "Call the API to get data on quantum computing, then summarize the response and save it to disk.",4,Agentic |
| "If the probability of rain is 49%, what is the chance it does not rain?",3,Reasoning |
| Localize this ad copy about renewable energy into Greek.,6,Translation |
| "Using the tools available, plan a reporting workflow to achieve this goal.",4,Agentic |
| Review this entire user manual and compare the revisions.,9,Long-context |
| Translate this recipe into Mandarin.,6,Translation |
| Translate these product instructions into Japanese.,6,Translation |
| What is 23 + 35?,10,Fast/simple |
| "Pull the dates, amounts, and identifiers out of this product review about the Great Barrier Reef.",8,Extraction |
| Determine whether this tweet about the human genome is positive or negative.,7,Classification |
| Condense this long email into 2 sentence(s).,1,Summary |
| Convert this pseudocode into working Python code.,2,Coding |
| "Return a JSON object with dates, amounts, and identifiers parsed from this legal document.",8,Extraction |
| What is the capital of Tokyo?,0,QA |
| "Provide a brief summary of the product review, focusing on the human brain.",1,Summary |
| "Use the file tool to open config.json, then the search tool for ancient Greek philosophy, then combine both.",4,Agentic |
| Translate the following business email into Italian.,6,Translation |
| Which topic does this paragraph best belong to? climate change mentions recipes.,7,Classification |
| "Coordinate a multi-agent system: one agent searches the human genome, another drafts a plan, and I review the result.",4,Agentic |
| What is 11 + 25?,10,Fast/simple |
| Summarize this article about the human genome in one line.,1,Summary |
| Give me the French equivalent of this idiom: 'spill the beans'.,6,Translation |
| What is the difference between machine learning and medieval castles?,0,QA |
| Determine whether this tweet about electric vehicles is positive or negative.,7,Classification |
| Extract all ingredients present in the following receipt.,8,Extraction |
| Write a Python function to tokenize a sentence.,2,Coding |
| Translate this recipe into Japanese.,6,Translation |
| Give me a clear definition of the human genome.,0,QA |
| Summarize the key findings from this research paper on wildlife conservation.,1,Summary |
| "Search the web for bees and pollination, then book a flight and send me a summary.",4,Agentic |
| "Given this 500-page legal document about modern architecture, answer the question about cost.",9,Long-context |
| Improve the performance of my Java program that validate an email.,2,Coding |
| Create a dialogue between two characters arguing about Mars exploration.,5,Creative |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the solar system.",4,Agentic |
| Translate 'Good morning' into Korean.,6,Translation |
| What is 3 + 47?,10,Fast/simple |
| What is the difference between volcanoes and Renaissance art?,0,QA |
| Write a letter to my future self about electric vehicles.,5,Creative |
| "If the probability of rain is 34%, what is the chance it does not rain?",3,Reasoning |
| Pull out the product SKU from this meeting minutes.,8,Extraction |
| Give me a TL;DR of the following passage about wildlife conservation.,1,Summary |
| What are the main causes of the stock market?,0,QA |
| What is the difference between ancient Greek philosophy and bees and pollination?,0,QA |
| What is the difference between the industrial revolution and climate change?,0,QA |
| Extract the key figures revenue from this insurance policy.,8,Extraction |
| Is this statement a fact or an opinion: 'the jazz era is largely misunderstood.'?,7,Classification |
| "Which of these topics is this text about: space telescopes, the solar system, skyscrapers?",7,Classification |
| A train departs at 7:00 traveling at 47 mph; how far in 4 hours?,3,Reasoning |
| Refactor this class to be more readable.,2,Coding |
| What is 7 + 4?,10,Fast/simple |
| Localize this ad copy about the Kalahari desert into Dutch.,6,Translation |
| What is 40 + 25?,10,Fast/simple |
| Here is a 10-page research paper. Analyze it and identify contradictions.,9,Long-context |
| Create a dialogue between two characters arguing about Renaissance art.,5,Creative |
| Extract the key figures quantity from this product review.,8,Extraction |
| "Coordinate a multi-agent system: one agent searches skyscrapers, another drafts a plan, and I review the result.",4,Agentic |
| Extract all dates present in the following news article.,8,Extraction |
| What is the difference between the Roman Empire and climate change?,0,QA |
| What is 26 + 45?,10,Fast/simple |
| Explain the human genome in beginner-friendly terms.,0,QA |
| What is 43 + 34?,10,Fast/simple |
| Write a Go script to rename all files.,2,Coding |
| Pull out the product SKU from this legal document.,8,Extraction |
| "Coordinate a multi-agent system: one agent searches the jazz era, another drafts a plan, and I review the result.",4,Agentic |
| What are the main causes of Renaissance art?,0,QA |
| "Extract named entities (people, places, organizations) from this text about bees and pollination.",8,Extraction |
| Which topic does this paragraph best belong to? the immune system mentions algorithms.,7,Classification |
| "Provide a brief summary of the product review, focusing on volcanoes.",1,Summary |
| "Which of these topics is this text about: culinary history, skyscrapers, coral reefs?",7,Classification |
| Fix this bug in my JavaScript: an async race condition.,2,Coding |
| Analyze this full 200-page report on deep-sea creatures and summarize the conclusions.,9,Long-context |
| Here is a 500-page research paper. Analyze it and return a structured overview.,9,Long-context |
| Write a JavaScript function to download and cache a page.,2,Coding |
| What are the main causes of oceanography?,0,QA |
| Give me a TL;DR of the following passage about modern architecture.,1,Summary |
| Translate this recipe into Arabic.,6,Translation |
| "Which of these topics is this text about: bees and pollination, the industrial revolution, the ocean floor?",7,Classification |
| Translate 'How are you today' into Greek.,6,Translation |
| Read this 10-page report and list every deadline.,9,Long-context |
| "Search the web for the industrial revolution, then book a flight and send me a summary.",4,Agentic |
| "List every people, places, organizations mentioned in this product review on the internet of things.",8,Extraction |
| Translate 'Good morning' into Spanish.,6,Translation |
| Write a docstring and type hints for a TypeScript function that processes renewable energy.,2,Coding |
| Create a Dockerfile for a Python project.,2,Coding |
| Which topic does this paragraph best belong to? medieval castles mentions flights.,7,Classification |
| Translate 'Thank you very much' into Dutch.,6,Translation |
| Which topic does this paragraph best belong to? blockchain mentions algorithms.,7,Classification |
| Localize this ad copy about the French Revolution into Greek.,6,Translation |
| Determine whether this tweet about volcanoes is positive or negative.,7,Classification |
| Write a docstring and type hints for a JavaScript function that processes photography.,2,Coding |
| A train departs at 5:00 traveling at 72 mph; how far in 5 hours?,3,Reasoning |
| Extract the key entities from this order confirmation about electric vehicles.,8,Extraction |
| Create a Makefile target for a Node project.,2,Coding |
| Scan this insurance policy and extract every organizations.,8,Extraction |
| Give me a clear definition of the stock market.,0,QA |
| Which topic does this paragraph best belong to? ancient Egypt mentions budgets.,7,Classification |
| "Given this 120-page research paper about skyscrapers, extract the recommendations.",9,Long-context |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: machine learning.",4,Agentic |
| Who is Nikola Tesla known for?,0,QA |
| "Read the whole 1,000-line report which covers machine learning and return a structured overview.",9,Long-context |
| "Return a JSON object with people, places, organizations parsed from this product review.",8,Extraction |
| Analyze this full 200-page report on the stock market and summarize the conclusions.,9,Long-context |
| What is the difference between renewable energy and deep-sea creatures?,0,QA |
| Give me a clear definition of the Kalahari desert.,0,QA |
| "Call the API to get data on the human genome, then summarize the response and save it to disk.",4,Agentic |
| Explain the time complexity of this algorithm that compute the Fibonacci sequence.,2,Coding |
| Translate this recipe into German.,6,Translation |
| Break down this business goal into an agentic plan with tool calls for each step.,4,Agentic |
| "Use the file tool to open notes.txt, then the search tool for the immune system, then combine both.",4,Agentic |
| "Which of these topics is this text about: deep-sea creatures, volcanoes, renewable energy?",7,Classification |
| What is 44 + 50?,10,Fast/simple |
| What is the capital of Toronto?,0,QA |
| "Search the web for the Roman Empire, then book a hotel and send me a summary.",4,Agentic |
| Localize this ad copy about the Great Barrier Reef into French.,6,Translation |
| What are the main causes of the French Revolution?,0,QA |
| "Which of these numbers is the odd one out and why: [4, 51, 46, 13]?",3,Reasoning |
| Draft a funny flash-fiction piece about deep-sea creatures.,5,Creative |
| Process this 500-page legal document on culinary history; then answer the question about cost.,9,Long-context |
| Compose a haiku about space telescopes.,5,Creative |
| Pull out the product SKU from this invoice.,8,Extraction |
| Write a one-paragraph summary of this report on the Silk Road.,1,Summary |
| Give me a clear definition of Renaissance art.,0,QA |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the Roman Empire.",4,Agentic |
| How do you say Mars exploration in Swahili?,6,Translation |
| "Call the API to get data on electric vehicles, then summarize the response and save it to disk.",4,Agentic |
| Write a letter to my future self about deep-sea creatures.,5,Creative |
| Here is the full research paper for traditional cuisine (25-page). Summarize it.,9,Long-context |
| Write a JavaScript script to scrape a webpage.,2,Coding |
| Write a Python function to download and cache a page.,2,Coding |
| Write a Java function to compute the Fibonacci sequence.,2,Coding |
| What is 37 + 40?,10,Fast/simple |
| "There are 6 boxes, one holds a prize. If I open 3 wrong boxes, how many remain?",3,Reasoning |
| Translate these product instructions into German.,6,Translation |
| "If 7 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| Analyze this full 200-page report on climate change and summarize the conclusions.,9,Long-context |
| Translate 'Where is the train station' into Mandarin.,6,Translation |
| Write a Rust script to batch-resize images.,2,Coding |
| Read the whole 300-page contract which covers Mars exploration and summarize the main conclusions.,9,Long-context |
| What is 6 + 45?,10,Fast/simple |
| Return a JSON object with key stats and numbers parsed from this report.,8,Extraction |
| Which topic does this paragraph best belong to? oceanography mentions budgets.,7,Classification |
| Write a docstring and type hints for a Java function that processes the internet of things.,2,Coding |
| "Which of these topics is this text about: the French Revolution, blockchain, the Kalahari desert?",7,Classification |
| What is 28 + 24?,10,Fast/simple |
| Write a SQL query to get the top 4 customers by revenue.,2,Coding |
| Explain the Roman Empire in simple terms.,0,QA |
| Summarize this entire year of meeting notes covering wildlife conservation.,9,Long-context |
| "Which of these topics is this text about: deep-sea creatures, the immune system, the Silk Road?",7,Classification |
| "Provide a brief summary of the meeting notes, focusing on culinary history.",1,Summary |
| Create a dialogue between two characters arguing about ancient Greek philosophy.,5,Creative |
| Is this statement a fact or an opinion: 'quantum computing is largely misunderstood.'?,7,Classification |
| Determine whether this tweet about ancient Greek philosophy is positive or negative.,7,Classification |
| Is this statement a fact or an opinion: 'culinary history is largely misunderstood.'?,7,Classification |
| Fix this bug in my JavaScript: NaN comparison.,2,Coding |
| Condense this chapter into the main takeaways about medieval castles.,1,Summary |
| "There are 4 boxes, one holds a prize. If I open 4 wrong boxes, how many remain?",3,Reasoning |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the human genome.",4,Agentic |
| Determine whether this tweet about the ocean floor is positive or negative.,7,Classification |
| Convert this English text to Russian and keep the tone formal.,6,Translation |
| What is 13 + 15?,10,Fast/simple |
| What are the main causes of medieval castles?,0,QA |
| Explain the time complexity of this algorithm that flatten a nested list.,2,Coding |
| Refactor this component to be more readable.,2,Coding |
| Write a SQL query to get the top 10 orders by revenue.,2,Coding |
| Draft a whimsical flash-fiction piece about the Silk Road.,5,Creative |
| Write a letter to my future self about bees and pollination.,5,Creative |
| How do I implement a decision tree in Python?,2,Coding |
| Write a short story about a time-traveling robot.,5,Creative |
| A train departs at 11:00 traveling at 86 mph; how far in 6 hours?,3,Reasoning |
| Localize this ad copy about volcanoes into Swahili.,6,Translation |
| Here is a 300-page contract. Analyze it and compare the revisions.,9,Long-context |
| Render this quote into Swahili and explain any nuances.,6,Translation |
| Summarize this article about deep-sea creatures in two sentences.,1,Summary |
| Draft a whimsical flash-fiction piece about the human genome.,5,Creative |
| "There are 5 boxes, one holds a prize. If I open 6 wrong boxes, how many remain?",3,Reasoning |
| "Using this long thesis about the jazz era, extract the recommendations.",9,Long-context |
| Is this statement a fact or an opinion: 'the Silk Road is largely misunderstood.'?,7,Classification |
| What are the main causes of traditional cuisine?,0,QA |
| Draft a funny flash-fiction piece about traditional cuisine.,5,Creative |
| Mary buys 3 apples at $2 each and a loaf for $4. Total cost?,3,Reasoning |
| Write a JavaScript function to validate an email.,2,Coding |
| Briefly summarize the plot of this story about modern architecture.,1,Summary |
| Which topic does this paragraph best belong to? Mars exploration mentions recipes.,7,Classification |
| Write a letter to my future self about traditional cuisine.,5,Creative |
| Determine whether this tweet about deep-sea creatures is positive or negative.,7,Classification |
| Write a Go function to tokenize a sentence.,2,Coding |
| How do you say space telescopes in Mandarin?,6,Translation |
| Explain space telescopes in simple terms.,0,QA |
| "Search the web for the solar system, then book a hotel and send me a summary.",4,Agentic |
| Scan this medical note and extract every legal terms.,8,Extraction |
| "There are 6 boxes, one holds a prize. If I open 1 wrong boxes, how many remain?",3,Reasoning |
| What is 22 + 40?,10,Fast/simple |
| Read the whole 10-page codebase README which covers the human genome and summarize the key provisions.,9,Long-context |
| Here is a 50-page user manual. Analyze it and give the setup steps.,9,Long-context |
| Give me a clear definition of culinary history.,0,QA |
| Write a poem about the ocean and happiness.,5,Creative |
| "Search the web for the human genome, then book a flight and send me a summary.",4,Agentic |
| Explain the Great Barrier Reef in technical terms.,0,QA |
| What is 19 + 9?,10,Fast/simple |
| Make a concise summary of the key points in this transcript about the moon landing.,1,Summary |
| Write a function that reverse a string and returns a list.,2,Coding |
| Use your registry of tools to update a database row and report back in a short message.,4,Agentic |
| "Search the web for the human brain, then book a flight and send me a summary.",4,Agentic |
| Write a function that parse a CSV and returns a list.,2,Coding |
| Extract the steps and ingredients from this recipe for Renaissance art.,8,Extraction |
| What is the difference between oceanography and the Silk Road?,0,QA |
| "Which of these numbers is the odd one out and why: [6, 27, 38, 72]?",3,Reasoning |
| Translate these product instructions into Italian.,6,Translation |
| What is the difference between the Great Barrier Reef and skyscrapers?,0,QA |
| Find the memory leak in this Node app.,2,Coding |
| Design a schema for an app that tracks wildlife conservation.,2,Coding |
| What is 49 + 49?,10,Fast/simple |
| Compose a haiku about the internet of things.,5,Creative |
| "Which of these numbers is the odd one out and why: [25, 52, 63, 38]?",3,Reasoning |
| What is the difference between bees and pollination and wildlife conservation?,0,QA |
| Translate 'Good morning' into Mandarin.,6,Translation |
| Which topic does this paragraph best belong to? the human brain mentions flights.,7,Classification |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the stock market.",4,Agentic |
| Summarize the key findings from this research paper on the Roman Empire.,1,Summary |
| Translate the following business email into Korean.,6,Translation |
| What is 34 + 35?,10,Fast/simple |
| "Which of these topics is this text about: quantum computing, the Roman Empire, renewable energy?",7,Classification |
| What time is it in Reykjavik right now?,10,Fast/simple |
| Condense this chapter into the main takeaways about quantum computing.,1,Summary |
| Extract the main keyword phrases and topics from this passage on ancient Egypt.,8,Extraction |
| Translate 'I would like a coffee' into Greek.,6,Translation |
| "Which of these numbers is the odd one out and why: [60, 17, 49, 62]?",3,Reasoning |
| Which topic does this paragraph best belong to? the Kalahari desert mentions players.,7,Classification |
| Write a Python function to reverse a string.,2,Coding |
| Scan this product review and extract every emails and phone numbers.,8,Extraction |
| Draft a melancholic flash-fiction piece about traditional cuisine.,5,Creative |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: photography.",4,Agentic |
| Summarize this article about machine learning in two sentences.,1,Summary |
| Use your registry of tools to deploy a model and report back in a CSV.,4,Agentic |
| "Based on the full research paper about the Roman Empire, return a structured overview.",9,Long-context |
| Fix this bug in my JavaScript: a closure leak.,2,Coding |
| Render this quote into Spanish and explain any nuances.,6,Translation |
| "Provide a brief summary of the meeting notes, focusing on basketball.",1,Summary |
| Summarize the key findings from this research paper on the Great Barrier Reef.,1,Summary |
| "Which of these topics is this text about: skyscrapers, ancient Greek philosophy, medieval castles?",7,Classification |
| Determine whether this tweet about the jazz era is positive or negative.,7,Classification |
| A train departs at 8:00 traveling at 43 mph; how far in 6 hours?,3,Reasoning |
| How do you say volcanoes in Mandarin?,6,Translation |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the French Revolution.",4,Agentic |
| "Search the web for Renaissance art, then book a dinner and send me a summary.",4,Agentic |
| List every named entities and keywords mentioned in this news article on skyscrapers.,8,Extraction |
| Translate the following business email into Swahili.,6,Translation |
| Write a letter to my future self about the Great Barrier Reef.,5,Creative |
| Write a Go function to sort a list of dicts.,2,Coding |
| Find and extract prices and geographic locations from this text.,8,Extraction |
| What is 3 + 31?,10,Fast/simple |
| Create a dialogue between two characters arguing about the human genome.,5,Creative |
| What is 48 + 16?,10,Fast/simple |
| "If the probability of rain is 83%, what is the chance it does not rain?",3,Reasoning |
| Interpret this legal clause into plain Swahili.,6,Translation |
| Localize this ad copy about culinary history into Italian.,6,Translation |
| Fix this bug in my JavaScript: undefined is not a function.,2,Coding |
| "Pull the id, total, and taxes out of this news article about the immune system.",8,Extraction |
| Find and extract key figures and key figures from this text.,8,Extraction |
| What's the weather like in Reykjavik today?,10,Fast/simple |
| Extract the steps and ingredients from this recipe for deep-sea creatures.,8,Extraction |
| Which topic does this paragraph best belong to? the Roman Empire mentions budgets.,7,Classification |
| "Search the web for the Roman Empire, then book a dinner and send me a summary.",4,Agentic |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: ancient Egypt.",4,Agentic |
| Write a TypeScript function to normalize a phone number.,2,Coding |
| Compose a haiku about the human brain.,5,Creative |
| "Which of these topics is this text about: climate change, photography, renewable energy?",7,Classification |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the internet of things.",4,Agentic |
| Read these 15 documents about basketball and synthesize a single answer.,9,Long-context |
| Read the whole 500-page user manual which covers wildlife conservation and give the setup steps.,9,Long-context |
| What are the main causes of culinary history?,0,QA |
| What is 36 + 28?,10,Fast/simple |
| Summarize this entire year of meeting notes covering medieval castles.,9,Long-context |
| Draft a melancholic flash-fiction piece about bees and pollination.,5,Creative |
| Build a a cron job with proper error handling.,2,Coding |
| Mary buys 4 apples at $3 each and a loaf for $2. Total cost?,3,Reasoning |
| "Pull the dates, amounts, and identifiers out of this report about basketball.",8,Extraction |
| Interpret this legal clause into plain Greek.,6,Translation |
| Why is ice less dense than water?,0,QA |
| Draft a melancholic flash-fiction piece about skyscrapers.,5,Creative |
| Extract all product names present in the following product review.,8,Extraction |
| What is 17 + 27?,10,Fast/simple |
| How do you say the solar system in German?,6,Translation |
| What is 8 + 47?,10,Fast/simple |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: climate change.",4,Agentic |
| Translate this paragraph into Dutch: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| A train departs at 4:00 traveling at 48 mph; how far in 2 hours?,3,Reasoning |
| "Call the API to get data on the industrial revolution, then summarize the response and save it to disk.",4,Agentic |
| Briefly summarize the plot of this story about culinary history.,1,Summary |
| Give me a one-word answer: stop.,10,Fast/simple |
| Extract the key entities from this report about renewable energy.,8,Extraction |
| Briefly summarize the plot of this story about skyscrapers.,1,Summary |
| Extract the key figures timestamp from this contract.,8,Extraction |
| Make a concise summary of the key points in this transcript about quantum computing.,1,Summary |
| Pull out the due date from this news article.,8,Extraction |
| "Use the file tool to open config.json, then the search tool for electric vehicles, then combine both.",4,Agentic |
| Review this entire regulatory filing and explain the compliance requirements.,9,Long-context |
| Write a Go function to validate an email.,2,Coding |
| "Which of these numbers is the odd one out and why: [6, 26, 47, 73]?",3,Reasoning |
| Interpret this legal clause into plain Hindi.,6,Translation |
| Extract all prices and ingredients from the text about coral reefs.,8,Extraction |
| "Extract the dates, locations, and attendees from this event invitation.",8,Extraction |
| "Which of these topics is this text about: the Kalahari desert, culinary history, electric vehicles?",7,Classification |
| "If the probability of rain is 86%, what is the chance it does not rain?",3,Reasoning |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on electric vehicles.",4,Agentic |
| Summarize this entire year of meeting notes covering blockchain.,9,Long-context |
| "Use the file tool to open notes.txt, then the search tool for the stock market, then combine both.",4,Agentic |
| "Coordinate a multi-agent system: one agent searches ancient Greek philosophy, another drafts a plan, and I review the result.",4,Agentic |
| What are the main causes of electric vehicles?,0,QA |
| Write a Go function to compute the Fibonacci sequence.,2,Coding |
| "Using this long legal document about the immune system, summarize the main conclusions.",9,Long-context |
| Is this statement a fact or an opinion: 'the human brain is largely misunderstood.'?,7,Classification |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Toronto.",4,Agentic |
| "Provide a brief summary of the customer interview, focusing on climate change.",1,Summary |
| Read the whole 25-page regulatory filing which covers traditional cuisine and summarize the key provisions.,9,Long-context |
| Determine whether this tweet about the Kalahari desert is positive or negative.,7,Classification |
| "Which of these topics is this text about: traditional cuisine, oceanography, basketball?",7,Classification |
| Write a letter to my future self about quantum computing.,5,Creative |
| Give me a TL;DR of the following passage about the Kalahari desert.,1,Summary |
| Give me a 3-bullet recap of medieval castles.,1,Summary |
| "Use the file tool to open notes.txt, then the search tool for ancient Egypt, then combine both.",4,Agentic |
| "What comes next in the sequence: 2, 6, 12, 20, 30?",3,Reasoning |
| What is the difference between the internet of things and culinary history?,0,QA |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Paris.",4,Agentic |
| Read this 10-page thesis and answer the question about cost.,9,Long-context |
| Give me a 3-bullet recap of blockchain.,1,Summary |
| Extract all dates and geographic locations from the text about volcanoes.,8,Extraction |
| Summarize this lengthy document in under 80 words.,1,Summary |
| Write a Java script to rename all files.,2,Coding |
| Give me the Portuguese equivalent of this idiom: 'break the ice'.,6,Translation |
| Review this entire transcript and summarize the key provisions.,9,Long-context |
| Extract all prices and key figures from the text about the human brain.,8,Extraction |
| Mary buys 4 apples at $1 each and a loaf for $4. Total cost?,3,Reasoning |
| Translate this recipe into Portuguese.,6,Translation |
| What is the difference between climate change and electric vehicles?,0,QA |
| A train departs at 7:00 traveling at 79 mph; how far in 6 hours?,3,Reasoning |
| Explain the time complexity of this algorithm that reverse a string.,2,Coding |
| Which topic does this paragraph best belong to? the immune system mentions budgets.,7,Classification |
| Who is Katherine Johnson known for?,0,QA |
| Write a JavaScript script to back up a database.,2,Coding |
| Write a high-level summary of this report on blockchain.,1,Summary |
| Summarize the key findings from this research paper on the stock market.,1,Summary |
| What is 43 + 17?,10,Fast/simple |
| Convert this English text to French and keep the tone formal.,6,Translation |
| Write a docstring and type hints for a TypeScript function that processes machine learning.,2,Coding |
| "Search the web for the internet of things, then book a dinner and send me a summary.",4,Agentic |
| How do I implement a hash map in Python?,2,Coding |
| Compose a haiku about wildlife conservation.,5,Creative |
| Pull out the total from this legal document.,8,Extraction |
| "Which of these topics is this text about: quantum computing, blockchain, modern architecture?",7,Classification |
| Read these 15 documents about artificial intelligence and synthesize a single answer.,9,Long-context |
| Write a function that reverse a string and returns a dict.,2,Coding |
| Extract the key entities from this invoice about deep-sea creatures.,8,Extraction |
| Convert this English text to Russian and keep the tone casual.,6,Translation |
| Determine whether this tweet about medieval castles is positive or negative.,7,Classification |
| Pull the dates and locations out of this product review about climate change.,8,Extraction |
| Determine whether this tweet about coral reefs is positive or negative.,7,Classification |
| Convert this English text to Hindi and keep the tone neutral.,6,Translation |
| Design a schema for an app that tracks blockchain.,2,Coding |
| "Which of these topics is this text about: basketball, quantum computing, electric vehicles?",7,Classification |
| "If the probability of rain is 61%, what is the chance it does not rain?",3,Reasoning |
| Create a Dockerfile for a Node project.,2,Coding |
| Compose a haiku about ancient Egypt.,5,Creative |
| What is the difference between the ocean floor and the Silk Road?,0,QA |
| Review this entire transcript and extract the recommendations.,9,Long-context |
| Make a concise summary of the key points in this transcript about blockchain.,1,Summary |
| Condense this chapter into the main takeaways about machine learning.,1,Summary |
| Which topic does this paragraph best belong to? medieval castles mentions recipes.,7,Classification |
| "Use the file tool to open notes.txt, then the search tool for wildlife conservation, then combine both.",4,Agentic |
| Explain the human genome in plain terms.,0,QA |
| What is 45 + 2?,10,Fast/simple |
| "There are 8 boxes, one holds a prize. If I open 6 wrong boxes, how many remain?",3,Reasoning |
| Write a one-paragraph summary of this report on medieval castles.,1,Summary |
| Which topic does this paragraph best belong to? deep-sea creatures mentions recipes.,7,Classification |
| "Provide a brief summary of the meeting notes, focusing on electric vehicles.",1,Summary |
| What is 3 + 30?,10,Fast/simple |
| Translate 'Good morning' into Greek.,6,Translation |
| A train departs at 9:00 traveling at 67 mph; how far in 3 hours?,3,Reasoning |
| Write a product description for a spatial keyboard.,5,Creative |
| What is 17 + 23?,10,Fast/simple |
| Build a HTML and CSS with proper error handling.,2,Coding |
| "Which of these topics is this text about: machine learning, traditional cuisine, Renaissance art?",7,Classification |
| Scan this order confirmation and extract every dates.,8,Extraction |
| "Search the web for ancient Egypt, then book a hotel and send me a summary.",4,Agentic |
| "Which of these numbers is the odd one out and why: [16, 34, 21, 10]?",3,Reasoning |
| What is 3 + 36?,10,Fast/simple |
| Translate this paragraph into Italian: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| What is 11 + 33?,10,Fast/simple |
| What is 34 + 19?,10,Fast/simple |
| Write a song lyric about summer nights.,5,Creative |
| Condense this long email into 3 sentence(s).,1,Summary |
| Compose a haiku about climate change.,5,Creative |
| Write a letter to my future self about skyscrapers.,5,Creative |
| What is 42 + 5?,10,Fast/simple |
| What is 23 + 24?,10,Fast/simple |
| "If the probability of rain is 25%, what is the chance it does not rain?",3,Reasoning |
| What is the difference between Mars exploration and the industrial revolution?,0,QA |
| Extract all prices present in the following meeting minutes.,8,Extraction |
| What is 19 + 38?,10,Fast/simple |
| Add error handling and logging to this data loader.,2,Coding |
| Condense this chapter into the main takeaways about basketball.,1,Summary |
| Scan this contract and extract every geographic locations.,8,Extraction |
| "If 4 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| "Search the web for the jazz era, then book a dinner and send me a summary.",4,Agentic |
| Create a CI workflow for a Rust project.,2,Coding |
| "Pull the people, places, organizations out of this insurance policy about renewable energy.",8,Extraction |
| "Pull the id, total, and taxes out of this contract about the ocean floor.",8,Extraction |
| Mary buys 5 apples at $1 each and a loaf for $5. Total cost?,3,Reasoning |
| "There are 3 boxes, one holds a prize. If I open 6 wrong boxes, how many remain?",3,Reasoning |
| Summarize this entire year of meeting notes covering ancient Egypt.,9,Long-context |
| Write a function that sort a list of dicts and returns a generator.,2,Coding |
| Give me an executive recap of the industrial revolution.,1,Summary |
| Determine whether this tweet about skyscrapers is positive or negative.,7,Classification |
| How do you say ancient Egypt in Greek?,6,Translation |
| Give me an executive recap of the ocean floor.,1,Summary |
| What is 46 + 9?,10,Fast/simple |
| What is 20 + 14?,10,Fast/simple |
| A train departs at 8:00 traveling at 65 mph; how far in 4 hours?,3,Reasoning |
| Write a docstring and type hints for a JavaScript function that processes basketball.,2,Coding |
| Invent a fantasy world with magic based on time.,5,Creative |
| "Which of these topics is this text about: the Silk Road, renewable energy, the stock market?",7,Classification |
| "If the probability of rain is 89%, what is the chance it does not rain?",3,Reasoning |
| Explain the French Revolution in beginner-friendly terms.,0,QA |
| Read this 500-page user manual and return a structured overview.,9,Long-context |
| "Return a JSON object with name, date, amount parsed from this report.",8,Extraction |
| Write a one-paragraph summary of this report on the immune system.,1,Summary |
| Mary buys 3 apples at $3 each and a loaf for $5. Total cost?,3,Reasoning |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on climate change.",4,Agentic |
| Draft a melancholic flash-fiction piece about modern architecture.,5,Creative |
| Explain bees and pollination in simple terms.,0,QA |
| Extract all prices present in the following invoice.,8,Extraction |
| "Provide a brief summary of the customer interview, focusing on deep-sea creatures.",1,Summary |
| Write a SQL query to get the top 4 orders by revenue.,2,Coding |
| "Pull the people, places, organizations out of this news article about artificial intelligence.",8,Extraction |
| What is 49 + 40?,10,Fast/simple |
| Give me a TL;DR of the following passage about electric vehicles.,1,Summary |
| Read these 15 documents about the ocean floor and synthesize a single answer.,9,Long-context |
| What is 47 + 30?,10,Fast/simple |
| What is 27 + 26?,10,Fast/simple |
| Write a high-level summary of this report on traditional cuisine.,1,Summary |
| Draft a funny flash-fiction piece about blockchain.,5,Creative |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the Kalahari desert.",4,Agentic |
| Determine whether this tweet about Mars exploration is positive or negative.,7,Classification |
| Pull the dates and locations out of this contract about blockchain.,8,Extraction |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on medieval castles.",4,Agentic |
| A train departs at 10:00 traveling at 87 mph; how far in 2 hours?,3,Reasoning |
| What is 50 + 15?,10,Fast/simple |
| Here is a 10-page user manual. Analyze it and answer the question about cost.,9,Long-context |
| "Pull the dates, amounts, and identifiers out of this legal document about the moon landing.",8,Extraction |
| Localize this ad copy about the jazz era into Portuguese.,6,Translation |
| Make a concise summary of the key points in this transcript about artificial intelligence.,1,Summary |
| Compose a haiku about the solar system.,5,Creative |
| Translate 'How are you today' into German.,6,Translation |
| What is 5 + 3?,10,Fast/simple |
| "Which of these topics is this text about: the Roman Empire, the Great Barrier Reef, traditional cuisine?",7,Classification |
| "Which of these topics is this text about: deep-sea creatures, the Kalahari desert, coral reefs?",7,Classification |
| What are the main causes of wildlife conservation?,0,QA |
| What is 30 + 23?,10,Fast/simple |
| Invent a fantasy world with magic based on dreams.,5,Creative |
| Review this entire report and answer the question about cost.,9,Long-context |
| Which topic does this paragraph best belong to? the solar system mentions algorithms.,7,Classification |
| Explain blockchain in plain terms.,0,QA |
| Write a product description for a smart water bottle.,5,Creative |
| "Which of these topics is this text about: medieval castles, blockchain, quantum computing?",7,Classification |
| Determine whether this tweet about the human brain is positive or negative.,7,Classification |
| Extract all prices and dates from the text about ancient Greek philosophy.,8,Extraction |
| What is 6 + 50?,10,Fast/simple |
| "Given this 10-page report about space telescopes, identify contradictions.",9,Long-context |
| Find and extract ingredients and emails and phone numbers from this text.,8,Extraction |
| What is 48 + 19?,10,Fast/simple |
| Search this whole corpus and answer: what changed between the versions?,9,Long-context |
| Pull out the tracking number from this insurance policy.,8,Extraction |
| Mary buys 5 apples at $1 each and a loaf for $3. Total cost?,3,Reasoning |
| Summarize this 50-page document in under 100 words.,1,Summary |
| "Which of these numbers is the odd one out and why: [36, 74, 42, 34]?",3,Reasoning |
| Is this statement a fact or an opinion: 'bees and pollination is largely misunderstood.'?,7,Classification |
| Mary buys 3 apples at $2 each and a loaf for $2. Total cost?,3,Reasoning |
| Summarize this entire year of meeting notes covering the human genome.,9,Long-context |
| Compose a haiku about coral reefs.,5,Creative |
| Pull the named entities and keywords out of this product review about the French Revolution.,8,Extraction |
| Write a SQL query to get the top 2 products by revenue.,2,Coding |
| Explain basketball in simple terms.,0,QA |
| Write a high-level summary of this report on the human genome.,1,Summary |
| Localize this ad copy about the human brain into Arabic.,6,Translation |
| Draft a whimsical flash-fiction piece about medieval castles.,5,Creative |
| Translate this paragraph into French: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Condense this chapter into the main takeaways about the Roman Empire.,1,Summary |
| A train departs at 8:00 traveling at 66 mph; how far in 6 hours?,3,Reasoning |
| Write a letter to my future self about the human genome.,5,Creative |
| "Pull the name, date, amount out of this medical note about space telescopes.",8,Extraction |
| "If the probability of rain is 59%, what is the chance it does not rain?",3,Reasoning |
| "Which of these topics is this text about: the stock market, traditional cuisine, deep-sea creatures?",7,Classification |
| Draft a melancholic flash-fiction piece about coral reefs.,5,Creative |
| "Search the web for modern architecture, then book a hotel and send me a summary.",4,Agentic |
| "Which of these numbers is the odd one out and why: [26, 53, 3, 36]?",3,Reasoning |
| A father is twice his daughter's age; in 4 years he'll be 1.5x. Find their ages.,3,Reasoning |
| What is 49 + 6?,10,Fast/simple |
| Which topic does this paragraph best belong to? the French Revolution mentions recipes.,7,Classification |
| Draft a melancholic flash-fiction piece about Renaissance art.,5,Creative |
| Give me a one-sentence recap of blockchain.,1,Summary |
| Is this statement a fact or an opinion: 'climate change is largely misunderstood.'?,7,Classification |
| Mary buys 5 apples at $3 each and a loaf for $3. Total cost?,3,Reasoning |
| Create a dialogue between two characters arguing about the industrial revolution.,5,Creative |
| "Which of these numbers is the odd one out and why: [44, 31, 74, 12]?",3,Reasoning |
| Draft a melancholic flash-fiction piece about basketball.,5,Creative |
| Determine whether this tweet about climate change is positive or negative.,7,Classification |
| Give me the Greek equivalent of this idiom: 'hit the nail on the head'.,6,Translation |
| Summarize this article about Renaissance art in two sentences.,1,Summary |
| Draft a whimsical flash-fiction piece about wildlife conservation.,5,Creative |
| Write a docstring and type hints for a TypeScript function that processes quantum computing.,2,Coding |
| What is the difference between Renaissance art and traditional cuisine?,0,QA |
| What is 34 + 32?,10,Fast/simple |
| Condense this chapter into the main takeaways about the industrial revolution.,1,Summary |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the stock market.",4,Agentic |
| Write a TypeScript script to back up a database.,2,Coding |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the jazz era.",4,Agentic |
| "Search the web for the human brain, then book a dinner and send me a summary.",4,Agentic |
| What is 8 + 43?,10,Fast/simple |
| Which topic does this paragraph best belong to? the internet of things mentions players.,7,Classification |
| Improve the performance of my Python program that validate an email.,2,Coding |
| "Search the web for the internet of things, then book a flight and send me a summary.",4,Agentic |
| What is 13 + 7?,10,Fast/simple |
| Which topic does this paragraph best belong to? the Silk Road mentions recipes.,7,Classification |
| "Using this long transcript about artificial intelligence, explain the compliance requirements.",9,Long-context |
| Here is the full regulatory filing for machine learning (500-page). Summarize it.,9,Long-context |
| "If the probability of rain is 73%, what is the chance it does not rain?",3,Reasoning |
| Create a dialogue between two characters arguing about medieval castles.,5,Creative |
| A train departs at 7:00 traveling at 69 mph; how far in 5 hours?,3,Reasoning |
| How do you say electric vehicles in Korean?,6,Translation |
| Explain the time complexity of this algorithm that normalize a phone number.,2,Coding |
| What is 29 + 49?,10,Fast/simple |
| Is this statement a fact or an opinion: 'the Kalahari desert is largely misunderstood.'?,7,Classification |
| "Pull the people, places, organizations out of this product review about oceanography.",8,Extraction |
| Fix this bug in my JavaScript: a stale state update.,2,Coding |
| Give me a clear definition of blockchain.,0,QA |
| Give me an executive recap of the immune system.,1,Summary |
| Explain this docker-compose file to me.,2,Coding |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on culinary history.",4,Agentic |
| Briefly summarize the plot of this story about the ocean floor.,1,Summary |
| "Search the web for space telescopes, then book a flight and send me a summary.",4,Agentic |
| "Which of these topics is this text about: skyscrapers, the Kalahari desert, coral reefs?",7,Classification |
| Write a Java function to restructure a nested JSON.,2,Coding |
| A train departs at 11:00 traveling at 75 mph; how far in 5 hours?,3,Reasoning |
| "There are 9 boxes, one holds a prize. If I open 3 wrong boxes, how many remain?",3,Reasoning |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Reykjavik'.,4,Agentic |
| "Return a JSON object with dates, amounts, and identifiers parsed from this insurance policy.",8,Extraction |
| Create a dialogue between two characters arguing about quantum computing.,5,Creative |
| What is 37 + 38?,10,Fast/simple |
| Analyze this full 200-page report on electric vehicles and summarize the conclusions.,9,Long-context |
| Determine whether this tweet about the industrial revolution is positive or negative.,7,Classification |
| When was the internet invented?,0,QA |
| What is 6 + 29?,10,Fast/simple |
| Compose a haiku about renewable energy.,5,Creative |
| Write a letter to my future self about Mars exploration.,5,Creative |
| Explain the stock market in beginner-friendly terms.,0,QA |
| "Which of these numbers is the odd one out and why: [47, 62, 18, 67]?",3,Reasoning |
| Draft a melancholic flash-fiction piece about the moon landing.,5,Creative |
| "Call the API to get data on culinary history, then summarize the response and save it to disk.",4,Agentic |
| A train departs at 4:00 traveling at 53 mph; how far in 4 hours?,3,Reasoning |
| "There are 8 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| What is 41 + 49?,10,Fast/simple |
| Process this 50-page transcript on the moon landing; then list every deadline.,9,Long-context |
| What is 42 + 36?,10,Fast/simple |
| Write a Java function to download and cache a page.,2,Coding |
| Write a high-level summary of this report on the industrial revolution.,1,Summary |
| Draft a whimsical flash-fiction piece about skyscrapers.,5,Creative |
| Localize this ad copy about bees and pollination into Greek.,6,Translation |
| Improve the performance of my Python program that merge two sorted arrays.,2,Coding |
| "List every name, date, amount mentioned in this legal document on artificial intelligence.",8,Extraction |
| "Which of these numbers is the odd one out and why: [55, 63, 66, 28]?",3,Reasoning |
| Give me a one-word answer: go.,10,Fast/simple |
| Write a TypeScript function to sort a list of dicts.,2,Coding |
| Explain Renaissance art in simple terms.,0,QA |
| Write a short story about a time-traveling poet.,5,Creative |
| "Which of these numbers is the odd one out and why: [39, 29, 11, 40]?",3,Reasoning |
| Write a high-level summary of this report on ancient Egypt.,1,Summary |
| "Which of these numbers is the odd one out and why: [43, 45, 25, 78]?",3,Reasoning |
| "If the probability of rain is 87%, what is the chance it does not rain?",3,Reasoning |
| Which topic does this paragraph best belong to? deep-sea creatures mentions flights.,7,Classification |
| Condense this chapter into the main takeaways about Renaissance art.,1,Summary |
| "If the probability of rain is 43%, what is the chance it does not rain?",3,Reasoning |
| Interpret this legal clause into plain German.,6,Translation |
| Mary buys 4 apples at $3 each and a loaf for $5. Total cost?,3,Reasoning |
| Design a schema for an app that tracks quantum computing.,2,Coding |
| Translate this recipe into Greek.,6,Translation |
| What is the difference between the Great Barrier Reef and wildlife conservation?,0,QA |
| Which topic does this paragraph best belong to? artificial intelligence mentions players.,7,Classification |
| What is 33 + 5?,10,Fast/simple |
| Extract all geographic locations and person names from the text about artificial intelligence.,8,Extraction |
| Translate the following business email into Mandarin.,6,Translation |
| What is the difference between the industrial revolution and the French Revolution?,0,QA |
| "Classify this email as spam, promotion, or important: 'You have won a free iPhone!'",7,Classification |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the Roman Empire.",4,Agentic |
| "If the probability of rain is 20%, what is the chance it does not rain?",3,Reasoning |
| Pull out the due date from this medical note.,8,Extraction |
| Create a dialogue between two characters arguing about artificial intelligence.,5,Creative |
| A train departs at 10:00 traveling at 49 mph; how far in 4 hours?,3,Reasoning |
| Write a high-level summary of this report on basketball.,1,Summary |
| Write a Java function to flatten a nested list.,2,Coding |
| What is 19 + 42?,10,Fast/simple |
| Write a short story about a time-traveling chef.,5,Creative |
| Explain wildlife conservation in technical terms.,0,QA |
| Determine whether this tweet about ancient Egypt is positive or negative.,7,Classification |
| "If the probability of rain is 38%, what is the chance it does not rain?",3,Reasoning |
| "If 8 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| Summarize this article about the immune system in two sentences.,1,Summary |
| Extract the steps and ingredients from this recipe for the jazz era.,8,Extraction |
| Read this 25-page dataset README and extract the risks.,9,Long-context |
| Summarize this article about wildlife conservation in three bullet points.,1,Summary |
| Draft a whimsical flash-fiction piece about the French Revolution.,5,Creative |
| Write a SQL query to get the top 2 customers by revenue.,2,Coding |
| Write a song lyric about a long drive.,5,Creative |
| What is 14 + 11?,10,Fast/simple |
| Translate this paragraph into Russian: 'The quick brown fox jumps over the lazy dog.',6,Translation |
| Extract the key entities from this news article about coral reefs.,8,Extraction |
| What is 47 + 45?,10,Fast/simple |
| Write unit tests for this utility function.,2,Coding |
| Debug this segfault that occurs when running my script.,2,Coding |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: skyscrapers.",4,Agentic |
| Write a function that tokenize a sentence and returns a dict.,2,Coding |
| Make a concise summary of the key points in this transcript about climate change.,1,Summary |
| Draft a funny flash-fiction piece about the industrial revolution.,5,Creative |
| Give me the Dutch equivalent of this idiom: 'spill the beans'.,6,Translation |
| Translate these product instructions into French.,6,Translation |
| What is 35 + 5?,10,Fast/simple |
| What is 6 + 14?,10,Fast/simple |
| What is opportunity cost?,0,QA |
| Find and extract geographic locations and product names from this text.,8,Extraction |
| What are the main causes of space telescopes?,0,QA |
| Compose a haiku about machine learning.,5,Creative |
| Write a Go function to download and cache a page.,2,Coding |
| Give me a TL;DR of the following passage about coral reefs.,1,Summary |
| Draft a melancholic flash-fiction piece about ancient Greek philosophy.,5,Creative |
| Render this quote into Portuguese and explain any nuances.,6,Translation |
| How do you say the internet of things in Swahili?,6,Translation |
| What is 46 + 26?,10,Fast/simple |
| "Extract named entities (people, places, organizations) from this text about the internet of things.",8,Extraction |
| "If the probability of rain is 33%, what is the chance it does not rain?",3,Reasoning |
| Extract the main keyword phrases and topics from this passage on the jazz era.,8,Extraction |
| What is 35 + 44?,10,Fast/simple |
| Make a concise summary of the key points in this transcript about ancient Greek philosophy.,1,Summary |
| "Search the web for medieval castles, then book a flight and send me a summary.",4,Agentic |
| Give me a TL;DR of the following passage about Renaissance art.,1,Summary |
| A train departs at 8:00 traveling at 89 mph; how far in 5 hours?,3,Reasoning |
| Read these 15 documents about skyscrapers and synthesize a single answer.,9,Long-context |
| "Call the API to get data on medieval castles, then summarize the response and save it to disk.",4,Agentic |
| "Solve this logic puzzle: two guards, one lies, one tells the truth, and one door leads to freedom.",3,Reasoning |
| Briefly summarize the plot of this story about renewable energy.,1,Summary |
| "Based on the full dataset README about culinary history, explain the compliance requirements.",9,Long-context |
| Extract the steps and ingredients from this recipe for the French Revolution.,8,Extraction |
| Draft a melancholic flash-fiction piece about machine learning.,5,Creative |
| "There are 8 boxes, one holds a prize. If I open 5 wrong boxes, how many remain?",3,Reasoning |
| "Provide a brief summary of the meeting notes, focusing on the French Revolution.",1,Summary |
| What is 39 + 45?,10,Fast/simple |
| Give me a clear definition of renewable energy.,0,QA |
| Compose a haiku about volcanoes.,5,Creative |
| What is 2 + 14?,10,Fast/simple |
| Which topic does this paragraph best belong to? modern architecture mentions algorithms.,7,Classification |
| What time is it in Paris right now?,10,Fast/simple |
| "Which of these numbers is the odd one out and why: [52, 60, 50, 19]?",3,Reasoning |
| "Use the file tool to open config.json, then the search tool for the human brain, then combine both.",4,Agentic |
| Extract the key figures timestamp from this report.,8,Extraction |
| Write a letter to my future self about oceanography.,5,Creative |
| "Use the file tool to open notes.txt, then the search tool for skyscrapers, then combine both.",4,Agentic |
| "There are 8 boxes, one holds a prize. If I open 4 wrong boxes, how many remain?",3,Reasoning |
| "Read this 1,000-line codebase README and explain the compliance requirements.",9,Long-context |
| Extract the main keyword phrases and topics from this passage on blockchain.,8,Extraction |
| Is this statement a fact or an opinion: 'Mars exploration is largely misunderstood.'?,7,Classification |
| "Using this long codebase README about volcanoes, give the setup steps.",9,Long-context |
| Interpret this legal clause into plain Arabic.,6,Translation |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on deep-sea creatures.",4,Agentic |
| "If 3 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| Determine whether this tweet about the Silk Road is positive or negative.,7,Classification |
| "Coordinate a multi-agent system: one agent searches the Kalahari desert, another drafts a plan, and I review the result.",4,Agentic |
| "If the probability of rain is 37%, what is the chance it does not rain?",3,Reasoning |
| Create a dialogue between two characters arguing about the ocean floor.,5,Creative |
| Create a catchy tagline for a company that sells AI tools.,5,Creative |
| "Is this a question, a command, or a statement?",7,Classification |
| "Search the web for electric vehicles, then book a dinner and send me a summary.",4,Agentic |
| "There are 1 boxes, one holds a prize. If I open 5 wrong boxes, how many remain?",3,Reasoning |
| Pull out all dates and dollar amounts from the following receipt text.,8,Extraction |
| Convert this pseudocode into working JavaScript code.,2,Coding |
| Make a concise summary of the key points in this transcript about modern architecture.,1,Summary |
| A train departs at 2:00 traveling at 77 mph; how far in 6 hours?,3,Reasoning |
| Translate 'Where is the train station' into Portuguese.,6,Translation |
| Draft a melancholic flash-fiction piece about renewable energy.,5,Creative |
| Give me a clear definition of volcanoes.,0,QA |
| Which topic does this paragraph best belong to? volcanoes mentions algorithms.,7,Classification |
| What is 14 + 38?,10,Fast/simple |
| Compose the opening lines of a mystery novel set in a rainy city.,5,Creative |
| Is this statement a fact or an opinion: 'the Great Barrier Reef is largely misunderstood.'?,7,Classification |
| "Which of these numbers is the odd one out and why: [22, 29, 9, 68]?",3,Reasoning |
| Extract the steps and ingredients from this recipe for wildlife conservation.,8,Extraction |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: culinary history.",4,Agentic |
| "Using this long thesis about the stock market, summarize the main conclusions.",9,Long-context |
| "Use the file tool to open data.csv, then the search tool for the Kalahari desert, then combine both.",4,Agentic |
| What is 32 + 23?,10,Fast/simple |
| Create a dialogue between two characters arguing about the Roman Empire.,5,Creative |
| Give me a clear definition of medieval castles.,0,QA |
| "Based on the full report about space telescopes, list every deadline.",9,Long-context |
| Extract all legal terms present in the following medical note.,8,Extraction |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: renewable energy.",4,Agentic |
| Scan this legal document and extract every person names.,8,Extraction |
| Summarize this entire year of meeting notes covering quantum computing.,9,Long-context |
| What is the difference between oceanography and medieval castles?,0,QA |
| Who is Albert Einstein known for?,0,QA |
| Analyze this full 200-page report on Renaissance art and summarize the conclusions.,9,Long-context |
| How does the immune system actually work?,0,QA |
| Summarize this entire year of meeting notes covering the human brain.,9,Long-context |
| Interpret this legal clause into plain Korean.,6,Translation |
| "There are 9 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| What time is it in Toronto right now?,10,Fast/simple |
| Write a short executive summary of this report on deep-sea creatures.,1,Summary |
| Write a Python function to restructure a nested JSON.,2,Coding |
| Explain ancient Greek philosophy in beginner-friendly terms.,0,QA |
| How do you say oceanography in Italian?,6,Translation |
| Draft a funny flash-fiction piece about machine learning.,5,Creative |
| Write a letter to my future self about medieval castles.,5,Creative |
| Is this statement a fact or an opinion: 'the industrial revolution is largely misunderstood.'?,7,Classification |
| Extract the main keyword phrases and topics from this passage on space telescopes.,8,Extraction |
| Explain this Makefile to me.,2,Coding |
| "Coordinate a multi-agent system: one agent searches the Great Barrier Reef, another drafts a plan, and I review the result.",4,Agentic |
| Translate 'Where is the train station' into Japanese.,6,Translation |
| "Use the file tool to open data.csv, then the search tool for the solar system, then combine both.",4,Agentic |
| Give me the Korean equivalent of this idiom: 'bite the bullet'.,6,Translation |
| Write a function that normalize a phone number and returns a dict.,2,Coding |
| What is 24 + 26?,10,Fast/simple |
| "Using this long contract about the solar system, explain the compliance requirements.",9,Long-context |
| Process this 10-page research paper on quantum computing; then compare the revisions.,9,Long-context |
| "Which of these topics is this text about: quantum computing, ancient Egypt, the Great Barrier Reef?",7,Classification |
| "You have access to a search tool, a calculator, and a file system. Please write and run the steps.",4,Agentic |
| Design a schema for an app that tracks renewable energy.,2,Coding |
| Read the whole 120-page report which covers quantum computing and summarize the key provisions.,9,Long-context |
| "Call the API to get data on machine learning, then summarize the response and save it to disk.",4,Agentic |
| Create a dialogue between two characters arguing about ancient Egypt.,5,Creative |
| Translate this recipe into Spanish.,6,Translation |
| Compose a haiku about artificial intelligence.,5,Creative |
| Which topic does this paragraph best belong to? the Silk Road mentions algorithms.,7,Classification |
| Mary buys 2 apples at $2 each and a loaf for $3. Total cost?,3,Reasoning |
| Convert this English text to Mandarin and keep the tone casual.,6,Translation |
| Translate the following business email into Arabic.,6,Translation |
| What time is it in Sydney right now?,10,Fast/simple |
| Build a a billing module with proper error handling.,2,Coding |
| Summarize this 10-page document in under 50 words.,1,Summary |
| "Which of these topics is this text about: electric vehicles, the Great Barrier Reef, the internet of things?",7,Classification |
| Create a dialogue between two characters arguing about the immune system.,5,Creative |
| Explain coral reefs in beginner-friendly terms.,0,QA |
| Interpret this legal clause into plain Portuguese.,6,Translation |
| Explain basketball in plain terms.,0,QA |
| Say hi to me.,10,Fast/simple |
| What is the difference between bees and pollination and ancient Greek philosophy?,0,QA |
| Extract the steps and ingredients from this recipe for renewable energy.,8,Extraction |
| Use your registry of tools to send an email and report back in a CSV.,4,Agentic |
| Compose a haiku about deep-sea creatures.,5,Creative |
| Read the whole 25-page thesis which covers modern architecture and extract the risks.,9,Long-context |
| How do you say electric vehicles in Arabic?,6,Translation |
| Process this 50-page thesis on photography; then explain the compliance requirements.,9,Long-context |
| Write a short executive summary of this report on bees and pollination.,1,Summary |
| Pull the dates and locations out of this meeting minutes about bees and pollination.,8,Extraction |
| Write a Python script to migrate a schema.,2,Coding |
| What is 18 + 44?,10,Fast/simple |
| Write a docstring and type hints for a Rust function that processes the Silk Road.,2,Coding |
| "Which of these numbers is the odd one out and why: [10, 34, 42, 2]?",3,Reasoning |
| Localize this ad copy about quantum computing into Japanese.,6,Translation |
| "Given this lengthy source, extract the section that discusses timeline.",9,Long-context |
| Extract the main keyword phrases and topics from this passage on deep-sea creatures.,8,Extraction |
| Write a letter to my future self about the stock market.,5,Creative |
| Translate the following business email into Greek.,6,Translation |
| Give me a 3-bullet recap of oceanography.,1,Summary |
| Compose the opening lines of a mystery novel set in a desert town.,5,Creative |
| What is the difference between wildlife conservation and Renaissance art?,0,QA |
| Summarize this article about the French Revolution in one line.,1,Summary |
| What is 19 + 49?,10,Fast/simple |
| "Use the file tool to open config.json, then the search tool for climate change, then combine both.",4,Agentic |
| "Coordinate a multi-agent system: one agent searches the solar system, another drafts a plan, and I review the result.",4,Agentic |
| Extract the key entities from this report about blockchain.,8,Extraction |
| How do you say the human brain in Swahili?,6,Translation |
| Give me a TL;DR of the following passage about artificial intelligence.,1,Summary |
| Write a letter to my future self about coral reefs.,5,Creative |
| A train departs at 10:00 traveling at 81 mph; how far in 3 hours?,3,Reasoning |
| Extract all legal terms and emails and phone numbers from the text about electric vehicles.,8,Extraction |
| Localize this ad copy about the Silk Road into Hindi.,6,Translation |
| "Which of these topics is this text about: traditional cuisine, the immune system, volcanoes?",7,Classification |
| Explain the human brain in simple terms.,0,QA |
| A train departs at 8:00 traveling at 77 mph; how far in 3 hours?,3,Reasoning |
| Localize this ad copy about the stock market into Japanese.,6,Translation |
| Read the whole 10-page research paper which covers oceanography and summarize the key provisions.,9,Long-context |
| Render this quote into French and explain any nuances.,6,Translation |
| Compose a haiku about the human genome.,5,Creative |
| A train departs at 6:00 traveling at 86 mph; how far in 2 hours?,3,Reasoning |
| Write a letter to my future self about ancient Greek philosophy.,5,Creative |
| Which topic does this paragraph best belong to? the human genome mentions flights.,7,Classification |
| "Return a JSON object with people, places, organizations parsed from this meeting minutes.",8,Extraction |
| Scan this order confirmation and extract every geographic locations.,8,Extraction |
| Compose a haiku about Renaissance art.,5,Creative |
| Summarize this lengthy document in under 50 words.,1,Summary |
| Write a letter to my future self about photography.,5,Creative |
| "If 12 books each have a unique title, how many distinct titles are there?",3,Reasoning |
| Condense this chapter into the main takeaways about the solar system.,1,Summary |
| "There are 8 boxes, one holds a prize. If I open 2 wrong boxes, how many remain?",3,Reasoning |
| Give me a TL;DR of the following passage about space telescopes.,1,Summary |
| "Using this long report about electric vehicles, explain the compliance requirements.",9,Long-context |
| Draft a melancholic flash-fiction piece about the human brain.,5,Creative |
| Explain this git merge conflict to me.,2,Coding |
| Find and extract emails and phone numbers and emails and phone numbers from this text.,8,Extraction |
| What time is it in Oslo right now?,10,Fast/simple |
| Give me a clear definition of the French Revolution.,0,QA |
| Draft a whimsical flash-fiction piece about Renaissance art.,5,Creative |
| How do you say wildlife conservation in Portuguese?,6,Translation |
| Create a dialogue between two characters arguing about the internet of things.,5,Creative |
| Summarize this article about volcanoes in one line.,1,Summary |
| Process this 500-page research paper on blockchain; then return a structured overview.,9,Long-context |
| Pull out the total from this report.,8,Extraction |
| What is 21 + 26?,10,Fast/simple |
| Explain this regex to me.,2,Coding |
| Review this entire thesis and summarize the key provisions.,9,Long-context |
| "Coordinate a multi-agent system: one agent searches volcanoes, another drafts a plan, and I review the result.",4,Agentic |
| Which topic does this paragraph best belong to? bees and pollination mentions players.,7,Classification |
| What is 11 + 50?,10,Fast/simple |
| Localize this ad copy about space telescopes into Spanish.,6,Translation |
| What is 34 + 47?,10,Fast/simple |
| Summarize this entire year of meeting notes covering the stock market.,9,Long-context |
| Review this entire legal document and extract the recommendations.,9,Long-context |
| What is 42 + 43?,10,Fast/simple |
| Read the attached long document and answer questions from it.,9,Long-context |
| Localize this ad copy about artificial intelligence into German.,6,Translation |
| Write a docstring and type hints for a Java function that processes skyscrapers.,2,Coding |
| Explain modern architecture in beginner-friendly terms.,0,QA |
| Translate 'Thank you very much' into Korean.,6,Translation |
| Determine whether this tweet about renewable energy is positive or negative.,7,Classification |
| Localize this ad copy about ancient Egypt into Arabic.,6,Translation |
| Determine whether this tweet about Renaissance art is positive or negative.,7,Classification |
| Is this statement a fact or an opinion: 'traditional cuisine is largely misunderstood.'?,7,Classification |
| Explain the French Revolution in plain terms.,0,QA |
| Draft a funny flash-fiction piece about volcanoes.,5,Creative |
| Improve the performance of my JavaScript program that sort a list of dicts.,2,Coding |
| Who is Ada Lovelace known for?,0,QA |
| What are the main causes of the ocean floor?,0,QA |
| What is 10 + 47?,10,Fast/simple |
| Explain this grep command to me.,2,Coding |
| "Which of these numbers is the odd one out and why: [44, 58, 42, 46]?",3,Reasoning |
| Determine whether this tweet about basketball is positive or negative.,7,Classification |
| Write a Python function to normalize a phone number.,2,Coding |
| Write a short executive summary of this report on blockchain.,1,Summary |
| Improve the performance of my Java program that normalize a phone number.,2,Coding |
| Localize this ad copy about Renaissance art into Russian.,6,Translation |
| What is the difference between ancient Greek philosophy and renewable energy?,0,QA |
| "Return a JSON object with name, date, amount parsed from this meeting minutes.",8,Extraction |
| What is 10 + 14?,10,Fast/simple |
| "Here is a 40-page contract. Based on it, what are the termination clauses?",9,Long-context |
| "Based on the full contract about the ocean floor, extract the risks.",9,Long-context |
| Condense this chapter into the main takeaways about volcanoes.,1,Summary |
| "Given this 120-page dataset README about basketball, give the setup steps.",9,Long-context |
| "Given this 500-page dataset README about medieval castles, list every deadline.",9,Long-context |
| What is 41 + 10?,10,Fast/simple |
| "There are 5 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| "Using this long dataset README about volcanoes, answer the question about cost.",9,Long-context |
| Pull out the due date from this order confirmation.,8,Extraction |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on the human genome.",4,Agentic |
| "If the probability of rain is 42%, what is the chance it does not rain?",3,Reasoning |
| Determine whether this tweet about space telescopes is positive or negative.,7,Classification |
| "Provide a brief summary of the customer interview, focusing on space telescopes.",1,Summary |
| "There are 2 boxes, one holds a prize. If I open 3 wrong boxes, how many remain?",3,Reasoning |
| What is 19 + 32?,10,Fast/simple |
| Give me a TL;DR of the following passage about the moon landing.,1,Summary |
| Write a short executive summary of this report on the industrial revolution.,1,Summary |
| Give me a one-sentence recap of space telescopes.,1,Summary |
| What is 3 + 9?,10,Fast/simple |
| Mary buys 5 apples at $2 each and a loaf for $4. Total cost?,3,Reasoning |
| Give me a one-sentence recap of the moon landing.,1,Summary |
| "Assign this article to one of: politics, business, science, culture.",7,Classification |
| "Provide a brief summary of the customer interview, focusing on the solar system.",1,Summary |
| What is 30 + 42?,10,Fast/simple |
| Draft a whimsical flash-fiction piece about the solar system.,5,Creative |
| Extract all geographic locations present in the following receipt.,8,Extraction |
| Compare two chapters of this long transcript and note contradictions.,9,Long-context |
| "Extract named entities (people, places, organizations) from this text about oceanography.",8,Extraction |
| Draft a melancholic flash-fiction piece about the jazz era.,5,Creative |
| Is this statement a fact or an opinion: 'the French Revolution is largely misunderstood.'?,7,Classification |
| "Search the web for basketball, then book a dinner and send me a summary.",4,Agentic |
| What is 32 + 4?,10,Fast/simple |
| Give me a clear definition of the jazz era.,0,QA |
| Here is a 50-page regulatory filing. Analyze it and explain the compliance requirements.,9,Long-context |
| Is this statement a fact or an opinion: 'basketball is largely misunderstood.'?,7,Classification |
| Find and extract geographic locations and emails and phone numbers from this text.,8,Extraction |
| How many planets are in our solar system?,0,QA |
| Draft a funny flash-fiction piece about medieval castles.,5,Creative |
| What is the capital of Sao Paulo?,0,QA |
| A train departs at 4:00 traveling at 53 mph; how far in 6 hours?,3,Reasoning |
| What is 37 + 5?,10,Fast/simple |
| Review this entire legal document and return a structured overview.,9,Long-context |
| Translate 'I would like a coffee' into Arabic.,6,Translation |
| Design a schema for an app that tracks the jazz era.,2,Coding |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Cairo'.,4,Agentic |
| Read these 15 documents about photography and synthesize a single answer.,9,Long-context |
| Draft a whimsical flash-fiction piece about artificial intelligence.,5,Creative |
| What is 32 + 31?,10,Fast/simple |
| What is 40 + 37?,10,Fast/simple |
| Write a docstring and type hints for a TypeScript function that processes the industrial revolution.,2,Coding |
| Process this 300-page report on bees and pollination; then answer the question about cost.,9,Long-context |
| What is 5 + 8?,10,Fast/simple |
| Extract the main keyword phrases and topics from this passage on oceanography.,8,Extraction |
| Draft a funny flash-fiction piece about the moon landing.,5,Creative |
| Make a concise summary of the key points in this transcript about bees and pollination.,1,Summary |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: the Silk Road.",4,Agentic |
| Design a schema for an app that tracks skyscrapers.,2,Coding |
| Improve the performance of my Rust program that flatten a nested list.,2,Coding |
| "Which of these topics is this text about: basketball, the human brain, coral reefs?",7,Classification |
| Draft a funny flash-fiction piece about photography.,5,Creative |
| Read this 120-page legal document and summarize the key provisions.,9,Long-context |
| Which topic does this paragraph best belong to? Renaissance art mentions recipes.,7,Classification |
| Give me an executive recap of artificial intelligence.,1,Summary |
| Explain quantum computing in beginner-friendly terms.,0,QA |
| Use your registry of tools to monitor a server and report back in a JSON summary.,4,Agentic |
| Design a schema for an app that tracks the industrial revolution.,2,Coding |
| "Coordinate a multi-agent system: one agent searches traditional cuisine, another drafts a plan, and I review the result.",4,Agentic |
| List every dates and locations mentioned in this report on the immune system.,8,Extraction |
| A train departs at 8:00 traveling at 60 mph; how far in 2 hours?,3,Reasoning |
| "Here is a 1,000-line codebase README. Analyze it and compare the revisions.",9,Long-context |
| What is 16 + 2?,10,Fast/simple |
| Write a function that flatten a nested list and returns a generator.,2,Coding |
| "Call the API to get data on the internet of things, then summarize the response and save it to disk.",4,Agentic |
| What is 20 + 40?,10,Fast/simple |
| What are the main causes of the industrial revolution?,0,QA |
| Write a SQL query to get the top 8 customers by revenue.,2,Coding |
| Give me an executive recap of ancient Greek philosophy.,1,Summary |
| Explain Mars exploration in beginner-friendly terms.,0,QA |
| Here is the full regulatory filing for machine learning (50-page). List the action items.,9,Long-context |
| Here is the full codebase README for photography (10-page). List the action items.,9,Long-context |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Cairo.",4,Agentic |
| "Which of these topics is this text about: the moon landing, the stock market, the jazz era?",7,Classification |
| Read this 10-page contract and give the setup steps.,9,Long-context |
| Draft a funny flash-fiction piece about the human brain.,5,Creative |
| Create a dialogue between two characters arguing about electric vehicles.,5,Creative |
| Create a CI workflow for a Python project.,2,Coding |
| Compose the opening lines of a mystery novel set in a small village.,5,Creative |
| Summarize this article about medieval castles in three bullet points.,1,Summary |
| Create a dialogue between two characters arguing about renewable energy.,5,Creative |
| Write a function that parse a CSV and returns a dict.,2,Coding |
| "Process this 1,000-line transcript on the Roman Empire; then summarize the main conclusions.",9,Long-context |
| Which topic does this paragraph best belong to? the Kalahari desert mentions recipes.,7,Classification |
| "Which of these topics is this text about: bees and pollination, Mars exploration, wildlife conservation?",7,Classification |
| Determine whether this tweet about blockchain is positive or negative.,7,Classification |
| Refactor this pipeline to be more readable.,2,Coding |
| Write a letter to my future self about the internet of things.,5,Creative |
| Analyze this full 200-page report on space telescopes and summarize the conclusions.,9,Long-context |
| Extract the key entities from this meeting minutes about the internet of things.,8,Extraction |
| What are the main causes of ancient Egypt?,0,QA |
| Which topic does this paragraph best belong to? the solar system mentions players.,7,Classification |
| Write a function that compute the Fibonacci sequence and returns a dict.,2,Coding |
| "Which of these topics is this text about: quantum computing, the ocean floor, traditional cuisine?",7,Classification |
| Give me a 3-bullet recap of Mars exploration.,1,Summary |
| "Search the web for skyscrapers, then book a dinner and send me a summary.",4,Agentic |
| "Which of these topics is this text about: ancient Greek philosophy, wildlife conservation, climate change?",7,Classification |
| Build a a prompt parser with proper error handling.,2,Coding |
| Extract the steps and ingredients from this recipe for modern architecture.,8,Extraction |
| "Which of these numbers is the odd one out and why: [13, 73, 66, 11]?",3,Reasoning |
| "There are 1 boxes, one holds a prize. If I open 2 wrong boxes, how many remain?",3,Reasoning |
| Write a TypeScript script to migrate a schema.,2,Coding |
| Extract all company names and product names from this ad copy.,8,Extraction |
| Is this statement a fact or an opinion: 'the human genome is largely misunderstood.'?,7,Classification |
| Create a CI workflow for a Node project.,2,Coding |
| Write a Python function to flatten a nested list.,2,Coding |
| "Using this long regulatory filing about the Great Barrier Reef, answer the question about cost.",9,Long-context |
| "Which of these topics is this text about: wildlife conservation, culinary history, the Silk Road?",7,Classification |
| Design a schema for an app that tracks modern architecture.,2,Coding |
| What is 43 + 21?,10,Fast/simple |
| Summarize this article about the jazz era in one line.,1,Summary |
| Give me a clear definition of ancient Egypt.,0,QA |
| What is 19 + 27?,10,Fast/simple |
| Compose a haiku about basketball.,5,Creative |
| "Which of these numbers is the odd one out and why: [39, 10, 64, 47]?",3,Reasoning |
| Extract all legal terms and legal terms from the text about electric vehicles.,8,Extraction |
| Optimize this recursive function to run faster.,2,Coding |
| List every key stats and numbers mentioned in this receipt on medieval castles.,8,Extraction |
| "Based on the full legal document about electric vehicles, compare the revisions.",9,Long-context |
| Extract all key figures and emails and phone numbers from the text about the Kalahari desert.,8,Extraction |
| What is 43 + 43?,10,Fast/simple |
| What is 3 + 40?,10,Fast/simple |
| Extract the key figures timestamp from this meeting minutes.,8,Extraction |
| Render this quote into Greek and explain any nuances.,6,Translation |
| "Coordinate a multi-agent system: one agent searches photography, another drafts a plan, and I review the result.",4,Agentic |
| Draft a whimsical flash-fiction piece about volcanoes.,5,Creative |
| What is 27 + 14?,10,Fast/simple |
| What is the difference between wildlife conservation and basketball?,0,QA |
| Write a Python script to scrape a webpage.,2,Coding |
| Write a letter to my future self about the Roman Empire.,5,Creative |
| Scan this news article and extract every dates.,8,Extraction |
| "If the probability of rain is 27%, what is the chance it does not rain?",3,Reasoning |
| What is 25 + 23?,10,Fast/simple |
| Translate the following business email into Spanish.,6,Translation |
| "You are an agent with browsing, retrieval, and code execution skills. Achieve: bees and pollination.",4,Agentic |
| Draft a whimsical flash-fiction piece about the internet of things.,5,Creative |
| Give me a clear definition of the Roman Empire.,0,QA |
| Draft a funny flash-fiction piece about the Roman Empire.,5,Creative |
| "Provide a brief summary of the customer interview, focusing on photography.",1,Summary |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on volcanoes.",4,Agentic |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Sao Paulo'.,4,Agentic |
| "Use the file tool to open config.json, then the search tool for quantum computing, then combine both.",4,Agentic |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on coral reefs.",4,Agentic |
| "Orchestrate a multi-step workflow: fetch data, clean it, train a model, and return a report on renewable energy.",4,Agentic |
| "Which of these topics is this text about: the Silk Road, ancient Egypt, modern architecture?",7,Classification |
| "Call the API to get data on oceanography, then summarize the response and save it to disk.",4,Agentic |
| Condense this chapter into the main takeaways about the Kalahari desert.,1,Summary |
| What does FYI mean?,10,Fast/simple |
| Draft a funny flash-fiction piece about the French Revolution.,5,Creative |
| How do you say wildlife conservation in French?,6,Translation |
| Explain the immune system in technical terms.,0,QA |
| Extract the key figures date from this medical note.,8,Extraction |
| "Which of these numbers is the odd one out and why: [47, 70, 31, 72]?",3,Reasoning |
| What is 25 + 4?,10,Fast/simple |
| What is the difference between ancient Egypt and volcanoes?,0,QA |
| Build a a regex validator with proper error handling.,2,Coding |
| Improve the performance of my JavaScript program that restructure a nested JSON.,2,Coding |
| Translate 'Thank you very much' into German.,6,Translation |
| What is the difference between volcanoes and the jazz era?,0,QA |
| Make a concise summary of the key points in this transcript about the jazz era.,1,Summary |
| What is compound interest?,0,QA |
| "Which of these numbers is the odd one out and why: [27, 30, 51, 23]?",3,Reasoning |
| Write a function that download and cache a page and returns a dict.,2,Coding |
| What is 50 + 46?,10,Fast/simple |
| What is the difference between the human brain and basketball?,0,QA |
| Mary buys 2 apples at $1 each and a loaf for $3. Total cost?,3,Reasoning |
| Extract all legal terms present in the following news article.,8,Extraction |
| Extract all organizations and emails and phone numbers from the text about electric vehicles.,8,Extraction |
| Process this 10-page regulatory filing on wildlife conservation; then return a structured overview.,9,Long-context |
| What is 23 + 44?,10,Fast/simple |
| Which topic does this paragraph best belong to? the industrial revolution mentions algorithms.,7,Classification |
| Plan how an autonomous agent would complete the goal 'book a table for 4 near Toronto'.,4,Agentic |
| "There are 5 boxes, one holds a prize. If I open 5 wrong boxes, how many remain?",3,Reasoning |
| What is 50 + 18?,10,Fast/simple |
| "Which of these numbers is the odd one out and why: [42, 77, 5, 75]?",3,Reasoning |
| "Here is the full codebase README for the immune system (1,000-line). Extract the key clauses.",9,Long-context |
| "Which of these numbers is the odd one out and why: [46, 51, 26, 53]?",3,Reasoning |
| Analyze this full 200-page report on the internet of things and summarize the conclusions.,9,Long-context |
| What is 42 + 31?,10,Fast/simple |
| Just answer yes or no: is the sky blue?,10,Fast/simple |
| "There are 1 boxes, one holds a prize. If I open 6 wrong boxes, how many remain?",3,Reasoning |
| Make a concise summary of the key points in this transcript about the industrial revolution.,1,Summary |
| What is 18 + 28?,10,Fast/simple |
| Summarize the key findings from this research paper on the industrial revolution.,1,Summary |
| "If the probability of rain is 69%, what is the chance it does not rain?",3,Reasoning |
| Design a schema for an app that tracks basketball.,2,Coding |
| A train departs at 2:00 traveling at 44 mph; how far in 3 hours?,3,Reasoning |
| Debug this ValueError that occurs when running my script.,2,Coding |
| Draft a melancholic flash-fiction piece about wildlife conservation.,5,Creative |
| Explain the time complexity of this algorithm that restructure a nested JSON.,2,Coding |
| Why is the moon visible at night?,0,QA |
| What is 7 + 22?,10,Fast/simple |
| What are the main causes of bees and pollination?,0,QA |
| Summarize the key findings from this research paper on renewable energy.,1,Summary |
| Summarize this entire year of meeting notes covering the Roman Empire.,9,Long-context |
| Determine whether this tweet about traditional cuisine is positive or negative.,7,Classification |
| Write a docstring and type hints for a Go function that processes traditional cuisine.,2,Coding |
| Write a one-paragraph summary of this report on bees and pollination.,1,Summary |
| Summarize the key findings from this research paper on electric vehicles.,1,Summary |
| Draft a whimsical flash-fiction piece about traditional cuisine.,5,Creative |
| What are the main causes of artificial intelligence?,0,QA |
| Write a letter to my future self about the moon landing.,5,Creative |
| Extract the key figures percentage from this insurance policy.,8,Extraction |
| Extract the key figures percentage from this news article.,8,Extraction |
| "If the probability of rain is 63%, what is the chance it does not rain?",3,Reasoning |
| A train departs at 3:00 traveling at 89 mph; how far in 3 hours?,3,Reasoning |
| Here is a 500-page regulatory filing. Analyze it and summarize the main conclusions.,9,Long-context |
| "Coordinate a multi-agent system: one agent searches renewable energy, another drafts a plan, and I review the result.",4,Agentic |
| "Provide a brief summary of the customer interview, focusing on artificial intelligence.",1,Summary |
| Add error handling and logging to this API route.,2,Coding |
| "Based on the full user manual about ancient Egypt, summarize the key provisions.",9,Long-context |
| What is 24 + 13?,10,Fast/simple |
| Explain the time complexity of this algorithm that tokenize a sentence.,2,Coding |
| How do you say the ocean floor in Russian?,6,Translation |
| Write a SQL query to get the top 10 products by revenue.,2,Coding |
| Summarize this entire year of meeting notes covering renewable energy.,9,Long-context |
| A train departs at 3:00 traveling at 83 mph; how far in 4 hours?,3,Reasoning |
| What is 5 + 32?,10,Fast/simple |
| Write a Go function to merge two sorted arrays.,2,Coding |
| Extract the steps and ingredients from this recipe for volcanoes.,8,Extraction |
| Create a dialogue between two characters arguing about the jazz era.,5,Creative |
| What is the difference between basketball and blockchain?,0,QA |
| What is the difference between ancient Greek philosophy and traditional cuisine?,0,QA |
| Write a docstring and type hints for a Go function that processes blockchain.,2,Coding |
| Design a schema for an app that tracks photography.,2,Coding |
| "Search the web for the human brain, then book a hotel and send me a summary.",4,Agentic |
| "Read the whole 1,000-line legal document which covers blockchain and answer the question about cost.",9,Long-context |
| Here is the full dataset README for the Silk Road (25-page). List the action items.,9,Long-context |
| Here is a 500-page research paper. Analyze it and list every deadline.,9,Long-context |
| "Extract named entities (people, places, organizations) from this text about the Silk Road.",8,Extraction |
| Give me the Hindi equivalent of this idiom: 'spill the beans'.,6,Translation |
| Read this 300-page thesis and answer the question about cost.,9,Long-context |
| Is this statement a fact or an opinion: 'renewable energy is largely misunderstood.'?,7,Classification |
| Write a Python script to back up a database.,2,Coding |
| What is the difference between skyscrapers and the human brain?,0,QA |
| Write a function that sort a list of dicts and returns a dict.,2,Coding |
| What is 39 + 6?,10,Fast/simple |
| "Which of these topics is this text about: volcanoes, the Silk Road, machine learning?",7,Classification |
| "Coordinate a multi-agent system: one agent searches the internet of things, another drafts a plan, and I review the result.",4,Agentic |
| Which topic does this paragraph best belong to? coral reefs mentions algorithms.,7,Classification |
| "Which of these topics is this text about: the Silk Road, oceanography, the jazz era?",7,Classification |
| When was the first computer invented?,0,QA |
| "Search the web for the stock market, then book a dinner and send me a summary.",4,Agentic |
| Extract the main keyword phrases and topics from this passage on volcanoes.,8,Extraction |
| "Based on the full codebase README about ancient Greek philosophy, give the setup steps.",9,Long-context |
| What is 35 + 29?,10,Fast/simple |
| Which topic does this paragraph best belong to? the internet of things mentions budgets.,7,Classification |
| "If the probability of rain is 71%, what is the chance it does not rain?",3,Reasoning |
| "Chain these actions: look up the weather, find a restaurant, and add it to my calendar in Oslo.",4,Agentic |
| Design a schema for an app that tracks the Roman Empire.,2,Coding |
| Convert this English text to Japanese and keep the tone casual.,6,Translation |
| Condense this chapter into the main takeaways about artificial intelligence.,1,Summary |
| Draft a melancholic flash-fiction piece about the industrial revolution.,5,Creative |
| Find and extract geographic locations and key figures from this text.,8,Extraction |
| Summarize the key findings from this research paper on climate change.,1,Summary |
| Translate the following business email into Japanese.,6,Translation |
| "Categorize this support ticket by urgency: low, medium, or high.",7,Classification |
| What is the difference between space telescopes and the industrial revolution?,0,QA |
| Convert this English text to Portuguese and keep the tone neutral.,6,Translation |
| What is the difference between space telescopes and the immune system?,0,QA |
| Create a dialogue between two characters arguing about coral reefs.,5,Creative |
| Write a Go function to normalize a phone number.,2,Coding |
| Extract the steps and ingredients from this recipe for medieval castles.,8,Extraction |
| Convert this English text to Dutch and keep the tone formal.,6,Translation |
| "Which of these topics is this text about: Mars exploration, the human brain, artificial intelligence?",7,Classification |
| "If the probability of rain is 54%, what is the chance it does not rain?",3,Reasoning |
| What is 47 + 21?,10,Fast/simple |
| Improve the performance of my Go program that compute the Fibonacci sequence.,2,Coding |
| "There are 9 boxes, one holds a prize. If I open 6 wrong boxes, how many remain?",3,Reasoning |
| Summarize the key findings from this research paper on oceanography.,1,Summary |
| "Call the API to get data on the immune system, then summarize the response and save it to disk.",4,Agentic |
| List every named entities and keywords mentioned in this report on Renaissance art.,8,Extraction |
| Extract the main keyword phrases and topics from this passage on the French Revolution.,8,Extraction |
| Create a dialogue between two characters arguing about skyscrapers.,5,Creative |
| Summarize this article about the Kalahari desert in three bullet points.,1,Summary |
| Write unit tests for this parser.,2,Coding |
| Mary buys 2 apples at $3 each and a loaf for $4. Total cost?,3,Reasoning |
| Translate this recipe into Hindi.,6,Translation |
| Render this quote into Korean and explain any nuances.,6,Translation |
| Summarize this article about the human genome in two sentences.,1,Summary |
| How does evaporation actually work?,0,QA |
| What's the weather like in Sydney today?,10,Fast/simple |
| "There are 3 boxes, one holds a prize. If I open 7 wrong boxes, how many remain?",3,Reasoning |
| Improve the performance of my TypeScript program that reverse a string.,2,Coding |
| Design a schema for an app that tracks the ocean floor.,2,Coding |
| What is 12 + 9?,10,Fast/simple |
| Write a letter to my future self about ancient Egypt.,5,Creative |
| Summarize this entire year of meeting notes covering the French Revolution.,9,Long-context |
| Extract the key figures timestamp from this insurance policy.,8,Extraction |
| "Search the web for the moon landing, then book a dinner and send me a summary.",4,Agentic |
|
|
