17 84 23

Yang Lee

innovation64

https://innovation64.github.io/

AI & ML interests

AGI

Recent Activity

reacted to mmhamdy's post with 👀 about 13 hours ago

⛓ Evaluating Long Context #2: SCROLLS and ZeroSCROLLS In this series of posts about tracing the history of long context evaluation, we started with Long Range Arena (LRA). Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation. But it wasn't introduced to evaluate LLMs, but rather the transformer architecture in general. 📜 The SCROLLS benchmark, introduced in 2022, addresses this gap in NLP/LLM research. SCROLLS challenges models with tasks that require reasoning over extended sequences (according to 2022 standards). So, what does it offer? 1️⃣ Long Text Focus: SCROLLS (unlike LRA) focus mainly on text and contain inputs with thousands of words, testing models' ability to synthesize information across lengthy documents. 2️⃣ Diverse Tasks: Includes summarization, question answering, and natural language inference across domains like literature, science, and business. 3️⃣ Unified Format: All datasets are available in a text-to-text format, facilitating easy evaluation and comparison of models. Building on SCROLLS, ZeroSCROLLS takes long text evaluation to the next level by focusing on zero-shot learning. Other features include: 1️⃣ New Tasks: Introduces tasks like sentiment aggregation and sorting book chapter summaries. 2️⃣ Leaderboard: A live leaderboard encourages continuous improvement and competition among researchers. 💡 What are some other landmark benchmarks in the history of long context evaluation? Feel free to share your thoughts and suggestions in the comments. - SCROLLS Paper: https://huggingface.co/papers/2201.03533 - ZeroSCROLLS Paper: https://huggingface.co/papers/2305.14196

updated a dataset 7 days ago

Apocalypse-AGI-DAO/CondAmbigQA

published a dataset 8 days ago

Apocalypse-AGI-DAO/CondAmbigQA

View all activity

Organizations

innovation64's activity

liked a Space 4 months ago

3.99k

Chatbot Arena Leaderboard

🏆

liked a dataset 4 months ago

google/frames-benchmark

Viewer • Updated Oct 15, 2024 • 824 • 1.59k • 180

liked a dataset 6 months ago

hbx/IN3

Viewer • Updated Feb 20, 2024 • 1.37k • 114 • 7

liked a Space 8 months ago

130

Hallucinations Leaderboard

🔥

View and submit LLM evaluations

liked 3 models 8 months ago

liked a Space 9 months ago

163

Low-bit Quantized Open LLM Leaderboard

🏆

Track, rank and evaluate open LLMs and chatbots

liked 3 models 10 months ago

meta-llama/Meta-Llama-3-8B-Instruct

Text Generation • Updated Sep 27, 2024 • 2.13M • • 3.8k

meta-llama/Meta-Llama-3-8B

Text Generation • Updated Sep 27, 2024 • 648k • 6.01k

NexaAIDev/Octopus-v2

Text Generation • Updated May 21, 2024 • 1.03k • 871

liked a Space 10 months ago

12.4k

Open LLM Leaderboard

🏆

Track, rank and evaluate open LLMs and chatbots

liked 5 models 11 months ago

bigcode/octogeex

Text Generation • Updated Aug 17, 2023 • 126 • 22

mistralai/Mixtral-8x7B-Instruct-v0.1

Text Generation • Updated Aug 19, 2024 • 488k • • 4.3k

mistralai/Mistral-7B-Instruct-v0.2

Text Generation • Updated Sep 27, 2024 • 3.24M • • 2.65k

xai-org/grok-1

Text Generation • Updated Mar 28, 2024 • 577 • 2.23k

meta-llama/Llama-2-7b-chat-hf

Text Generation • Updated Apr 17, 2024 • 1.33M • • 4.21k

liked 2 datasets 12 months ago

akariasai/PopQA

Viewer • Updated Dec 22, 2022 • 14.3k • 3.39k • 21

selfrag/selfrag_train_data

Viewer • Updated Oct 31, 2023 • 146k • 116 • 69

liked a model 12 months ago

google/gemma-7b

Text Generation • Updated Jun 27, 2024 • 66.2k • • 3.12k