17 84 23

Yang Lee

innovation64

https://innovation64.github.io/

AI & ML interests

AGI

Recent Activity

reacted to mmhamdy's post with 👀 about 13 hours ago

⛓ Evaluating Long Context #2: SCROLLS and ZeroSCROLLS In this series of posts about tracing the history of long context evaluation, we started with Long Range Arena (LRA). Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation. But it wasn't introduced to evaluate LLMs, but rather the transformer architecture in general. 📜 The SCROLLS benchmark, introduced in 2022, addresses this gap in NLP/LLM research. SCROLLS challenges models with tasks that require reasoning over extended sequences (according to 2022 standards). So, what does it offer? 1️⃣ Long Text Focus: SCROLLS (unlike LRA) focus mainly on text and contain inputs with thousands of words, testing models' ability to synthesize information across lengthy documents. 2️⃣ Diverse Tasks: Includes summarization, question answering, and natural language inference across domains like literature, science, and business. 3️⃣ Unified Format: All datasets are available in a text-to-text format, facilitating easy evaluation and comparison of models. Building on SCROLLS, ZeroSCROLLS takes long text evaluation to the next level by focusing on zero-shot learning. Other features include: 1️⃣ New Tasks: Introduces tasks like sentiment aggregation and sorting book chapter summaries. 2️⃣ Leaderboard: A live leaderboard encourages continuous improvement and competition among researchers. 💡 What are some other landmark benchmarks in the history of long context evaluation? Feel free to share your thoughts and suggestions in the comments. - SCROLLS Paper: https://huggingface.co/papers/2201.03533 - ZeroSCROLLS Paper: https://huggingface.co/papers/2305.14196

updated a dataset 7 days ago

Apocalypse-AGI-DAO/CondAmbigQA

published a dataset 8 days ago

Apocalypse-AGI-DAO/CondAmbigQA

View all activity

Organizations

innovation64's activity

New activity in huggingchat/chat-ui about 1 year ago

[ASSISTANTS] Community thread

185

#356 opened about 1 year ago by

victor

New activity in HuggingFace-CN-community/translation almost 2 years ago

[glossary] Episodic task - 片段任务

#27 opened about 2 years ago by

innovation64

Upload using-ml-for-disasters-cn.md

#63 opened almost 2 years ago by

innovation64

Upload red-teaming-cn.md

#51 opened almost 2 years ago by

innovation64

Upload red-teaming-cn.md

#52 opened almost 2 years ago by

innovation64

[glossary] red-teaming-红色团队

#50 opened almost 2 years ago by

innovation64

Upload red-teaming-cn.md

#49 opened almost 2 years ago by

innovation64

New activity in innovation64/flyfood-pet-heywhale almost 2 years ago

Add `scale_factor` to vae config.

#1 opened about 2 years ago by

valhalla

New activity in HuggingFace-CN-community/translation almost 2 years ago

Upload speecht5-cn.md

#42 opened almost 2 years ago by

innovation64

New activity in HuggingFace-CN-community/translation about 2 years ago

Upload time-series-transformers-cn.md

#36 opened about 2 years ago by

innovation64

Upload time-series-transformers-cn.md

#35 opened about 2 years ago by

innovation64

[glossary] The Exploration/Exploitation trade-off - 探索利用权衡

#26 opened about 2 years ago by

innovation64

Upload lora_cn.md

#20 opened about 2 years ago by

innovation64

Upload Thinking Like Transformers-cn.md

#16 opened about 2 years ago by

innovation64

Revise the space problem about Training-SD-with-Dreambooth-cn.md

#13 opened about 2 years ago by

innovation64

[published] Upload Training-SD-with-Dreambooth-cn.md

#12 opened about 2 years ago by

innovation64

Upload From- PyTorch-DDP-to-Accelerate-to-Trainer-cn.md

#9 opened about 2 years ago by

innovation64