🥇Top ML Papers of the Week

The top ML Papers of the Week (April 3 - April 9)

1). Segment Anything Model - presents a set of resources to establish foundational models for image segmentation; releases the largest segmentation dataset with over 1 billion masks on 11M licensed images; the model’s zero-shot performance is competitive with or even superior to fully supervised results. (paper)

Meta AI @MetaAI

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️ bit.ly/433YuBI

---

2). Instruction Tuning with GPT-4 - presents GPT-4-LLM, a "first attempt" to use GPT-4 to generate instruction-following data for LLM fine-tuning; the dataset is released and includes 52K unique English and Chinese instruction-following data; the dataset is used to instruction-tune LLaMA models which leads to superior zero-shot performance on new tasks. (paper)

elvis @omarsar0

Okay, this is awesome! Instruction Tuning with GPT-4! This paper presents GPT-4-LLM, a "first attempt" to use GPT-4 to generate instruction-following data for LLM fine-tuning. The dataset is released and includes 52K unique English and Chinese instruction-following data.… https://t.co/OnFyePSrqR

---

3). 8 Things to Know about LLMs - discusses important considerations regarding the capabilities and limitations of LLMs. (paper)

Sam Bowman @sleepinyourhat

I’m sharing a draft of a slightly-opinionated survey paper I’ve been working on for the last couple of months. It's meant for a broad audience—not just LLM researchers. (🧵)

---

4). A Survey of LLMs - a new 50 pages survey on large language models. (paper)

elvis @omarsar0

A Survey of LLMs A new 50 pages survey on large language models just dropped on arXiv. arxiv.org/abs/2303.18223

---

5). Baize - an open-source chat model fine-tuned with LoRA. Leverages 100K dialogs generated from ChatGPT chatting with itself; it releases the dialogs along with 7B, 13B, and 30B parameter models. (paper)

Canwen Xu @XuCanwen

We are announcing Baize, an open-source chat model trained with ChatGPT self-chat data. We are releasing 150k high-quality dialogs with 7B, 13B and 30B models. ArXiv: arxiv.org/abs/2304.01196 Demo: huggingface.co/spaces/project… Github: github.com/project-baize/…

---

6). MACHIAVELLI - a new benchmark of 134 text-based Choose-Your-Own-Adventure games to evaluate the capabilities and unethical behaviors of LLMs. (paper)

Dan Hendrycks @DanHendrycks

Do models like GPT-4 behave safely when given the ability to act? We develop the Machiavelli benchmark to measure deception, power-seeking tendencies, and other unethical behaviors in complex interactive environments that simulate the real world. Paper: arxiv.org/abs/2304.03279

---

7). Better Language Models of Code through Self-Improvement - generates pseudo data from knowledge gained through pre-training and fine-tuning; adds the data to the training dataset for the next step; results show that different frameworks can be improved in performance using code-related generation tasks. (paper)

John Nay @johnjnay

Simple Self-Improvement of Code LLMs 1) Pre-train & Fine-tune code LLM, gaining knowledge 2) LLM then generates pseudo outputs 3) Add that to original data & train for next epoch Significantly improves code summarization & code generation performance arxiv.org/abs/2304.01228

---

8). Summary of ChatGPT/GPT-4 Research - an overview of applications of ChatGPT and GPT-4; the analysis is done on 194 relevant papers and discusses capabilities, limitations, concerns, and more. (paper)

elvis @omarsar0

Good overview of applications of ChatGPT and GPT-4. The analysis is done on 194 relevant papers and discusses capabilities, limitations, concerns, and more. A great read for AI developers and researchers. arxiv.org/abs/2304.01852

---

9). Pythia - a suite for analyzing LLMs across training and scaling; includes 16 LLMs trained on public data and ranging in size from 70M to 12B parameters. (paper)

Stella Rose Biderman @BlancheMinerva

Have you ever wanted to do an experiment on LLMs and found that none of the existing model suites met your needs? At @AiEleuther we got tired of this happening and so designed a model suite that centers enabling scientific research as its primary goal arxiv.org/abs/2304.01373

---

10). SegGPT - unifies segmentation tasks into a generalist model through an in-context framework that supports different kinds of data. (paper)

AK @_akhaliq

SegGPT: Segmenting Everything In Context abs: arxiv.org/abs/2304.03284 github: github.com/baaivision/Pai… @Gradio demo: dev.ssi.plus:43533