🥇Top ML Papers of the Week

The top ML Papers of the Week (Feb 6 - Feb 12)

In this issue, we cover the top ML Papers of the Week (Feb 6 - Feb 12).

---

1). Toolformer - introduces language models that teach themselves to use external tools via simple API calls. (paper)

Timo Schick @timo_schick

🎉 New paper 🎉 Introducing the Toolformer, a language model that teaches itself to use various tools in a self-supervised way. This significantly improves zero-shot performance and enables it to outperform much larger models. 🧰 🔗 Link: arxiv.org/abs/2302.04761

2). Describe, Explain, Plan, and Select - proposes using language models for open-world game playing. (paper)

Xiaojian Ma @jeasinema

Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents Can ChatGPT help with open-world game playing, like Minecraft? It sure can! 🧵👇 arxiv.org/abs/2302.01560 github.com/CraftJarvis(code will be shipped soon)

3). A Categorical Archive of ChatGPT Failures - a comprehensive analysis of ChatGPT failures for categories like reasoning, factual errors, maths, and coding. (paper)

elvis @omarsar0

A Categorical Archive of ChatGPT Failures Comprehensive analysis of ChatGPT failures for categories like reasoning, factual errors, maths, and coding. If you are developing with LLMs it's important to know these failures. Good to see them documented. arxiv.org/abs/2302.03494 https://t.co/UYmdranQvr

4). Hard Prompts Made Easy - optimizing hard text prompts through efficient gradient-based optimization. (paper)

Tom Goldstein @tomgoldsteincs

We rack our brains making prompts for #StableDiffusion and Language Models. But a lot of prompt engineering can be done *automatically* using simple gradient-based optimization. And the cold calculating efficiency of the machine crushes human creativity.

5). Data Selection for LMs - proposes a cheap and scalable data selection framework based on an importance resampling algorithm to improve the downstream performance of LMs. (paper)

Sang Michael Xie @sangmichaelxie

Data selection for LMs (GPT-3, PaLM) is done with heuristics that select data by training a classifier for high-quality text. Can we do better? Turns out we can boost downstream GLUE acc by 2+% by adapting the classic importance resampling algorithm.. arxiv.org/abs/2302.03169 🧵

6). Gen-1 - proposes an approach for structure and content-guided video synthesis with diffusion models. (paper)

Lior⚡ @AlphaSignalAI

Runway just released the paper behind their new Diffusion-based video generation tool! "Our model is trained on images+videos which exposes explicit control of temporal consistency through a novel guidance method." 📄: arxiv.org/abs/2302.03011 🛠️: research.runwayml.com/gen1 1/🧵

7). Multitask, Multilingual, Multimodal Evaluation of ChatGPT - performs a more rigorous evaluation of ChatGPt on reasoning, hallucination, and interactivity. (paper)

elvis @omarsar0

A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Another new paper doing a more rigorous evaluation of ChatGPT on reasoning, hallucination, and interactivity. arxiv.org/abs/2302.04023 https://t.co/Vo9aAzYCVT

8). Noise2Music - proposes diffusion models to generate high-quality 30-second music clips via text prompts. (paper)

AK @_akhaliq

Noise2Music: Text-conditioned Music Generation with Diffusion Models introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts abs: arxiv.org/abs/2302.03917 project page: google-research.github.io/noise2music/

9). Offsite-Tuning - introduces an efficient, privacy-preserving transfer learning framework to adapt foundational models to downstream data without access to the full model. (paper)

Aran Komatsuzaki @arankomatsuzaki

Offsite-Tuning: Transfer Learning without Full Model Achieves comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, gaining 6.5x speedup and 5.6x memory reduction. repo: github.com/mit-han-lab/of… abs: arxiv.org/abs/2302.04870

10). pix2pix-zero - proposes a model for zero-shot image-to-image translation. (paper)

Aran Komatsuzaki @arankomatsuzaki

Zero-shot Image-to-Image Translation Proposes pix2pix-zero, an image-to-image translation method that can preserve the content of the original image without manual prompting. proj: pix2pixzero.github.io abs: arxiv.org/abs/2302.03027

See you next week for another round of awesome ML papers!