🥇Top ML Papers of the Week

The top ML Papers of the Week (Jan 23-29)

elvis

Jan 29, 2023

In this edition of the NLP Newsletter, we cover the top ML Papers of the Week (Jan 23-29).

1) MusicLM - a generative model for generating high-fidelity music from text descriptions. (Paper | Tweet)

AK @_akhaliq

MusicLM: Generating Music From Text abs: arxiv.org/abs/2301.11325 project page: google-research.github.io/seanet/musiclm…

2) H3 - an approach to reduce the gap, in terms of performance and hardware utilization, between state space models and attention for language modeling. (Paper | Tweet)

Dan Fu @realDanFu

Attention is all you need... but how much of it do you need? Announcing H3 - a new generative language models that outperforms GPT-Neo-2.7B with only *2* attention layers! Accepted as a *spotlight* at #ICLR2023! 📣 w/ @tri_dao 📜 arxiv.org/abs/2212.14052 1/n

3) A Watermark for LLMs - a watermarking framework for proprietary language models. (Paper | Tweet)

Tom Goldstein @tomgoldsteincs

#OpenAI is planning to stop #ChatGPT users from making social media bots and cheating on homework by "watermarking" outputs. How well could this really work? Here's just 23 words from a 1.3B parameter watermarked LLM. We detected it with 99.999999999994% confidence. Here's how 🧵

4) Make-A-Video3D - a new text-to-4D model for dynamic scene generation from input text. (Paper | Tweet | Project)

Devi Parikh @deviparikh

Introducing Make-A-Video3D! Generating 3D dynamic (mini) scenes from input text. That is, text --> 4D! Needs no 4D data (i.e., no dynamic 3D data), no static 3D data, no paired text-video data. Paper: arxiv.org/abs/2301.11280 Website: make-a-video3d.github.io

5). ClimaX - a foundation model for weather and climate, including many capabilities for atmospheric science tasks. (Paper | Tweet | Blog)

Tung Nguyen @tungnd_13

Introducing ClimaX, the first foundation model for weather and climate. A fast and accurate one-stop AI solution for a range of atmospheric science tasks. Paper: arxiv.org/abs/2301.10343 Blog: microsoft.com/en-us/research… Thread🧵 #ML #Climate #Weather #FoundationModel

6) Open Problems in Applied Deep Learning - a new reference to learn about interesting open problems in deep learning. (Paper | Tweet)

elvis @omarsar0

Open Problems in Applied Deep Learning If you're looking for interesting open problems in DL, this is a good reference. Not sure if intentional but it also looks useful to get a general picture of current trends in deep learning with ~300 references. arxiv.org/abs/2301.11316 https://t.co/XGqIo9Hjnk

7) DetectGPT - an approach for zero-shot machine-generated text detection. Uses raw log probabilities from the LLM to determine if the passage was sampled from it. (Paper | Tweet)