🥇Top ML Papers of the Week

The top ML Papers of the Week (Mar 27 - April 2)

1). BloombergGPT - a new 50B parameter large language model for finance. Claims the largest domain-specific dataset yet with 363 billion tokens... further augmented with 345 billion tokens from general-purpose datasets; outperforms existing models on financial tasks while not sacrificing performance on general LLM benchmarks. (paper)

elvis @omarsar0

BloombergGPT is a new LLM for finance. It's a 50 billion parameter language model trained on financial data. Claims the largest domain-specific dataset yet with 363 billion tokens... further augmented with 345 billion tokens from general purpose arxiv.org/abs/2303.17564…… https://t.co/s6AfoEPWCx

---

2). ALOHA - a low-cost system that performs end-to-end imitation learning from real demonstrations; also presents an algorithm called Action Chunking with Transformers to learn a generative model that allows a robot to learn difficult tasks in the real world. (paper | code)

Tony Z. Zhao @tonyzzhao

Introducing ALOHA 🏖: 𝐀 𝐋ow-cost 𝐎pen-source 𝐇𝐀rdware System for Bimanual Teleoperation After 8 months iterating @Stanford and 2 months working with beta users, we are finally ready to release it! Here is what ALOHA is capable of:

---

3). HuggingGPT - a system that leverages LLMs like ChatGPT to conduct task planning, select models and act as a controller to execute subtasks and summarize responses according to execution results. (paper)

John Nay @johnjnay

HuggingGPT -Human requests something -ChatGPT 1 Plans tasks 2 Selects AI models based on HuggingFace descriptions 3 Manages cooperation of expert models to execute subtasks 4 Summarizes results Covers many sophisticated tasks across modalities & domains arxiv.org/abs/2303.17580

---

4). ChatDoctor - a medical chat model fine-tuned on LLaMA using medical domain knowledge. Collects data on around 700 diseases and generated 5K doctor-patient conversations to finetune the LLM. (paper | code)

elvis @omarsar0

ChatDoctor: A medical chat model fine-tuned on LLaMA using medical domain knowledge. Collects data on around 700 diseases and generated 5K doctor-patient conversations to finetune the LLM. paper: arxiv.org/abs/2303.14070 code: github.com/Kent0n-Li/Chat… https://t.co/PdBl3fiijv

---

5). LLaMA-Adapter - a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model; generates responses comparable to Alpaca with fully fine-tuned 7B parameter; it’s also extended for multi-modal input support. (paper | code)

Sebastian Raschka @rasbt

LLaMA-Adapter: finetuning large language models (LLMs) like LLaMA and matching Alpaca's modeling performance with greater finetuning efficiency Let's have a look at this new paper (arxiv.org/abs/2303.16199) that proposes an adapter method for LLaMA instruction finetuning 1/5

---

6). ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks - demonstrates that ChatGPT can outperform crowd-workers for several annotation tasks such as relevance, topics, and frames detection; besides better zero-shot accuracy, the per-annotation cost of ChatGPT is less 20 times cheaper than MTurk. (paper)

Lior⚡ @AlphaSignalAI

This is big news, ChatGPT just outperformed mechanical turk workers on text annotation tasks! We're getting closer to complete AI-based data annotation, which in turn, can be used to train AI models. It will cause a big shift in the industry. Paper: arxiv.org/pdf/2303.15056…

---

7). LLMs for Computer Tasks - shows that a pre-trained LLM agent can execute computer tasks using a simple prompting scheme where the agent recursively criticizes and improves its outputs. (paper)

Aran Komatsuzaki @arankomatsuzaki

Language Models can Solve Computer Tasks Letting LLM to recursively criticize and improve its output significantly outperforms existing LLM methods on computer tasks and surpasses supervised learning (SL) and RL approaches. arxiv.org/abs/2303.17491

---

8). Dialog-Enabled Resolving Agents (DERA) - a paradigm to enhance large language model completions by allowing models to communicate feedback and iteratively improve output; DERA outperforms base GPT-4 on clinically-focused tasks. (paper)

John Nay @johnjnay

Forums for LLM Agents to Communicate Can Improve Outputs 1) Human provides task 2) "Decider" Agent produces output 3) "Researcher" & Decider Agents discuss 4) Decider decides Big improvement over base GPT4 on medical summarization & care plan generation arxiv.org/abs/2303.17071

---

9). Natural Selection Favors AIs over Humans - discusses why AI systems will become more fit than humans and the potential dangers and risks involved, including ways to mitigate them. (paper)

Dan Hendrycks @DanHendrycks

As AI systems become more useful, people will delegate greater authority to them across more tasks. AIs are evolving in an increasingly frenzied and uncontrolled manner. This carries risks as natural selection favors AIs over humans. Paper: arxiv.org/abs/2303.16200 (🧵 below)

---

10). ML for Partial Differential Equations - a review examining avenues of partial differential equations research advanced by machine learning. (paper)

DynamicalSystemsSIAM @DynamicsSIAM

"Machine Learning for Partial Differential Equations" (by Steven L. Brunton, J. Nathan Kutz): arxiv.org/abs/2303.17078

---