NLP Newsletter

Share this post

🥇Top ML Papers of the Week

nlp.elvissaravia.com

Discover more from NLP Newsletter

The NLP newsletter covers the latest trending natural language processing (NLP) and machine learning (ML) news, projects, resources, and research papers.
Over 10,000 subscribers
Continue reading
Sign in

🥇Top ML Papers of the Week

The top ML Papers of the Week (June 19 - 25)

elvis
Jun 25, 2023
11
Share this post

🥇Top ML Papers of the Week

nlp.elvissaravia.com
Share

1). Textbooks Are All You Need - introduces a new 1.3B parameter LLM called phi-1; it’s significantly smaller in size and trained for 4 days using a selection of textbook-quality data and synthetic textbooks and exercises with GPT-3.5; achieves promising results on the HumanEval benchmark. (paper | tweet)


2). RoboCat - a new foundation agent that can operate different robotic arms and can solve tasks from as few as 100 demonstrations; the self-improving AI agent can self-generate new training data to improve its technique and get more efficient at adapting to new tasks. (paper | tweet)


3). ClinicalGPT - a language model optimized through extensive and diverse medical data, including medical records, domain-specific knowledge, and multi-round dialogue consultations. (paper | tweet)


4). An Overview of Catastrophic AI Risks - provides an overview of the main sources of catastrophic AI risks; the goal is to foster more understanding of these risks and ensure AI systems are developed in a safe manner. (paper | tweet)


5). LOMO - proposes a new memory-efficient optimizer that combines gradient computation and parameter update in one step; enables tuning the full parameters of an LLM with limited resources. (paper | tweet)


6). SequenceMatch - formulates sequence generation as an imitation learning problem; this framework allows the ability to incorporate backtracking into text generation through a backspace action; this enables the generative model to mitigate compounding errors by reverting sample tokens that lead to sequence OOD. (paper | tweet)


7). LMFlow - an extensible and lightweight toolkit that simplifies finetuning and inference of general large foundation models; supports continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, and large model inference. (paper | tweet)


8). MotionGPT - uses multimodal control signals for generating consecutive human motions; it quantizes multimodal control signals intro discrete codes which are converted to LLM instructions that generate motion answers. (paper | tweet)


9). Wanda - introduces a simple and effective pruning approach for LLMs; it prunes weights with the smallest magnitudes multiplied by the corresponding input activations, on a per-output basis; the approach requires no retraining or weight update and outperforms baselines of magnitude pruning. (paper | tweet)


10). AudioPaLM - fuses text-based and speech-based LMs, PaLM-2 and AudioLM, into a multimodal architecture that supports speech understanding and generation; outperforms existing systems for speech translation tasks with zero-shot speech-to-text translation capabilities. (paper | tweet)


Reach out to ellfae@gmail.com if you want to sponsor the next issue of the newsletter.

11
Share this post

🥇Top ML Papers of the Week

nlp.elvissaravia.com
Share
Comments
Top
New
Community

No posts

Ready for more?

© 2023 elvis
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing