NLP News Cypher | 03.08.20

Author(s): Ricky Costa

Originally published on Towards AI.

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER

NLP News Cypher U+007C 03.08.20

It Rages that Way

Onward, we go. In last week’s column, I posed a problem w/r/t the problem of complexity in sentiment analysis. In our example:

Gold is up 6% in the pre-market as the downward pressure of the coronavirus outbreak weighs on equity stocks.

The consequence of when dealing with complex systems, especially in dealing with this example, is that a generalized approach is difficult. Very difficult to reduce complexity down to a vector. But if we go bottom-up, if we instead localize it, to the client, we see light at the end of the tunnel. In order to know the ground-truth we must localize sentiment to the holdings of the user. If the user holds Gold, then it’s Bullish, If the client is betting against Gold, then it’s Bearish, if there is no position it’s a neutral statement. In other words, for this example, personalization isn’t just a marketing gimmick, but it’s a functional requirement. There is no perfect solution and every unique domain will require it’s own local rules for ground-truth interpretation.

Albeit, this statement was one of the most difficult to analyze and is usually an edge case. But with deep learning, outliers in datasets are how models get smoked.

(There are other complex bottlenecks that we may encounter, n-order logic, ambiguous ground-truth, domain-shift etc. I will discuss these and other factors in an upcoming white paper. Stay tuned!)

How was your week?

BTW, we updated the BBN database, thank you to all contributors!

This Week:

Walking Wikipedia

Learning From Unlabeled Data

A Jiant Among Us

Hugging Face’s Notebooks

Composing Semantics

Transformer Size, Training and Compression

Graphing Knowledge

Dataset of the Week: DVQA

Walking Wikipedia

The ongoing research in localizing graph structures with transformers continues to march forward. Research shows how a new model is able to follow an English Wikipedia reasoning path to answer multi-hop questions as found in HotpotQA. This is meant for the open domain scale.

(Last week’s column had a similar paper, looks like open-domain, multi-hop is really gaining steam among researchers.U+2728U+1F60E)

GitHub:

AkariAsai/learning_to_retrieve_reasoning_paths

This is the official implementation of the following paper: Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard…

github.com

Paper:

Learning From Unlabeled Data

Slides from Thang’s talk, sometime during February 2020 (Thang was an author on the recent Meena chatbot paper/model from Google). He goes over the importance of self-supervised learning that has revolutionized NLP and how it‘s’ spreading to Computer Vision. Towards the end he discusses the Meena paper:

ThangLuong-talk-Feb-2020.pdf

Edit description

drive.google.com

A Jiant Among Us

We have a new NLU framework to play with! The great thing about frameworks is that it allows you to scale experiments, and it’s the reason why Jiant leverages the use of config files. And yes, it was built on top of PyTorch.

Blog:

The jiant toolkit for general-purpose text understanding models

jiant is a work-in-progress software toolkit for natural language processing research, designed to facilitate work on…

jiant.info

GitHub:

nyu-mll/jiant

jiant is a software toolkit for natural language processing research, designed to facilitate work on multitask learning…

github.com

Hugging Face’s Notebooks

Several days ago, U+1F917 revealed 4 Colab notebooks to help get your NLP pipeline jump-started with their library. Keep an eye on this GitHub page as they are looking for the community to contribute.

huggingface/transformers

You can find here a list of the official notebooks provided by Hugging Face. Also, we would like to list here…

github.com

Composing Semantics

Google released a new dataset called Compositional Freebase Questions (CFQ) and a new benchmark for measuring how well a program generalizes compositionality. If you think semantic parsing has been over-looked, you should check this out:

Measuring Compositional Generalization

People are capable of learning the meaning of a new word and then applying it to other language contexts. As Lake and…

ai.googleblog.com

Transformer Size, Training and Compression

So increasing the size of the model improves training/inference speed? Counter-intuitive right? New research out of Berkeley highlights an interesting trade-off. It shows that training very large models and cutting them off early is a lot better than using smaller models that train for more epochs. In addition, you get more of a payoff when compressing very large models than marginally compressing smaller ones.

Blog:

Speeding Up Transformer Training and Inference By Increasing Model Size

In deep learning, using more compute (e.g., increasing model size, dataset size, or training steps) often leads to…

bair.berkeley.edu

Graphing Knowledge

If you want to learn about all things knowledge graphs, a comprehensive paper was released this past weekU+1F440 :

Dataset of the Week: DVQA

What is it?

The DVQA dataset converts bar chart understanding into a question answering framework.

Sample:

Where is it?

kushalkafle/DVQA_dataset

This repository provides the images, metadata and question-answer pairs described in the paper: DVQA: Understanding…

github.com

Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.

If you enjoyed this article, help us out and share with friends!

For complete coverage, follow our Twitter: @Quantum_Stat

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI