NLP News Cypher | 03.08.20
Last Updated on July 24, 2023 by Editorial Team
Author(s): Ricky Costa
Originally published on Towards AI.
NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER
NLP News Cypher U+007C 03.08.20
It Rages that Way
Onward, we go. In last weekβs column, I posed a problem w/r/t the problem of complexity in sentiment analysis. In our example:
Gold is up 6% in the pre-market as the downward pressure of the coronavirus outbreak weighs on equity stocks.
The consequence of when dealing with complex systems, especially in dealing with this example, is that a generalized approach is difficult. Very difficult to reduce complexity down to a vector. But if we go bottom-up, if we instead localize it, to the client, we see light at the end of the tunnel. In order to know the ground-truth we must localize sentiment to the holdings of the user. If the user holds Gold, then itβs Bullish, If the client is betting against Gold, then itβs Bearish, if there is no position itβs a neutral statement. In other words, for this example, personalization isnβt just a marketing gimmick, but itβs a functional requirement. There is no perfect solution and every unique domain will require itβs own local rules for ground-truth interpretation.
Albeit, this statement was one of the most difficult to analyze and is usually an edge case. But with deep learning, outliers in datasets are how models get smoked.
(There are other complex bottlenecks that we may encounter, n-order logic, ambiguous ground-truth, domain-shift etc. I will discuss these and other factors in an upcoming white paper. Stay tuned!)
How was your week?
BTW, we updated the BBN database, thank you to all contributors!
This Week:
Walking Wikipedia
Learning From Unlabeled Data
A Jiant Among Us
Hugging Faceβs Notebooks
Composing Semantics
Transformer Size, Training and Compression
Graphing Knowledge
Dataset of the Week: DVQA
Walking Wikipedia
The ongoing research in localizing graph structures with transformers continues to march forward. Research shows how a new model is able to follow an English Wikipedia reasoning path to answer multi-hop questions as found in HotpotQA. This is meant for the open domain scale.
(Last weekβs column had a similar paper, looks like open-domain, multi-hop is really gaining steam among researchers.U+2728U+1F60E)
GitHub:
AkariAsai/learning_to_retrieve_reasoning_paths
This is the official implementation of the following paper: Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richardβ¦
github.com
Paper:
Learning From Unlabeled Data
Slides from Thangβs talk, sometime during February 2020 (Thang was an author on the recent Meena chatbot paper/model from Google). He goes over the importance of self-supervised learning that has revolutionized NLP and how itβsβ spreading to Computer Vision. Towards the end he discusses the Meena paper:
ThangLuong-talk-Feb-2020.pdf
Edit description
drive.google.com
A Jiant Among Us
We have a new NLU framework to play with! The great thing about frameworks is that it allows you to scale experiments, and itβs the reason why Jiant leverages the use of config files. And yes, it was built on top of PyTorch.
Blog:
The jiant toolkit for general-purpose text understanding models
jiant is a work-in-progress software toolkit for natural language processing research, designed to facilitate work onβ¦
jiant.info
GitHub:
nyu-mll/jiant
jiant is a software toolkit for natural language processing research, designed to facilitate work on multitask learningβ¦
github.com
Hugging Faceβs Notebooks
Several days ago, U+1F917 revealed 4 Colab notebooks to help get your NLP pipeline jump-started with their library. Keep an eye on this GitHub page as they are looking for the community to contribute.
huggingface/transformers
You can find here a list of the official notebooks provided by Hugging Face. Also, we would like to list hereβ¦
github.com
Composing Semantics
Google released a new dataset called Compositional Freebase Questions (CFQ) and a new benchmark for measuring how well a program generalizes compositionality. If you think semantic parsing has been over-looked, you should check this out:
Measuring Compositional Generalization
People are capable of learning the meaning of a new word and then applying it to other language contexts. As Lake andβ¦
ai.googleblog.com
Transformer Size, Training and Compression
So increasing the size of the model improves training/inference speed? Counter-intuitive right? New research out of Berkeley highlights an interesting trade-off. It shows that training very large models and cutting them off early is a lot better than using smaller models that train for more epochs. In addition, you get more of a payoff when compressing very large models than marginally compressing smaller ones.
Blog:
Speeding Up Transformer Training and Inference By Increasing Model Size
In deep learning, using more compute (e.g., increasing model size, dataset size, or training steps) often leads toβ¦
bair.berkeley.edu
Graphing Knowledge
If you want to learn about all things knowledge graphs, a comprehensive paper was released this past weekU+1F440 :
Dataset of the Week: DVQA
What is it?
The DVQA dataset converts bar chart understanding into a question answering framework.
Sample:
Where is it?
kushalkafle/DVQA_dataset
This repository provides the images, metadata and question-answer pairs described in the paper: DVQA: Understandingβ¦
github.com
Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.
If you enjoyed this article, help us out and share with friends!
For complete coverage, follow our Twitter: @Quantum_Stat
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming aΒ sponsor.
Published via Towards AI