NLP News Cypher | 03.15.20

Author(s): Ricky Costa

Originally published on Towards AI.

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER

NLP News Cypher U+007C 03.15.20

Keep it Movin’

How was your week? U+1F637

Same here.

In light of recent events, we released a COVID tracker to stay-up-date on the latest COVID news from sources across the 50 US states. We have linked local and national news sources in addition to the Health Departments and other authorities from each state. We are using 3 different APIs: one from Datawrapper’s server that connects to John Hopkins COVID data, Feedly’s news API, and Twitter’s streaming API. U+1F60B

Check it out:

US COVID Tracker – Quantum Stat

Currently, the COVID-19 pandemic is spreading across the United States. Under these conditions, one must know how to…

covid.quantumstat.com

With regards to the Big Bad NLP Database, we also added research papers to ~90% of the database! Special thank you to our researcher Gabi Alexandru for doing an amazing job. U+1F60E

The Big Bad NLP Database – Quantum Stat – Quantum Stat

Datasets for various tasks in Natural Language Processing – Quantum Stat

datasets.quantumstat.com

FYI, stay indoors!

This week, the newsletter will be shorter than usual given the slow news cycle, I’m assuming it’s related to the current pandemic U+1F60C.

This Week:

TensorFlow Quantum

Haste

Electra Feel

Hugging Papers

Dataset of the Week: Jeopardy Questions

TensorFlow Quantum

Google introduced an open-source library for the rapid prototyping of quantum ML models!

In order to understand quantum models, you need to familiarize yourself with two concepts : quantum data and hybrid quantum-classical models (current approach).

Quantum Data: (which can be generated) can be used for the simulation of chemicals and quantum matter, quantum control, quantum communication networks, quantum metrology, and much more.

Hybrid quantum-classical models: OK spoiler alert, these quantum models are not YET using quantum powered hardware (still too noisy), so we are left with using GPUs. So that’s why they are “hybrid”.

The good thing about this library is: if we can get used to these models now, by the time the processors are ready for prime-time, we will be able to crunch HUMONGOUS amount of data using quantum principles. But first, we need to dip our feet with the quantum framework — this is what Google is doing for us with this library.

Blog

TensorFlow Quantum

TensorFlow Quantum is a library for hybrid quantum-classical machine learning. TensorFlow Quantum (TFQ) is a quantum…

www.tensorflow.org

Paper:

Haste

Hey remember RNNs? U+1F923. So even though we all want to marry transformers for life, RNNs are still very useful. Why? Because most companies that use AI models are still using RNNs for sequential NLP data. (It takes a couple of years for them to catch up to transformers).

Props to Mr. Nanavat for creating the RNN library called Haste:

lmnt-com/haste

Haste is a CUDA implementation of fused LSTM, Layer Normalized LSTM, and GRU layers with built-in DropConnect and…

github.com

Electra Feel

The ELECTRA transformer is pretty cool. Why? Because they changed the representation model. Instead of masking during pre-training, they have turned to substituting words with fake words and making the model choose the right one. It’s like GANs but for NLP!

This new pre-training technique allows ELECTRA to outperform current NLP transformers given the same compute during training! (SOTA on SQuAD v2)

“We compare ELECTRA against other state-of-the-art NLP models and found that it substantially improves over previous methods, given the same compute budget, performing comparably to RoBERTa and XLNet while using less than 25% of the compute.”

In terms of downstream tasks, ELECTRA supports text classification, question answering and sequence tagging.

More Efficient NLP Model Pre-training with ELECTRA

Recent advances in language pre-training have led to substantial gains in the field of natural language processing…

ai.googleblog.com

GitHub:

google-research/electra

ELECTRA is a new method for self-supervised language representation learning. It can be used to pre-train transformer…

github.com

Hugging Papers

Hugging Face shared their favorite NLP research papers with their community. In the link below, you will find their fav research papers and also a schedule for future papers to be discussed. They are pulling us into the matrix! Take the blue pill!

huggingface/awesome-papers

Each week, the Hugging Face team has a science day where one team member presents an awesome NLP paper. We've decided…

github.com

Dataset of the Week: Jeopardy Questions

What is it?

A dataset containing 216,930 Jeopardy questions & answers used for, you guessed it, question answering.

Sample:

Here’s the metadata descriptors:

• ‘category’ : the question category, e.g. “HISTORY”
• ‘value’ : $ value of the question as string, e.g. “$200”
• Note: This is “None” for Final Jeopardy! and Tiebreaker questions
• ‘question’ : text of question
• Note: This sometimes contains hyperlinks and other things messy text such as when there’s a picture or video question
• ‘answer’ : text of answer
• ‘round’ : one of “Jeopardy!”,”Double Jeopardy!”,”Final Jeopardy!” or “Tiebreaker”
• Note: Tiebreaker questions do happen but they’re very rare (like once every 20 years)
• ‘show_number’ : string of show number, e.g ‘4680’
• ‘air_date’ : the show air date in format YYYY-MM-DD

Where is it?

Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.

If you enjoyed this article, help us out and share with friends!

For complete coverage, follow our Twitter: @Quantum_Stat

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI