NLP News Cypher | 03.29.20

Author(s): Ricky Costa

Originally published on Towards AI.

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER

NLP News Cypher U+007C 03.29.20

With a Little Help From My Friends

Recorded a live feed of the (almost done) demo. I’m using 2 classifiers to classify financial news tweets for topic and sentiment and they classify in real-time U+1F440. Deploying large transformer models for online inference is hardcore!

Meanwhile, back at the ranch,

Refinitiv released a blog post showing how they trained BERT from scratch using Reuters news corpus and then fine-tuned for downstream tasks.

They were going after events in financial news, more specifically, environmental, social and governance (ESG) controversies. (From an investor perspective, they are interested in how societal events may have an impact on companies.)

Blog:

FYI: Releasing a new update for the Big Bad NLP Database very soon!

How was your week? U+1F60E

This Week:

Software is Eating the World

Decoding Output

Deep Dive in QA BERT

Y Combinator DEMO Day

Summarization with BART

AI Adoption in the Enterprise 2020

XTREME Benchmark

Dataset of the Week: XNLI

Software is Eating the World

From Andreessen Horowitz’s blog, they give a very sobering look on the underworld of AI startups. The post highlights the difficulties in dealing with edge cases, domain shift, computational cost and other headaches. Having deployed AI models myself, this post is fairly accurate.

Decoding Output

When you hear an ML engineer talking about beam search, temperature, or top-k sampling it could be a head-scratcher. But taking your time to understand these parameters has a lasting impact on text generation models like GPT-2! Read all about it in a new Hugging Face blog post:

Deep Dive in QA BERT

McCormick strikes back with a great blog post on the inner-workings of question answering w/ BERT. Part 1 covers how BERT deals with question answering tasks and in Part 2 Chris goes over the generalization capabilities of BERT (fine-tuned on SQuAD) on data it’s never seen before:

Surprise, there’s a Colab notebook too:

Y Combinator DEMO Day

Y Combinator Demo day event brought us several up-and-coming AI companies. This TechCrunch article covered each company and what they had to offer. This is short-term glimpse of where tech is going. Right now, there’s a lot of companies doing developer tools (spoiler alertU+1F601 ):

Summarization with BART: Colab

Peeps always want more summarization examples. Well here’s an awesome Colab notebook to do just that with the BART model!

Colab:

BART Paper:

AI Adoption in the Enterprise 2020

O’Reilly, the maker of those books you love, came out with their AI Adoption survey earlier this month.

Seems institutional folks are still skeptical of AI tech:

“..lack of ML and AI skills isn’t the biggest impediment to AI adoption. Almost 22% of respondents identified a lack of institutional support as the most significant issue.”

Check out the report’s highlights here:

XTREME Benchmark

When possible, researchers like to transfer representational embeddings from one language to another, especially so for low-resource languages. Well, a new benchmark emerged to test the capabilities of cross-lingual generalization called XTREME! Currently, it covers multilingual representations across 40 languages and 9 tasks.

Paper:

Dataset of the Week: XNLI

What is it?

“Corpus is a crowd-sourced collection of 5,000 test and 2,500 dev pairs for the MultiNLI corpus. The pairs are annotated with textual entailment and translated into 14 languages: French, Spanish, German, Greek, Bulgarian, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, Hindi, Swahili and Urdu.”

Sample:

Where is it?

Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.

If you enjoyed this article, help us out and share with friends!

For complete coverage, follow our Twitter: @Quantum_Stat

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI