NLP News Cypher | 03.29.20

Author(s): Ricky Costa

Originally published on Towards AI.

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER

NLP News Cypher U+007C 03.29.20

With a Little Help From My Friends

Recorded a live feed of the (almost done) demo. I’m using 2 classifiers to classify financial news tweets for topic and sentiment and they classify in real-time U+1F440. Deploying large transformer models for online inference is hardcore!

Meanwhile, back at the ranch,

Refinitiv released a blog post showing how they trained BERT from scratch using Reuters news corpus and then fine-tuned for downstream tasks.

They were going after events in financial news, more specifically, environmental, social and governance (ESG) controversies. (From an investor perspective, they are interested in how societal events may have an impact on companies.)

Blog:

Next-level NLP and potential ESG controversies U+007C Refinitiv Perspectives

Refinitiv Labs focuses on harnessing the power of Big Data and Machine Learning (ML) to drive innovation and shape the…

www.refinitiv.com

FYI: Releasing a new update for the Big Bad NLP Database very soon!

How was your week? U+1F60E

This Week:

Software is Eating the World

Decoding Output

Deep Dive in QA BERT

Y Combinator DEMO Day

Summarization with BART

AI Adoption in the Enterprise 2020

XTREME Benchmark

Dataset of the Week: XNLI

Software is Eating the World

From Andreessen Horowitz’s blog, they give a very sobering look on the underworld of AI startups. The post highlights the difficulties in dealing with edge cases, domain shift, computational cost and other headaches. Having deployed AI models myself, this post is fairly accurate.

The New Business of AI (and How It's Different From Traditional Software)

At a technical level, artificial intelligence seems to be the future of software. AI is showing remarkable progress on…

a16z.com

Decoding Output

When you hear an ML engineer talking about beam search, temperature, or top-k sampling it could be a head-scratcher. But taking your time to understand these parameters has a lasting impact on text generation models like GPT-2! Read all about it in a new Hugging Face blog post:

How to generate text: using different decoding methods for language generation with Transformers

In recent years, there has been an increasing interest in open-ended language generation thanks to the rise of large…

huggingface.co

Deep Dive in QA BERT

McCormick strikes back with a great blog post on the inner-workings of question answering w/ BERT. Part 1 covers how BERT deals with question answering tasks and in Part 2 Chris goes over the generalization capabilities of BERT (fine-tuned on SQuAD) on data it’s never seen before:

Question Answering with a Fine-Tuned BERT

What does it mean for BERT to achieve "human-level performance on Question Answering"? Is BERT the greatest search…

mccormickml.com

Surprise, there’s a Colab notebook too:

Google Colaboratory

Edit description

colab.research.google.com

Y Combinator DEMO Day

Y Combinator Demo day event brought us several up-and-coming AI companies. This TechCrunch article covered each company and what they had to offer. This is short-term glimpse of where tech is going. Right now, there’s a lot of companies doing developer tools (spoiler alertU+1F601 ):

All the companies from Y Combinator's W20 Demo Day, Part III: Hardware, Robots, AI and Developer…

Y Combinator's Demo Day was a bit different this time around. As concerns grew over the spread of COVID-19, Y…

techcrunch.com

Summarization with BART: Colab

Peeps always want more summarization examples. Well here’s an awesome Colab notebook to do just that with the BART model!

Colab:

Google Colaboratory

Edit description

colab.research.google.com

BART Paper:

AI Adoption in the Enterprise 2020

O’Reilly, the maker of those books you love, came out with their AI Adoption survey earlier this month.

Seems institutional folks are still skeptical of AI tech:

“..lack of ML and AI skills isn’t the biggest impediment to AI adoption. Almost 22% of respondents identified a lack of institutional support as the most significant issue.”

Check out the report’s highlights here:

AI adoption in the enterprise 2020

Join Roger Magoulas on March 26 for a live and interactive online session exploring recent O'Reilly AI/ML research. Get…

www.oreilly.com

XTREME Benchmark

When possible, researchers like to transfer representational embeddings from one language to another, especially so for low-resource languages. Well, a new benchmark emerged to test the capabilities of cross-lingual generalization called XTREME! Currently, it covers multilingual representations across 40 languages and 9 tasks.

Paper:

Dataset of the Week: XNLI

What is it?

“Corpus is a crowd-sourced collection of 5,000 test and 2,500 dev pairs for the MultiNLI corpus. The pairs are annotated with textual entailment and translated into 14 languages: French, Spanish, German, Greek, Bulgarian, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, Hindi, Swahili and Urdu.”

Sample:

Where is it?

XNLI

Introduction The Cross-lingual Natural Language Inference (XNLI) corpus is a crowd-sourced collection of 5,000 test and…

www.nyu.edu

Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.

If you enjoyed this article, help us out and share with friends!

For complete coverage, follow our Twitter: @Quantum_Stat

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI