The NLP Cypher | 12.06.20
Last Updated on July 24, 2023 by Editorial Team
Author(s): Ricky Costa
Originally published on Towards AI.
NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER
The NLP Cypher U+007C 12.06.20
Orion
Hey, welcome back! Plenty of NLP to discuss this week as NeurIPS takes off today. Over the last couple of days, the usual suspects opened the research paper firehose. Have a look U+1F447
Carnegie Mellon University at NeurIPS 2020
Carnegie Mellon University is proud to present 88 papers at the 34th Conference on Neural Information Processingβ¦
blog.ml.cmu.edu
OpenAI at NeurIPS 2020
Live demos and discussions at our virtual booth.
openai.com
Microsoft at NeurIPS 2020 – Microsoft Research
Microsoft is delighted to sponsor and attend the 34th Annual Conference on Neural Information Processing Systemβ¦
www.microsoft.com
Salesforce Research at NeurIPS 2020
This year marks the 34th annual conference on Neural Information Processing Systems (NeurIPS) reimagined for the firstβ¦
blog.einstein.ai
Super Duper NLP Repo U+270C
We recently made an awesome contribution to the Super Duper NLP Repo, adding 47 notebooks bringing us to 313 total! Added a decent selection of notebooks relating to adapters, the NEMO library, GEDI GPT-2, and PERIN for semantic parsing. Want to thank Abhilash Majumder & Eyal Gruss for their awesome contribution! U+1F60E
Oh, and EMNLP has yet to go away, Eric Wallace et al. released his slides from the conference on the interpretability of NLP models predictions.
ToC
(1) Overview of Interpretability
(2) What Parts of An Input Led to a Prediction?
(3) What Decision Rules Led to a Prediction?
(4) Which Training Examples Caused a Prediction?
(5) Implementing Interpretations
(6) Open Problems
Jraph U+007C DeepMindβs GNN Lib
While DeepMind isnβt solving age-old problems in protein folding, they just released a GNN library (in jax). It probably flew under everyoneβs radarβ¦
Hereβs a basic script for working with graph tuples:
deepmind/jraph
Permalink GitHub is home to over 50 million developers working together to host and review code, manage projects, andβ¦
github.com
El GitHub
deepmind/jraph
Jraph (pronounced giraffe) is a lightweight library for working with graph neural networks in jax. It provides a dataβ¦
github.com
Kaggle Data Science and ML 2020 Survey
Everyoneβs favorite data science survey was released:
TL;DR
Coursera most popular learning resource.
A lot data scientists working in small companies (less than 50 employees).
Wow, Jupyter is the go-to IDE in data science(U+1F62C).
Only 15% say transformers are the most commonly used model architecture.
AWS leads cloud, but Google comes in 2nd, (that was a surprise, I wouldβve guessed Azure).
Tensorboard more popular than I thought.
Survey
State of Data Science and Machine Learning 2020
Download our executive summary for a profile of today's working data scientist and their tools
www.kaggle.com
Data Flow
A blog from Google Cloud (with code snippets) discussing how to create data pipelines for your ML models. It focuses on batching, the singleton model pattern, and dealing with threading/processing. A helpful read for those deploying in the enterprise.
ML inference in Dataflow pipelines U+007C Google Cloud Blog
In this blog, we covered some of the patterns for running remote/local inference calls, including; batching, theβ¦
cloud.google.com
MSFP U+007C Data Type for Efficient Inference
Microsoft invented a new data type used in data representation with a focus on improved latency during model inference called⦠MSFP.
[MSFP] enables dot product operations β the core of the matrix-matrix and matrix-vector multiplication operators critical to DNN inference β to be performed nearly as efficiently as with integer data types, but with accuracy comparable to floating point.
Apparently MS uses MSFP in Project Brainwave, their real-time production-scale DNN inference in the cloud. As models get bigger, big tech is getting smarter on how to deal with scale and inference in production.
A Microsoft custom data type for efficient inference – Microsoft Research
AI is taking on an increasingly important role in many Microsoft products, such as Bing and Office 365. In some casesβ¦
www.microsoft.com
Recommenders Update
When we first spoke about TensorFlowβs Recommenders library several newsletters ago, I was really excited but TF has upped the ante by building deep learning recommender models βthat can retrieve the best candidates out of millions in milliseconds.β U+1F440
It uses Googleβs ScaNN library released this past summer, you can check out the repo here: https://github.com/google-research/google-research/tree/master/scann
The second part of their update is their leveraging of DCN (Deep cross networks) models.
TensorFlow Recommenders: Scalable retrieval and feature interaction modelling
November 30, 2020 – Posted by Ruoxi Wang, Phil Sun, Rakesh Shivanna and Maciej Kula (Google) In September, weβ¦
blog.tensorflow.org
Repo Cypher U+1F468βU+1F4BB
A collection of repos/papers that caught our U+1F441
DframCy
DframCy provides clean APIs to convert spaCyβs linguistic annotations, Matcher and PhraseMatcher information to Pandas dataframe.
yash1994/dframcy
DframCy is a light-weight utility module to integrate Pandas Dataframe to spaCy's linguistic annotation and trainingβ¦
github.com
Wolframβs Model Stash
Wolfram has his own Deep Learning model hub. Just stumbled upon this one when I saw one of Wolframβs tweets earlier this week. U+1F648
Wolfram Neural Net Repository
The Wolfram Neural Net Repository is a public resource that hosts an expanding collection of trained and untrainedβ¦
resources.wolframcloud.com
Novel2Graph
The algorithm receives a book and it discovers main characters and main relations between characters.
Oldie but goodie.
IDSIA/novel2graph
The algorithm receives a book and it discovers main characters, main relations between characters and more powerfulβ¦
github.com
EDGEBert
New research paper on the improvement of memory and latency w/r/t BERT inference that utilizes several techniques in compression and model architecture. The authors boast of βachieving up to 2.4Γ and 13.4Γ inference latency and memory savings, respectively, with less than 1%-pt. drop in accuracy.β U+1F440
Paper: https://arxiv.org/pdf/2011.14203.pdf
OCR and Deep Learning
Couple of weeks ago on LinkedIn I posted a question regarding current OCR techniques that led to a great discussion with my connections. This week, I found this U+1F447. WINNING!
Paper: https://arxiv.org/pdf/2011.13534.pdf
Long Text Classification with BERT
Looking to classify text documents with more than 250 words per doc?
Notebook (U+1F525)
ArmandDS/bert_for_long_text
Permalink GitHub is home to over 50 million developers working together to host and review code, manage projects, andβ¦
github.com
Blog
Using BERT For Classifying Documents with Long Texts
How to ο¬ne-tuning Bert for inputs longer than a few words or sentences
medium.com
Dataset of the Week: XED
What is it?
A multi-lingual dataset consisting of emotion annotated movie subtitles from OPUS used for sentiment analysis. The task is formulated as multi-label classification.
Where is it?
Helsinki-NLP/XED
This is the XED dataset. The dataset consists of emotion annotated movie subtitles from OPUS. We use Plutchik's 8 coreβ¦
github.com
Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.
For complete coverage, follow our Twitter: @Quantum_Stat
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming aΒ sponsor.
Published via Towards AI
Related posts
Popular posts
Updates
Recent Posts
