NLP News Cypher | 07.26.20
Last Updated on January 6, 2023 by Editorial Team
Author(s): Quantum Stat

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER
Primus
The Liber Primus is unsolved to this day. A book of 58 pages written in Runes, of which, its bewildering encryption continues to haunt hacker gunslingers around the globe who choose only to communicate and study its content via IRCs (internet chatย relays).
The cryptic book arrived on the internet in the mid 2010โs by the now wildly popular but mysterious internet group 3301. While the groupโs identity remains hidden, it is speculated they are a remnant of the cypherpunk activist movement (birthed somewhere out of Berkley in the 80s). At least this is the most plausible explanation given to us by one of the few known hackers thatโs made it inside the clandestine groupโโโMarcus Wanner. But whoย knowsโฆ
3301โs Cicada project started with a random 4chan post in 2012 leading many thrill seekers, with a cult-like following, on a puzzle hunt that encompassed everything from steganography to cryptography. While most of their puzzles were eventually solved, the very last one, the Liber Primus, is still (mostly) encrypted. The last known comms from 3301 came in April 2017 via Pastebin post. Itย reads:
Message from 3301/Cicada – Pastebin.com
FYI, thereโs a standard PGP (pretty good privacy) key for all 3301 posts. If you see a 3301 online post without their PGP signature, donโt trust it (plenty of troll accounts to beย found).
For a Summary/Timeline:
Visit Noxโs YouTube channel if you are interested in understanding how they cracked previous Cicada puzzles ante-Liber Primus.
Meanwhile back at theย ranchโฆ
I luckily found my way in creating a training script for adapters (the modular add-ons discussed in last weekโs blog). The script works for the GLUE datasets. Will keep everyone updated as new events unfold regarding the AdapterHub. Very excited about this new framework, once again thanks to Jonas for nudging me in the right direction.
Stay Frostyย โโ
This Week
SimpleTOD
TurboTransformers
NLP & Audio Pretrained Models
NERtwork
AllenNLP Library Step-by-Step
Search Engining is Hardย Bruh
Dataset of the Week:ย ODSQA
SimpleTOD
Previous task oriented dialogues, especially from those chatbots we all dream of one day building, are built using a standard modular pipeline (similar to what you find in the RASA framework). However, Salesforce Research has recently released a unidirectional language model called SimpleTOD, that attempts to solve all the sub-tasks in an end-to-end manner. It was built with Transformers on the MultiWOZย dataset.
Blog:
SimpleTOD: A Simple Language Model for Task-Oriented Dialogue
Paper
GitHub:
TurboTransformers
A recent transformer runtime library, TurboTransformers, for inference came to my attention. This library optimizes what everyone wants in production, lower latency. Theyย claim:
It brings 1.88x acceleration to the WeChat FAQ service, 2.11x acceleration to the public cloud sentiment analysis service, and 13.6x acceleration to the QQ recommendation system.
The sell is that it can support various lengths of input sequences without preprocessing which reduces overhead in computation. ๐ง
GitHub:
NLP & Audio Pretrained Models
A nice collection of pretrained model libraries found on GitHub. These 2 repos encompass NLP and Speech modeling. Conveniently, the models are indexed by framework and includes a brief description.
NLP
balavenkatesh3322/NLP-pretrained-model
Speech/Audio
balavenkatesh3322/audio-pretrained-model
NERtwork
Awesome new shell/python script that graphs a network of co-occurring entities from plainย text!
It combines Stanfordโs NER for the model, OpenRefine (to deal with data normalization: i.e. B. Obama and Barrack are same entity) and NetworkX for graph creation.
Blog: http://brandontlocke.com/2020/07/22/announcing-nertwork.html
GitHub (Profile photo of theย week):
AllenNLP Library Step-by-Step
Best step-by-step guide into AllenNLPโs library to date. Lengthy but worthwhile with code pasted along the way. The demo is for building/training an NER LSTMย model.
Blog:
Search Engining is Hardย Bruh
Research scientist from AI2 discusses the hardships of building the Semantic Scholar search engine, which currently indexes 190M scientific papers.ย ๐
It uses the 2 model architecture: sparse search via Elasticsearch and then a ranker MLย model.
The blog goes in-depth into the challenges they faced while building the search engine such as data complexity, and evaluation problems. It offers a ton of detail, more than I can handle on this post to give it justice, so give it a read if your are interested inย search.
Building a Better Search Engine for Semantic Scholar
Dataset of the Week:ย ODSQA
What isย it?
ODSQA is a Chinese dataset for spoken question answering (extractive). It contains 3,654 question answerย pairs.
Paper: https://arxiv.org/pdf/1808.02280.pdf
Where isย it?
Every Sunday we do a weekly round-up of NLP news and code drops from researchers around theย world.
If you enjoyed this article, help us out and share withย friends!
For complete coverage, follow our Twitter: @Quantum_Stat

NLP News Cypher | 07.26.20 was originally published in Towards AIโโโMultidisciplinary Science Journal on Medium, where people are continuing the conversation by highlighting and responding to this story.
Published via Towards AI