NLP News Cypher 12.22.19

NLP News Cypher | 12.22.19

Riders on the Storm…

This week’s Cypher was inspired by The Doors harrowing song “Riders on the Storm”, which perfectly sets the mood as winter approaches us in the Northern Hemisphere.


Symbolic and Connectionist AI will duel tomorrow. Yoshua Bengio and Gary Marcus will be the avatars.

As such, I started a drinking game. Tomorrow, every time Gary says “hybrid model” we take a shot of Jose Cuervo.

Gary hints we’ll be trashed by 7:00pm EST, FYI, debate’s at 6:30. …🤣🤣🤣🤣

Gary Marcus

This Week:

Those O’Reilly Jupyter Notebooks Live on GitHub!
The World of Conversational AI in 1 Paper
Named Entity Disambiguation (NED)
Dive In to Complexity
Tuning In to Hyper-Parameters
Meet ALBERT: BERT’s More Efficient Cousin
Socket to Me!
IBM Wants All the Smoke!

Those O’Reilly Jupyter Notebooks Live on GitHub!

Examples from the Aurélien Géron ML book series are on GitHub AND Colab, say what?! Wish I knew about this when I was diving into machine learning back in the day.


Machine Learning Notebooks

The World of Conversational AI in 1 Paper

Ok, if you want to recap the industry’s research and development in Conversation AI, there is only one place you should look into, this paper:

Neural Approaches

Named Entity Disambiguation (NED)

Let’s say a financial company wants to investigate how relations between companies mentioned in a news article will affect markets. Well a new model using NED (uses Knowledge Graphs 🔥🔥) went up on Medium that can help you with this scenario:

Named Entity Disambiguation Boosted with Knowledge Graphs

We combine text and graph based approaches to build a Named Entity Disambiguation pipeline.

Dive In to Complexity

Complexity affects us all. A system with many inter-dependent parts act funny (aka non-linearly). In order to get our intuition focused right, NECSI released a non-technical introductory paper for us. If you work with neural networks and natural language (i.e. 2 complex systems) read this:


Cite as: Alexander F. Siegenfeld and Yaneer Bar-Yam, An introduction to complex systems science and its applications, arXiv:1912.05088 (December 11, 2019).

Berkeley also published a blog post that touches in the same vein (Chaos Theory is a subset of Complexity):

Emergent Behavior by Minimizing Chaos

All living organisms carve out environmental niches within which they can maintain relative predictability amidst the ever-increasing entropy…


I know, when you are not debugging dtypes in your CSV files, you are probably wondering what is the current state-of-the-art model for Named Entity Recognition?


Cloze Driven

Tuning In to Hyper-Parameters

Ok, you have your ML algorithm ready for fine-tuning. You’re past the pre-processing stage and now you wanna achieve the highest accuracy, with least amount of compute, in the least amount of time. Here’s an easy guide towards setting your parameters with Scikit-Learn, includes code:

How to Do Hyperparameter Tuning on Any Python Script in 3 Easy Steps

You wrote a Python script that trains and evaluates your machine learning model. Now, you would like to automatically tune hyperparameters to improve its performance?…

Meet ALBERT: BERT’s More Efficient Cousin

Somehow I managed to find Albert’s paper 2 days before Google announced it publicly??? That’s some Illuminati stuff by me. I don’t even remember how I did it, I may have seen it on Twitter by one of their researchers. Anyway:

ALBERT brings SQuAD’s (v2.0) F1 score to 92.2! 💪💪💪

Dec. 18:

Dec. 20:

ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations

Ever since the advent of BERT a year ago, natural language research has embraced a new paradigm, leveraging large amounts of existing text…

Socket to Me!

If you‘re balling in machine learning deployment, you probably have conducted some horizontal scaling on your back-end. Well, if you no idea what any of this means, check out how you can scale ML deployments:

WebSockets for fun and profit

Seamless communication is a must on the modern web. As internet speeds increase, we expect our data in real time.

IBM Wants All the Smoke!

Gotta give em’ credit, IBM just put everyone on notice. “No big players, except for us”

IBM Watson exec on AI virtual agent providers: ‘No big players, except for us

One area of AI that’s red-hot is virtual agents — smart software that companies are building to chat with their customers through text, voice, or a web chat box…

“At the core of it is a model for intent classification. So we do a really good job of understanding intent. Just based on the questions you ask, we can get a feel for what you’re trying to accomplish. That’s kind of the secret sauce.”

Google be like:

Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.

If you enjoyed this article please share with friends or social media!

And for complete coverage, follow our twitter: @Quantum_Stat