NLP News Cypher | 12.08.19

NLP News Cypher | 12.08.19

Ok, ok. Lots of exciting events happened this past week that we want to share with you. While peeps are in-flight to Vancouver for NeurIPS 2019 (starting today), Twitter has been a’buzz with data galore. And last not but least, we gave a talk about hosting a low-resource BERT in the cloud for Question Answering. Check out my “pointing at screens” skills here… 👇

Ricky Costa Talk

declassified | for eyes only

This Week:

– Top Papers From NeurIPS Ranked by GitHub Popularity
– Twas the Night Before NeurIPS
– Decoding Transformers
– Hyper-Excited for Graphs
– TensorFlow Kills Python 2
– Moving Away From Intents in Conversation
– SOTA Multi-Hop QA for Open Domain

Top Papers From NeurIPS Ranked by GitHub Popularity

Papers with Code: Conference – NeurIPS Dec 2019

Twas the Night Before NeurIPS

When all through the house not a creature was stirring, not even a mouse. On a cold December night, IBM dropped this

IBM Research AI at NeurIPS 2019

Deep learning to better understand video. 8-bit training. AI for genomics and beer. The 33rd Conference on Neural Information Processing Systems (NeurIPS) in Vancouver is…

2 days later, Facebook Research dropped this…

Facebook Research: NeurIPS 2019

Neural Information Processing Systems (NeurIPS) is the largest conference in AI, with machine learning and neuroscience experts traveling from around the world to discuss the latest advances in the field…

Ian GoodFellow:


Decoding Transformers

The lads from Brooklyn just added decoders to their encoders. Yea, this may be a surprise to some of you but most transformers (excluding Bart and T5) only have an encoder stack:

In a really enjoyable read from Hugging Face, they share how they added decoders to its library in an attempt to get better at language understanding and generation in the same model. They also provide an intuitive history of the transformer architecture.


🦄🤝🦄 Encoder-decoders in Transformers: a hybrid pre-trained architecture for seq2seq

Our Transformers library implements many (11 at the time of writing) state-of-the-art transformer models. It is used by researchers…

In addition, 🤗 has a new demo with even more decoding controls (BOOM):

Write With Transformer

PPLM builds on top of other large transformer-based generative models (like GPT-2)…

Hyper-Excited for Graphs

Yes graphs are cool. Yes they are popular. But have you heard of hypergraphs? In this piece from Grakn.AI, they define hypergraphs and show how they differ from traditional SQL databases and regular property graphs (RDF triplet stores).

Remember: edge is a bridge, vertices are endpoints of bridges…
  1. A hypergraph consists of a non-empty set of vertices and a set of hyperedges;
  2. A hyperedge is a finite set of vertices (distinguishable by specific roles they play in that hyperedge);
  3. A hyperedge is also a vertex itself and can be connected by other hyperedges.


Modelling Data with Hypergraphs

In our previous post “Knowledge Graph Representation: GRAKN.AI or OWL?”, we explained why…

If you are interested in knowledge graphs and you are in NYC this week, Grakn will be giving a talk (we will be there)… They are calling for all hands on deck:

TensorFlow Kills Python 2

For those of you who refuse to use parentheses in your print commands, Python 2.7 will be deprecated after TensorFlow 2.1.


Moving Away From Intents in Conversational AI

In a thought piece by Alan Nichol of Rasa, he discusses how intents limit the possibilities of conversation and if we want to move towards the next wave of innovation in chatbot technology, intents need to be replaced by a more flexible means of NLU.


It’s about time we get rid of intents

In early 2016 I wrote that we don’t know how to build conversational software yet. I’m writing a couple of posts to follow up on that one, taking one slice at a time discussing progress and open problems.

SOTA Multi-Hop QA for Open Domain

Salesforce/Allen Institute/ Uni of Wash. dropped heat this week with their open domain question answering research revealing they had advanced SOTA results on HotpotQA by 10 points. 🤫 It’s a retriever / reader model:

A Thread Written by @AkariAsi

New work with Kazuma Hashimoto, @HannaHajishirzi, @RichardSocher, and @CaimingXiong at @SFResearch and @uwnlp! Our trainable…


Sad news: In a pithy tweet exchange with Francois Chollet, he told us he won’t be at NeurIPS (don’t cry for me Argentina):

This column is a weekly roundup of NLP news and code drops from researchers worldwide.

Follow our Twitter for complete coverage: @Quantum_Stat