Ok, ok. Lots of exciting events happened this past week that we want to share with you. While peeps are in-flight to Vancouver for NeurIPS 2019 (starting today), Twitter has been a’buzz with data galore. And last not but least, we gave a talk about hosting a low-resource BERT in the cloud for Question Answering. Check out my “pointing at screens” skills here… 👇
declassified | for eyes only
This Week:
– Top Papers From NeurIPS Ranked by GitHub Popularity
– Twas the Night Before NeurIPS
– Decoding Transformers
– Hyper-Excited for Graphs
– TensorFlow Kills Python 2
– Moving Away From Intents in Conversation
– SOTA Multi-Hop QA for Open Domain
Top Papers From NeurIPS Ranked by GitHub Popularity
Twas the Night Before NeurIPS
When all through the house not a creature was stirring, not even a mouse. On a cold December night, IBM dropped this
Deep learning to better understand video. 8-bit training. AI for genomics and beer. The 33rd Conference on Neural Information Processing Systems (NeurIPS) in Vancouver is…
2 days later, Facebook Research dropped this…
Neural Information Processing Systems (NeurIPS) is the largest conference in AI, with machine learning and neuroscience experts traveling from around the world to discuss the latest advances in the field…
Ian GoodFellow:
DECLASSIFIED
Decoding Transformers
The lads from Brooklyn just added decoders to their encoders. Yea, this may be a surprise to some of you but most transformers (excluding Bart and T5) only have an encoder stack:
In a really enjoyable read from Hugging Face, they share how they added decoders to its library in an attempt to get better at language understanding and generation in the same model. They also provide an intuitive history of the transformer architecture.
Medium:
Our Transformers library implements many (11 at the time of writing) state-of-the-art transformer models. It is used by researchers…
In addition, 🤗 has a new demo with even more decoding controls (BOOM):
PPLM builds on top of other large transformer-based generative models (like GPT-2)…
Hyper-Excited for Graphs
Yes graphs are cool. Yes they are popular. But have you heard of hypergraphs? In this piece from Grakn.AI, they define hypergraphs and show how they differ from traditional SQL databases and regular property graphs (RDF triplet stores).
Remember: edge is a bridge, vertices are endpoints of bridges…
- A hypergraph consists of a non-empty set of vertices and a set of hyperedges;
- A hyperedge is a finite set of vertices (distinguishable by specific roles they play in that hyperedge);
- A hyperedge is also a vertex itself and can be connected by other hyperedges.
Medium:
In our previous post “Knowledge Graph Representation: GRAKN.AI or OWL?”, we explained why…
If you are interested in knowledge graphs and you are in NYC this week, Grakn will be giving a talk (we will be there)… They are calling for all hands on deck:
Can’t wait @Quantum_Stat who else will be in NYC with us?
— Grakn Labs (@GraknLabs) December 4, 2019
Looking at you @SpotifyEng @dataiku @TWTechTalksNYC @PinterestEng @facebookai @googledevs @MSFTResearch @msdev https://t.co/1HnKCFhg7r
TensorFlow Kills Python 2
For those of you who refuse to use parentheses in your print commands, Python 2.7 will be deprecated after TensorFlow 2.1.
R.I.P.
🎉TensorFlow 2.1.0-rc0 has been released!
— TensorFlow (@TensorFlow) December 4, 2019
TensorFlow 2.1 will be the last TF release supporting Python 2. Please see the full release notes for details on added features and changes.
Full release notes ↓ https://t.co/pvDXNV0WRU
Moving Away From Intents in Conversational AI
In a thought piece by Alan Nichol of Rasa, he discusses how intents limit the possibilities of conversation and if we want to move towards the next wave of innovation in chatbot technology, intents need to be replaced by a more flexible means of NLU.
Blog:
In early 2016 I wrote that we don’t know how to build conversational software yet. I’m writing a couple of posts to follow up on that one, taking one slice at a time discussing progress and open problems.
SOTA Multi-Hop QA for Open Domain
Salesforce/Allen Institute/ Uni of Wash. dropped heat this week with their open domain question answering research revealing they had advanced SOTA results on HotpotQA by 10 points. 🤫 It’s a retriever / reader model:
New work with Kazuma Hashimoto, @HannaHajishirzi, @RichardSocher, and @CaimingXiong at @SFResearch and @uwnlp! Our trainable…
Paper
Sad news: In a pithy tweet exchange with Francois Chollet, he told us he won’t be at NeurIPS (don’t cry for me Argentina):
Keras more important, we agree.
— Quantum Stat (@Quantum_Stat) December 6, 2019