NeurIPS 2019 starts in one week on Dec. 8th in Vancouver!!! Should be a good one. As of September, 1428 papers were accepted (record). Per usual, I will gather top tweets and insights from the conference. Follow our Twitter @Quantum_Stat for closer coverage. FYI, we went over a more detailed version of what the conference holds in our previous post here.
Also…
Set phasers to stun — for all you NLPers in NYC tomorrow — I will be giving a talk on hosting BERT in the cloud. Will discuss pipeline I used to deploy the transformer for question answering 👀.
Festivities start at 6PM, sign-up below:
Our December 2 meetup features two speakers, Ricky Costa on How to Host a Transformer Model in the Cloud…
This Week:
– Transfer Learning Coming to the Nasdaq – Python Tools for Finance – DistilBERT + Logistic Regression for Sentence Classification – Lex Interviews Noam – How Instagram’s Explore Recommendation Engine Was Built – Hugging Face Releases New Models – AWS is Pretty Popular – Yann LeCun vs. Gary Marcus |Crouching Tiger Hidden Dragon
Transfer Learning Coming to the Nasdaq
We already knew that AI is used extensively in Finance, but now it looks like the Nasdaq peeps are going to deploy transfer learning and human-in-the-loop models to analyze the stock market. For those playing the home game, Finance peeps are mostly interested in event extraction |sentiment analysis and their impact on stock performance during the active trading day.
Press Release:
NEW YORK, Nov. 07, 2019 (GLOBE NEWSWIRE) — Nasdaq, Inc. today announced the successful go-live of artificial intelligence for surveillance patterns…
segue…
Python Tools for Finance
The TL;DR version is that they use Pandas for those lovely DataFrames we all enjoy, Cython to run Python with C speed, use message queues (Celery, Redis) for communicating between servers, and Plotty becoming popular for data viz. (sorry MatPlotLib 🤫)
Blog:
The data science benefits of Python are now being felt across financial markets. Dedicated ‘Pythonista’ Saeed Amen takes us on a tour of the best Python tools and libraries…
DistilBERT + Logistic Regression for Sentence Classification
Check out this dude’s repo, he used DistilBERT+Logistic Regression for sentence classification trained on the SST2 dataset (Boolean labeled positive/negative). In detail, he used a 2 model methodology. First, DistilBERT was used for creating sentence embeddings, then he trained a Logistic Regression for binary classification on those embeddings. Massive props for the illustrations, really intuitive! Colab + notebook included!
Progress has been rapidly accelerating in machine learning models that process language over the last couple of years…
Lex Interviews Noam
Mr. Fridman, of the MIT variety, interviewed Noam Chomsky on his podcast! Really cool and insightful if you like Linguistics:
How Instagram’s Explore Recommendation Engine Was Built
According to Facebook, over half of Instagram’s roughly 1 billion users visit Instagram Explore to discover videos, photos, live-streams, and Stories each month.
So how does it work? Well for starters the model is freaking huge, they had to create their own framework just to engage with the data! Explorer comes in two stages. The first stage attempts to topic model the Instagram accounts you have previously visited and engaged with. Then the 2nd stage ranks the most suitable accounts that fits the criteria discovered in the 1st stage. I completely over-simplified the real thing, however, if you want a closer look at the details look here:
According to Facebook, over half of Instagram’s roughly 1 billion users visit Instagram Explore to discover videos, photos, livestreams…
Hugging Face Releases New Models
No need to summarize:
Transformers v2.2 is out, with *4* new models and seq2seq capabilities!
— Hugging Face (@huggingface) November 26, 2019
ALBERT is released alongside CamemBERT, implemented by the authors, DistilRoBERTa (twice as fast as RoBERTa-base!) and GPT-2 XL!
Encoder-decoder with ⭐Model2Model⭐
Available on https://t.co/rNsF2qEV2C pic.twitter.com/r6M39jYPHf
AWS is Pretty Popular
A new Indeed report suggests that 14% of job listings require applicants to have AWS skills! Even Azure beats Google Cloud… What’s up with that? Mostly, this reflects the rise in MLDevOps and data science jobs.
…But one of the most striking changes in recruitment website Indeed.com’s tech jobs listings over that period is Python’s growth. …
Yann LeCun vs. Gary Marcus | Crouching Tiger Hidden Dragon
Hey remember 2 weeks ago when I mentioned Gary Marcus’s quest for the amalgamation of symbolic and connectionist AI? Well, Yann LeCun sent Gary a message…ice ice baby 🥶🥶:
GM is the root cause of the purported war between connectionists and symbolists
— Yann LeCun (@ylecun) December 1, 2019
He created it to make himself relevant.
But it does not actually exist.
Clamoring that DL should be hybridized with symbol manipulation is useless.
Showing how to do it would be useful. https://t.co/EKxnnk4dzw
In other words:
Gary Marcus sleeps with the fishes…