Beyond Here Lies Nothing…
As you may have guessed, current laws that govern simple systems (i.e. Newtonian physics) allow for the observation of independent variables. That is, if you were to observe a variable in this so-called simple system, information isn’t lost from other variables in the system of observation.
Observing the trajectory of Venus doesn’t impact the trajectory of Earth.
However, this independence becomes difficult to observe accurately as we increase the number of variables, and where we find existing relationships between variables. These variables, that usually come in clusters or systems, which are inter-dependent are the units that make up a complex system.
There exist various complex systems in reality, or what Neo would call the Matrix, such as evolutionary processes, climate, the brain, language (NLP?), and even the stock market…
Let’s take a variable “sentiment” and a complex system “stock market”. If we were to observe the state of the stock market from the perspective of news headlines and attempt to classify whether a headline is bullish, bearish or neutral; we would soon realize we are in the throes of complexity.
Here’s an example, let’s say we have this headline:
Gold is up 6% in the pre-market as the downward pressure of the coronavirus outbreak weighs on equity stocks.
What do you think is the ground-truth sentiment for this headline? Well gold is up that’s good (bullish right?), but wait it’s saying equities are down (bearish then?), right but it’s saying both pos/neg statements in the headline (so it’s neutral right?!🤷♂️). It seems as though the packets of information in the headline are not independent, we lose information from reducing sentiment down to clause. Natural language is hard homies!
Let us meditate on this until next week (cliffhanger), in the meantime, this is what our upcoming demo of a real-time platform that classifies financial news by topics/sentiment looks like 👇. Haven’t finished deploying, if you want early access please DM me on El Twitter.
How was your week?