Cool. I think we may have a problem with the term "symbolic AI NLP & Linguistics" I know how the LLMs actually work and they do not "just predict the word". If you had said "language comprehension using manually written non-stochastic processing rules" I would better understand. The vectors associated with the words contain more than just a list of other words and their probabilities.
That said, for example, I wrote the NL processor used by Caere's Pagekeeper Product back in the mid 1980s. For years this was the only 'OCR' product that could find pages similar, in content, to other pages. And yes, this used stochastic processing to process out word concepts in grammatical context. Much as, for example, all LLMs today. That product was very successful. My Ph.D. (1976) was on the subject "Causal Relations Between Sentences" here it is with a free link…read Chapt 4.
https://drive.google.com/file/d/16Uk3xxEQpivMa1mYXNO4GqrtPk4YdZYS/view?usp=sharing
One of my papers "Action Perception" (pure conceptual inference) was about watching cartoons and providing and English language narration. https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1002_1 (Sorry it is not my paywall: BLAME WILEY WITH EVERY OUNCE OF YOUR INTELLIGENCE. This was my work and they profit from it for a hundred Mickey Mouse years! And you wonder why I do my own publishing! www.privust.com)
I personally wrote all the code for most everything I published. And, by the way, it was freely given away to everybody at CMU regardless of whether I wrote it personally. Sadly, this seems to be coming to an end at OpenAI and such.
That said, it is good to hear about your experience and I certainly wish you well. I do disagree that LLMs do not, for example, know the difference between a noun and a verb. They do.
One thing that almost no one mentions, ever, which is utterly amazing about human natural language, is that dictionaries work. Any word can be defined using other words. This is not a "skinnerian conditioning" thing. It is fundamentally innate and we have known for many years that you can do this in surface structure parsing, such as you see in GPT.
That said, we also know and have known for many years how to process symbolically (re: dictionaries) in useful ways, but simply lacked the hardware to do it right. That is the situation today. We are still a long way from that in a fully comprehensive manner .. on the scale at which the brain (most any brain) computes. Things like 'Tensor Flow' are just cheap engineering tricks to speed up matrix algebra when the proper hardware computationa are not available. The human brain does not do matrix algebra except with some difficulty.
The "causation" work famously by a statistician, Judea Pearl, who thinks he is an AI person (but isn't), completely fails in his understanding of NLP. He thinks people keep "causal graphs" in their heads. You might correctly call that "symbolic ai" and I would agree that doesn't work. Yet he freely writes long-winded matrix algebra equations about causal graphs and people think he knows what he is talking about. I have never seen a word from him addressing the speed with which humans (or dogs for that matter) comprehend causal transitive closure. For him it is a problem solving walk over a long graph. But advanced perceptrons, done correctly (which we cannot yet do due to hardware) have no problem. Again, we've known this since the late 1970s.
Here is my latest (patent pending) gambit to give GPT a run for its money. Deceit finds holes in advanced perceptron processing even at brain scale.
This system is designed to find those holes until brain scale can be achieved:
(And, yes, we have to use living brains to enumerate the domain. Static Rule based systems provably cannot work here either).
https://medium.com/@rhtcmu/three-types-of-trial-in-the-internet-court-of-truth-and-lies-36d22f5e39be
