eroche
Emmanuel Roche’s personal page
Keywords: NLP, Computational Linguistics, LLMs, Language Learning (Chinese), Language Evolution
Interest in language and languages can easily become obsesive. It is for me. Here are some of the questions that I have been considering for a long time:
- What is syntax? Does it exist? Can it be described formally and operationally? How much of it can be done mechanically? I look at these questions through my work on parsing/finite-state processing mostly?
- Can we separate what LLMs know about language from what they know about the world?
- How much symbolicity, logic in Language vs how much of “soft” processing (a la Deep Learning)?
- what type of logic permeates through language? (partial answer: many kinds of logic interacting together)
- What’s the limit in efficiency for Second Language Acquisition? What does it teach us about the nature of language?
- The origin of language (I am very much of the Daniel L. Everett school on that matter): can parsing and formal syntax analysis help plot a plausible, and, ideally, testable, path of language evolution? (at the scale of human evolution; not at the century or millenia scale, which is a different kind of question)
- More generally, what is the nature of language?
Recent Work (NLP)
Parsing with Geometric Transductions
Symbolic parsing is possible! This is a large part of what I have been working on this past few years and I will be releasing a sequence of drafts (comments welcome).
Emmanuel Roche. 2023. (I) Finite-State Representation of Geometric Transductions. DRAFT
Transformer/Symbolic Hyridization with Finite-State Transducers
Deep learning has long been opposed to the kind of symbolic processing that was developed during the first decades of natural language processing. Geoffrey Hinton declared in his Turing price acceptance speech (2018) that deep learning has won and symbolic processing has lost. I think that language is more probably a combination of both approaches. It is possible to combine these approaches with transformers and the kind of finite-state transducers alluded to in the previous section.
Here is a preview about what this current work:
Work on Secondary Language Acquisition (Chinese)
My first few years of learning Chinese were both exhilarating and incredibly frustrating. Exhilaration from the sheer about of knowledge about the word. It was frustrating because I seemed to be learning way too slowly. It was like putting gas in an engine and getting back a 1% efficiency. Was it possible to push that much higher? 50%, maybe? There was only one way to find out: build a solution that tested all the hypotheses and release it as an app. That’s what we did with a few friends. You can check out our FullChinese solution (still evolving; many ideas to try).
You can also check out a recent write-up that Chen Tong (teaching Chinese at MIT) and I have recently put together.
Full CV here.