In the paper they talk about "first generation compositional systems" and I believe they would include differentiable programming in that category. It has some compositional structure, but the structure is created by the programmer.

Ideally the system would be able to create it's own arbitrarily complex structures and systems to understand abstract ideas, like humans can.