Basically, I feel like if you are going to give LLMs much more capabilities through utilizing 3rd party plugins, then you should probably use a weaker version of the LLM to save computational power. The amount of computation involved in answering a single prompt is much higher for LLMs with larger number of parameters compared to that of smaller number. However, you are seemingly getting better/more accurate answers as a result of using GPT-4 vs say GPT-3. But if the 3rd party apps can compensate for the LLMs in thousands of different ways, it would be prudent to use GPT-3 with TaskMatrix.ai as opposed to GPT-4. At least that is how I see it.