Will be very interesting to see if the development will be to run the models locally (on mobile, PC etc) or the need for the cloud.

Will vary by use-case. Image generation is for example fitted in to much smaller models them languages. The rout it will take will effect the tech ecosystem in many ways with someone like Apple benefitting much more on local models, and AWS, Azure etc benefiting from larger models.