Bringing Open Large Language Models to Consumer Devices

• This post describes our effort on streamlining the deployment of Open LLMs through a versatile machine learning compilation infrastructure. We bring RedPajama, a permissive open language model to WebGPU, iOS, GPUs, and various other platforms. Furthermore, the workflow we have established can be easily adapted to support a wide range of models with fine-tuned (personalized) weights, promoting flexibility and customization in LLM deployment.

The overparameterized, paralyzed generation

SBIRs

• Sprint demos. Need to make slides – done
• Sent off the Q5 report

GPT Agents

• Got a lot done in reading the json files and making spreadsheets
• Created a rollup spreadsheet that I think I’ll use for the paper