GPT Agents:

• Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
  • Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models’ novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. In this work, we discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions and techniques for encouraging deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, we also introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. Finally, we discuss how these more general methods of interacting with language models can be incorporated into existing and future benchmarks and practical applications.
• Language models are 0-shot interpreters
  • In this post, I present evidence that the efficacy of 0-shot prompts for GPT-3 has been underestimated, and that more powerful models are more effective at deriving information from 0-shot prompts, while less powerful models have greater need for examples on equivalent tasks. From this evidence, I extrapolate three principal claims:
    • Few-shot prompts are not always an efficient or necessary means of task specification for GPT-3.
    • For some tasks, such as translation between well-known languages, GPT-3 is a 0-shot interpreter – a short task description or signifier suffices to invoke its full capabilities.
    • 0-shot performance scales with model size more drastically than few-shot performance, suggesting that 0-shot task specification will become a more important prompting strategy as language models increase in capability.
• Started training the January 2020 model
• It looks like sim got the new format model trained? It’s up through Feb 21. Need to adjust the query code and parser and do some runs for the queries we discussed last night as well as month/year prompts. And combos, e.g. October 2020 USA [[COVID-19 happened because
• Here’s the first try:

• Clearly, we’re going to have to look across a range of places to get good counts
• Another thought, we could use autoencoding models like BERT to for fill-in-the-blank. Here’s how to do it: How to use BERT from the Hugging Face transformer library

SBIR

• Working on the status report. I’ll distribute tomorrow for input to the financial section

GOES

• 2:00 Meeting
• Still waiting on Vadim to get the reaction wheel efficiency in the right place and inertialess reset

JuryRoom

• Reading “Purakau: Maori Myths Retold by Maori Writers”. Some interesting perspectives on group problem solving and education, particularly in the story ‘Rata’, by Hemi Kelly:
  • ‘“Sail towards the rising sun,” she instructed him, “there you will find Pariroa, the home of Matuku.” After saying this, she handed Rata an old toki, “You will need this to fashion your waka.”’
  • “You didn’t recite the correct karakia – or in fact any karakia. Instead you carelessly chopped down your ancestor, a child of Tāne, for your own gain without offering anything in return.”
  • ‘It’s the same with our rongoā. Anybody can go and pick a leaf and eat it but it’s the process we follow that makes it right. It’s the time we go, the area we visit and the careful selection. The most important thing, though, is our acknowledgement of Tāne through karakia, as it’s the karakia that gives the rongoā its healing properties that make us better.’