Monthly Archives: March 2023

Phil 3.9.2023

ChatAug: Leveraging ChatGPT for Text Data Augmentation

Text data augmentation is an effective strategy for overcoming the challenge of limited sample sizes in many natural language processing (NLP) tasks. This challenge is especially prominent in the few-shot learning scenario, where the data in the target domain is generally much scarcer and of lowered quality. A natural and widely-used strategy to mitigate such challenges is to perform data augmentation on the training data to better capture the data invariance and increase the sample size. However, current text data augmentation methods either can not ensure the correct labeling of the generated data (lacking faithfulness) or can not ensure sufficient diversity in the generated data (lacking completeness), or both. Inspired by the recent success of large language models, especially the development of ChatGPT, which demonstrated improved language comprehension abilities, in this work, we propose a text data augmentation approach based on ChatGPT (named ChatAug). ChatGPT is trained on data with unparalleled linguistic richness and employs a reinforcement training process with large-scale human feedback, which endows the model with affinity to the naturalness of human language. Our text data augmentation approach ChatAug rephrases each sentence in the training samples into multiple conceptually similar but semantically different samples. The augmented samples can then be used in downstream model training. Experiment results on few-shot learning text classification tasks show the superior performance of the proposed ChatAug approach over state-of-the-art text data augmentation methods in terms of testing accuracy and distribution of the augmented samples.

GPT Agents

Add a “no context” checkbox – done. Super interesting how things change. The question is “what is the best way to hunt whales?”:

——————————— Without prompt tuning

The best way to hunt whales is to use a harpoon. Harpoons are designed to penetrate the thick skin of whales and are the most effective way to hunt them. Additionally, it is important to use a boat that is large enough to handle the size of the whale and to have a crew that is experienced in whale hunting.

———————————- With prompt tuning

The best way to hunt whales is to use two harpoons connected to the same line and throw them into the water, with the spare coils of box line making it possible for the harpooneer to pitch the second iron even if the whale runs away after receiving the first. Additionally, the whaleman must use the manoeuver of pitchpoling with a lance to accurately dart it from a violently rocking boat.

Make it so that the active tab in GPTContextFrame is switched to gen_tab when any of the “actions” buttons are pressed – done
Set the summary engine to chatGPT and evaluate
Add in charting of speech categories (and saving to spreadsheet)
Add moderation json field to narrative maps – done

SBIRs

Submitted Q4 report to Lauren. It looks good!
9:15 standup. Need to close tasks
9:30 GA discussion with Rukan
10:00 GPT for BD
More UMAP with Aaron
Create a “military” group and add Clausewitz and Sun Tzu to begin. This means I need to add the * for multiple texts in one group
- Downloaded, trimmed, and loaded

Phil 3.8.2023

Human heuristics for AI-generated language are flawed

Human communication is increasingly intermixed with language generated by AI. Across chat, email, and social media, AI systems suggest words, complete sentences, or produce entire conversations. AI-generated language is often not identified as such but presented as language written by humans, raising concerns about novel forms of deception and manipulation. Here, we study how humans discern whether verbal self-presentations, one of the most personal and consequential forms of language, were generated by AI. In six experiments, participants (N = 4,600) were unable to detect self-presentations generated by state-of-the-art AI language models in professional, hospitality, and dating contexts. A computational analysis of language features shows that human judgments of AI-generated language are hindered by intuitive but flawed heuristics such as associating first-person pronouns, use of contractions, or family topics with human-written language. We experimentally demonstrate that these heuristics make human judgment of AI-generated language predictable and manipulable, allowing AI systems to produce text perceived as “more human than human.” We discuss solutions, such as AI accents, to reduce the deceptive potential of language generated by AI, limiting the subversion of human intuition.

GPT Agents

Added call to Moderation API as described here: platform.openai.com/docs/guides/moderation/quickstart?lang=python
Need to get the embedding and moderation vals added to TweetEmbedExplorer
2:30 NIST meeting
4:00 Weekly GPT meeting
New plots using binary data

SBIRs

Review and submit the Q4 report

Phil 3.7.2023

Open science involves sharing of code, and Python is a popular language for that code. Scientists may be reluctant, though, to try shared Python code when doing so involves many installation steps, like installing Conda, installing packages, installing other packages with Pip, possibly resolving package conflicts, etc.

An appealing alternative is to “bundle” the Python code and its dependencies into a single executable that can be downloaded from the “Releases” section of a GitHub site. This project is a test bed for working out the detals of such an approach. This project is called a “demo” rather than a “test” just in case any of the tools involved implicitly assume that items with names including “test” are parts of an internal test suite.

SBIRs

9:15 standup
9:30 USNA meeting
1:00 BMD bi-weekly
More document loading, embedding, and storing to db
I also need a “*” option to load all groups added to the list when appropriate
There is a “moderation” endpoint on the OpenAI API. Add that to twitter_v2.table_tweet. Probably just the category_scores json object

GPT Agents

Read in the King James Bible
Got sources working!

Phil 3.6.2023

Back from GSAW. It was nice to be at a conference physically again

10:00 Dentist

SBIRs

Working on the quarterly report. Also need to set up the Q5 files and folders on Overleaf – done
2:00 MDA Meeting – done
Talk to Aaron about paper? Also trip to VA? Done

GPT Agents

Fixed a bunch of small bugs
Need to get the loading of data, summaries, and embeddings – progress
Fix TweetEmbedExplorer to use BLOBs. Then re-embed and cluster

Phil 3.2.2023

Flying home from GSAW

Evidence of a predictive coding hierarchy in the human brain listening to speech

Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition.