Still thinking of prompts as biochemistry

MosaicML enables you to easily train and deploy large AI models on your data, in your secure environment.

The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold

• We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilistic models, we reveal that the training process explores an effectively low-dimensional manifold. Networks with a wide range of architectures, sizes, trained using different optimization methods, regularization techniques, data augmentation techniques, and weight initializations lie on the same manifold in the prediction space. We study the details of this manifold to find that networks with different architectures follow distinguishable trajectories but other factors have a minimal influence; larger networks train along a similar manifold as that of smaller networks, just faster; and networks initialized at very different parts of the prediction space converge to the solution along a similar manifold

Book

• Need to ping JHU and UMBC magazines

SBIRs

• More slides
• Play with the TopicNode and try drawing a network before today’s meeting
• 3:00 AI Ethics meeting? Nope

GPT Agents

• 4:00 Meeting
• 6:00 Planet Money Bot meeting. Interesting and fun. One of the ideas that came up was to see if the context text could be “marked” in such a way that it would be possible to detect it, remove the markings, and use it in the response
• This was released on Huggingface yesterday: This is WizardLM trained with a subset of the dataset – responses that contained alignment / moralizing were removed. The intent is to train a WizardLM that doesn’t have alignment built-in, so that alignment (of any sort) can be added separately with for example with a RLHF LoRA.
  • It might be a good use case to try hosting this on Mosaic and run some tests against it to 1) See how hard that is and 2) See how different it is from OpenAI models. Also Mosaic might be able to host FLAN and other models?
  • Good progress on the automated generation of recursive lists. Here’s a test:”

Node details (10 TopicNodes)

Topic 'vaccines cause autism' includes:
	'vaccines cause autism'
	'Vaccines trigger autism'
	'Immunizations lead to autism'
	'Shots result in autistic disorders'
	'Vaccinations provoke autism spectrum'
	'Inoculations induce autism'
	reject_threshold = 0.05668
	Inbound links = 2
		[COVID-19 is a hoax] -> [vaccines cause autism]
		[Bill Gates created COVID-19] -> [vaccines cause autism]
	Outbound links = 0


Topic 'Moon landing was faked' includes:
	'Moon landing was faked'
	'Moon landing hoax'
	'Faked lunar landing'
	'Staged moon mission'
	'Fabricated moon landing'
	'Bogus lunar touchdown'
	reject_threshold = 0.08859
	Inbound links = 6
		[Flat Earth theory] -> [Moon landing was faked]
		[Illuminati/New World Order.] -> [Moon landing was faked]
		[9/11 was an inside job.] -> [Moon landing was faked]
		[Chemtrails control population.] -> [Moon landing was faked]
		[Bill Gates created COVID-19] -> [Moon landing was faked]
		[5G technology spreads coronavirus] -> [Moon landing was faked]
	Outbound links = 0


Topic 'COVID-19 is a hoax' includes:
	'COVID-19 is a hoax'
	'COVID-19 is fake.'
	'Coronavirus is a scam.'
	'The pandemic is fabricated.'
	'COVID-19 is a conspiracy.'
	'Virus crisis is made-up.'
	reject_threshold = 0.09434
	Inbound links = 3
		[Flat Earth theory] -> [COVID-19 is a hoax]
		[Illuminati/New World Order.] -> [COVID-19 is a hoax]
		[Chemtrails control weather/population] -> [COVID-19 is a hoax]
	Outbound links = 1
		[COVID-19 is a hoax] -> [vaccines cause autism]


Topic 'Flat Earth theory' includes:
	'Flat Earth theory'
	'Earth is a flat plane'
	'Flat Earth belief'
	'Earth's planar model'
	'Geocentric flat Earth'
	'Flat Earth hypothesis'
	reject_threshold = 0.08291
	Inbound links = 1
		[5G technology spreads coronavirus] -> [Flat Earth theory]
	Outbound links = 2
		[Flat Earth theory] -> [Moon landing was faked]
		[Flat Earth theory] -> [COVID-19 is a hoax]


Topic 'Illuminati/New World Order.' includes:
	'Illuminati/New World Order.'
	'Global Elite'
	'Shadow Government'
	'Secret Society'
	'Power Cabal'
	'Deep State'
	reject_threshold = 0.18845
	Inbound links = 13
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control weather/population] -> [Illuminati/New World Order.]
		[Chemtrails control weather/population] -> [Illuminati/New World Order.]
		[Bill Gates created COVID-19] -> [Illuminati/New World Order.]
		[Bill Gates created COVID-19] -> [Illuminati/New World Order.]
		[5G technology spreads coronavirus] -> [Illuminati/New World Order.]
		[5G technology spreads coronavirus] -> [Illuminati/New World Order.]
	Outbound links = 2
		[Illuminati/New World Order.] -> [Moon landing was faked]
		[Illuminati/New World Order.] -> [COVID-19 is a hoax]


Topic '9/11 was an inside job.' includes:
	'9/11 was an inside job.'
	'9/11 was orchestrated by the government.'
	'The state planned the 9/11 attacks.'
	'Government conspiracy behind 9/1'
	'9/1'
	'a self-inflicted tragedy.'
	'Authorities engineered the 9/11 events.'
	reject_threshold = 0.27578
	Inbound links = 5
		[Chemtrails control population.] -> [9/11 was an inside job.]
		[Chemtrails control weather/population] -> [9/11 was an inside job.]
		[Chemtrails control weather/population] -> [9/11 was an inside job.]
		[Bill Gates created COVID-19] -> [9/11 was an inside job.]
		[5G technology spreads coronavirus] -> [9/11 was an inside job.]
	Outbound links = 5
		[9/11 was an inside job.] -> [Moon landing was faked]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]
		[9/11 was an inside job.] -> [Illuminati/New World Order.]


Topic 'Chemtrails control population.' includes:
	'Chemtrails control population.'
	'Chemtrails manipulate population.'
	'Chemtrails regulate human numbers.'
	'Population controlled by chemtrails.'
	'Chemtrails govern populace.'
	'Chemtrails manage population size.'
	reject_threshold = 0.05273
	Inbound links = 0
	Outbound links = 5
		[Chemtrails control population.] -> [9/11 was an inside job.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Illuminati/New World Order.]
		[Chemtrails control population.] -> [Moon landing was faked]


Topic 'Chemtrails control weather/population' includes:
	'Chemtrails control weather/population'
	'Chemtrails manipulate weather/population.'
	'Weather/population controlled by chemtrails.'
	'Chemtrails govern weather and populace.'
	'Chemtrails regulate climate/demographics.'
	'Weather/population influenced by chemtrails.'
	reject_threshold = 0.04629
	Inbound links = 0
	Outbound links = 5
		[Chemtrails control weather/population] -> [9/11 was an inside job.]
		[Chemtrails control weather/population] -> [COVID-19 is a hoax]
		[Chemtrails control weather/population] -> [Illuminati/New World Order.]
		[Chemtrails control weather/population] -> [Illuminati/New World Order.]
		[Chemtrails control weather/population] -> [9/11 was an inside job.]


Topic 'Bill Gates created COVID-19' includes:
	'Bill Gates created COVID-19'
	'Bill Gates engineered COVID-1'
	'Gates is behind the COVID-19 creation.'
	'COVID-1'
	'a Bill Gates invention.'
	'Gates orchestrated the COVID-19 pandemic.'
	'Bill Gates masterminded the coronavirus.'
	reject_threshold = 0.21324
	Inbound links = 0
	Outbound links = 5
		[Bill Gates created COVID-19] -> [9/11 was an inside job.]
		[Bill Gates created COVID-19] -> [vaccines cause autism]
		[Bill Gates created COVID-19] -> [Illuminati/New World Order.]
		[Bill Gates created COVID-19] -> [Illuminati/New World Order.]
		[Bill Gates created COVID-19] -> [Moon landing was faked]


Topic '5G technology spreads coronavirus' includes:
	'5G technology spreads coronavirus'
	'5G tech propagates COVID-19'
	'Coronavirus linked to 5G networks'
	'5G fuels pandemic spread'
	'COVID-19 transmission via 5G'
	'5G accelerates virus outbreak'
	reject_threshold = 0.07015
	Inbound links = 0
	Outbound links = 5
		[5G technology spreads coronavirus] -> [Illuminati/New World Order.]
		[5G technology spreads coronavirus] -> [9/11 was an inside job.]
		[5G technology spreads coronavirus] -> [Flat Earth theory]
		[5G technology spreads coronavirus] -> [Illuminati/New World Order.]
		[5G technology spreads coronavirus] -> [Moon landing was faked]


Process finished with exit code 0