Still tired from yesterday’s adventure:

Thinking about @scottbot’s thread on TalkToATransformner.com. In a way, the GPT-2 and other models are an arrangement of all the text that the system has incorporated. I think I’s like to go through the dictionary, one word at a time, and use that word as a seed for building a low-dimensional threaded term model that we can visually examine, in the same way that I used text from Tymora to show the “belief space” around each room. For example, here are some responses to silence:

• Lecture, Thursday 17 June 2014, 10:30am
  In the course of our study we will observe the behaviour of animals at two distinct locations – one to an animal’s immediate (neonatal) environment, the other to two distinct environments in its home territory – the home environment in which the animal lives. We will also observe the behaviour of animals in an adjacent environment to each animal we observe in one of these new environments; one for the social or ‘prey’ species and the other for the ‘host’ species.
• In addition to a few other notable games, like the two I mentioned earlier, the team was able to pick up several top players to form one of the premier teams in the region of Korea. At this time, the team was ranked 11th in the league of legends scene.
• Sensing my heart beating at a frantic pace, I raised my head to look at the place where the girl’s mouth had been. A large, blue, round hole had appeared there.
•  キャラクシー 土方の親愛神・レイラン Aamir AB Black Bird Bomb AB Bluebird Blues AB Bomb AB Chuck AB Dashing Wingman AB Green Bird Hal AB Hal AB Matilda AB Red AB Red Bird Red AB The Blues AB White Bird Matilda AB Wingman AB Yellow Bird Chuck ABE Archer Blues ABE Capt’n Bomb ABE Cleric Matilda ABE Fencer Red ABE Guard Captain Pig ABE Knight Red ABE Mage Chuck ABE Pirate Bomb ABE Priest Matilda ABE Royal Guard Pig ABE Trickster Blues ABE Wizard Chuck Abiding Sniper Dragonbound, Myr Absolute Zero Hrungnir Abyssal CyberDragon, Sadalmelik Abyssal 
• is about what you do, not what you say. And the answer is: it’s not something you should say. In fact, I want you to make an effort to talk to others.
• I was so excited for my wedding day, and I had to be there. My mom had an idea to hold my wedding in a different room and I made it happen. I had my brother, sister, bridesmaid, and a couple of guys wait in the front. It was so beautiful and relaxing. Thank you, thank you, THANK YOU!

My intuition is that these word sequences can be treated like sequential threads, since they are sentences/paragraphs, and have an implicit sense of time. And like like the room conversations in the temple, that there will be denser and sparser areas on the network/landscape that we might be able to map. If we can build these small pieces, we can then experiment with ways to assemble the larger quilt.

Just to reinforce this idea, I thought about using more specific terms or phrases. Here’s what we get with bird sanctuary. I’d a much more constrained landscape:

• is open only 24 hours a day and is open on the following holidays:
• Tower of the Winds – Cave of Wonders – Rune Isle
• The idea of an animal sanctuary for a big-cat sanctuary is one of the most amazing things that a lot of people will ever come up with that they can’t see in the current environment of wildlife protection. 
• an annual four-day event that promotes conservation efforts.
• (2) Pescado Bay Nature Preserve (2) Pacific Coast Aquarium (11) Pacific Grove (1) Pacifica Harbor (1) Philadelphia Zoo (1) Philadelphia Museum of Art (1) Philadelphia World’s Fair (2) Piebald Beach (1) Pinnacle Beach (1) Placid Bay (1) Point Park and Wildlife Management area

Based on David Massad’s tweet, I think the phrases to use are news headlines, that can be compared to some sort of ground truth contained in the story.