7:00 – 8:00 Research

• I searched for better Electome images (unannotated) and found a few. They are on the previous post. I’ll seriously start to work on the poster this weekend.
• Cleaned up some code, and made the writing of an image file optional. Still figuring out the best way to do helper classes in Python
• Starting to learn the analytic capabilities of Networkx, which is its core as I understand. Going to start to characterize the networks and compare against stored sets (like the Karate Club) that is in the library
• List of algorithms

8:30 – 4:30 BRC

• Working on cluster member sequence visualization.
  • Needed to make unclustered user configurable
  • Needed to make timeline sequence, where OPEN and CLOSED states are enabled, user configurable
• Results! LLE looks to be the best by far:

  

  

  

  

  

  
  

7:00 – 8:00, 4:00-7:00 Research

• Guest lecture for CSCW class! Notes!
  • And it went well. Fun, actually.
• Working on making the line width based on the time spent in the cluster, the cluster size a function of its lifespan, and the agent size, a function of time spent in a cluster
  • While working on the last part, I realized that I was including ‘unclustered’ (-1) as a cluster. This made all the agents the same size, and also messed up the cluster collating, since unclustered can be the majority in some circumstances. Fixing this made everything much better: so now I need to rerun all the variations. Done! Rough boaster poster:
• Found better Electome images. More project info here and here: 

9:00 – 3:00 BRC

• Finished temporal coherence. Now we can compare across multiple cluster attempts. Tomorrow I’ll set up the cluster_optomizer to make multiple runs and produce an Excel file containing columns of cluster attempts

7:00 – 8:00

• Systematic exploration of unsupervised methods for mapping behavior
• Thinking about the stories I can tell with the GP sim.
  • Start together with same settings.
  • Disconnect
  • Slide exploit to max
• Need to download blog entries
• Working on graphing. Success!!!!! Now I need to discriminate agents from clusters, and exploit from explore. But this shows polarized vs. diverse clustering. I’m pretty sure I can get all kinds of statistics out of this too!
• Better version. Ran all the permutations:
• explore_1.6_exploit_3.2_ran 04_14_17-08_38_48. Green are clusters, Red are Exploit, Blue are Explore
• Need to make the line width based on the time spent in the cluster, and the cluster size a function of its lifespan

9:00 – 5:00 BRC

• Working on showing where the data broke. Looks like Talend
• For future referrence, How to turn a dict of rows into a DataFrame and then how to access all the parts:

  import pandas as pd
  
  d1 = {'one':1.1, 'two':2.1, 'three':3.1}
  d2 = {'one':1.2, 'three':3.2}
  d3 = {'one':1.3, 'two':2.3, 'three':3.3}
  rows = {'row1':d1, 'row2':d2}
  rows['row3'] = d3
  df = pd.DataFrame(rows)
  df = df.transpose()
  print(df)
  
  for index, row in df.iterrows():
      print(index)
      for key, val in row.iteritems():
          print("{0}:{1}".format(key, val))

• Helped Aaron with the writeups
• And it turns out that all the work I did “could be done in an hour”. So back to clustering and AI work. If there is a problem with the data, I know that it works with the test data. Others can figure out where the problem is, since they can handle it so quickly.

7:00 – 8:00 Research

• More clustering. Here’s the list of agents by clusters. An OPEN state means that the simulation finished with agents in the cluster. Num_entries is: the lifetime of the cluster. For these runs, the max is 200. Id is the ‘name’ of the cluster. Tomorrow, I’ll try to get this drawn using networkx.

  timeline[0]:
  Id = cluster_0
  State = ClusterState.OPEN
  Num entries = 200
  {'ExploitSh_52', 'ExploreSh_43', 'ExploitSh_56', 'ExploreSh_2', 'ExploreSh_5', 'ExploitSh_73', 'ExploitSh_95', 'ExploreSh_19', 'ExploreSh_4', 'ExploitSh_87', 'ExploitSh_76', 'ExploreSh_3', 'ExploitSh_93', 'ExploreSh_32', 'ExploreSh_41', 'ExploreSh_17', 'ExploitSh_88', 'ExploitSh_77', 'ExploreSh_39', 'ExploitSh_85', 'ExploreSh_40', 'ExploitSh_64', 'ExploreSh_34', 'ExploreSh_22', 'ExploitSh_99', 'ExploreSh_1', 'ExploitSh_97', 'ExploitSh_69', 'ExploreSh_29', 'ExploitSh_58', 'ExploitSh_62', 'ExploreSh_23', 'ExploreSh_36', 'ExploreSh_11', 'ExploitSh_80', 'ExploitSh_82', 'ExploreSh_21', 'ExploitSh_75', 'ExploitSh_72', 'ExploitSh_89', 'ExploitSh_86', 'ExploreSh_37', 'ExploitSh_84', 'ExploitSh_81', 'ExploreSh_15', 'ExploitSh_51', 'ExploreSh_44', 'ExploitSh_83', 'ExploitSh_94', 'ExploreSh_16', 'ExploitSh_53', 'ExploitSh_67', 'ExploitSh_74', 'ExploreSh_45', 'ExploreSh_26', 'ExploreSh_12', 'ExploreSh_13', 'ExploitSh_92', 'ExploreSh_9', 'ExploreSh_28', 'ExploitSh_50', 'ExploreSh_8', 'ExploreSh_30', 'ExploreSh_49', 'ExploitSh_59', 'ExploitSh_57', 'ExploreSh_42', 'ExploitSh_65', 'ExploitSh_54', 'ExploitSh_61', 'ExploitSh_66', 'ExploitSh_55', 'ExploitSh_78', 'ExploitSh_68', 'ExploitSh_79', 'ExploitSh_91', 'ExploitSh_71', 'ExploreSh_7', 'ExploitSh_98', 'ExploitSh_60', 'ExploitSh_70', 'ExploreSh_10', 'ExploitSh_90', 'ExploreSh_46', 'ExploitSh_96', 'ExploreSh_47', 'ExploitSh_63'}
  
  timeline[1]:
  Id = cluster_1
  State = ClusterState.OPEN
  Num entries = 200
  {'ExploreSh_25', 'ExploreSh_6', 'ExploreSh_38', 'ExploreSh_43', 'ExploreSh_49', 'ExploreSh_1', 'ExploreSh_2', 'ExploreSh_20', 'ExploreSh_33', 'ExploreSh_48', 'ExploreSh_5', 'ExploreSh_29', 'ExploreSh_15', 'ExploreSh_42', 'ExploreSh_24', 'ExploreSh_19', 'ExploreSh_4', 'ExploreSh_44', 'ExploreSh_16', 'ExploreSh_23', 'ExploreSh_36', 'ExploreSh_11', 'ExploreSh_3', 'ExploreSh_27', 'ExploreSh_35', 'ExploreSh_32', 'ExploreSh_17', 'ExploreSh_26', 'ExploreSh_21', 'ExploreSh_12', 'ExploreSh_18', 'ExploreSh_45', 'ExploreSh_41', 'ExploitSh_79', 'ExploreSh_13', 'ExploreSh_0', 'ExploreSh_39', 'ExploreSh_7', 'ExploreSh_9', 'ExploreSh_28', 'ExploreSh_40', 'ExploreSh_31', 'ExploreSh_10', 'ExploreSh_46', 'ExploreSh_37', 'ExploreSh_14', 'ExploreSh_47', 'ExploreSh_8', 'ExploreSh_30', 'ExploreSh_34', 'ExploreSh_22'}
  
  timeline[2]:
  Id = cluster_2
  State = ClusterState.CLOSED
  Num entries = 56
  {'ExploreSh_25', 'ExploreSh_1', 'ExploreSh_33', 'ExploreSh_29', 'ExploreSh_5', 'ExploreSh_48', 'ExploreSh_15', 'ExploreSh_19', 'ExploreSh_36', 'ExploreSh_3', 'ExploreSh_11', 'ExploreSh_35', 'ExploreSh_45', 'ExploreSh_17', 'ExploreSh_26', 'ExploreSh_41', 'ExploitSh_79', 'ExploreSh_13', 'ExploreSh_9', 'ExploreSh_40', 'ExploreSh_31', 'ExploreSh_37', 'ExploreSh_47', 'ExploreSh_30', 'ExploreSh_22'}
  
  timeline[3]:
  Id = cluster_3
  State = ClusterState.CLOSED
  Num entries = 16
  {'ExploreSh_25', 'ExploreSh_6', 'ExploreSh_43', 'ExploreSh_2', 'ExploreSh_48', 'ExploreSh_5', 'ExploreSh_15', 'ExploreSh_42', 'ExploreSh_24', 'ExploreSh_4', 'ExploreSh_44', 'ExploreSh_3', 'ExploreSh_26', 'ExploreSh_17', 'ExploreSh_41', 'ExploreSh_21', 'ExploreSh_32', 'ExploreSh_13', 'ExploreSh_9', 'ExploreSh_7', 'ExploreSh_28', 'ExploreSh_37', 'ExploreSh_8', 'ExploreSh_30', 'ExploreSh_49', 'ExploreSh_22'}
  
  timeline[4]:
  Id = cluster_4
  State = ClusterState.CLOSED
  Num entries = 30
  {'ExploreSh_6', 'ExploreSh_1', 'ExploreSh_2', 'ExploreSh_20', 'ExploreSh_33', 'ExploreSh_48', 'ExploreSh_15', 'ExploreSh_24', 'ExploreSh_4', 'ExploreSh_16', 'ExploreSh_23', 'ExploreSh_3', 'ExploreSh_11', 'ExploreSh_26', 'ExploreSh_41', 'ExploreSh_17', 'ExploreSh_32', 'ExploreSh_18', 'ExploreSh_13', 'ExploreSh_9', 'ExploreSh_46', 'ExploreSh_37', 'ExploreSh_8', 'ExploreSh_30', 'ExploreSh_49', 'ExploreSh_22'}
  
  timeline[5]:
  Id = cluster_5
  State = ClusterState.CLOSED
  Num entries = 28
  {'ExploreSh_25', 'ExploreSh_43', 'ExploreSh_2', 'ExploreSh_48', 'ExploreSh_29', 'ExploreSh_42', 'ExploreSh_24', 'ExploreSh_4', 'ExploreSh_44', 'ExploreSh_36', 'ExploreSh_35', 'ExploreSh_45', 'ExploreSh_17', 'ExploreSh_26', 'ExploreSh_12', 'ExploreSh_0', 'ExploreSh_28', 'ExploreSh_40', 'ExploreSh_31', 'ExploreSh_46', 'ExploreSh_37', 'ExploreSh_14', 'ExploreSh_47', 'ExploreSh_8', 'ExploreSh_30', 'ExploreSh_22'}
  
  timeline[6]:
  Id = cluster_6
  State = ClusterState.CLOSED
  Num entries = 10
  {'ExploreSh_40', 'ExploreSh_25', 'ExploreSh_18', 'ExploreSh_27', 'ExploreSh_10', 'ExploreSh_13', 'ExploreSh_20', 'ExploreSh_0', 'ExploreSh_37', 'ExploreSh_14', 'ExploreSh_36', 'ExploreSh_11', 'ExploreSh_39', 'ExploreSh_42', 'ExploreSh_22'}
  
  timeline[7]:
  Id = cluster_7
  State = ClusterState.CLOSED
  Num entries = 9
  {'ExploreSh_38', 'ExploreSh_2', 'ExploreSh_4', 'ExploreSh_46', 'ExploreSh_16', 'ExploreSh_33', 'ExploreSh_47', 'ExploreSh_14', 'ExploreSh_11', 'ExploreSh_27', 'ExploreSh_35', 'ExploreSh_45'}
  
  timeline[8]:
  Id = cluster_8
  State = ClusterState.CLOSED
  Num entries = 25
  {'ExploreSh_21', 'ExploreSh_38', 'ExploreSh_19', 'ExploreSh_2', 'ExploreSh_13', 'ExploreSh_44', 'ExploreSh_1', 'ExploreSh_10', 'ExploreSh_16', 'ExploreSh_47', 'ExploreSh_5', 'ExploreSh_48', 'ExploreSh_42', 'ExploreSh_35', 'ExploreSh_22', 'ExploreSh_32'}
  
  timeline[9]:
  Id = cluster_9
  State = ClusterState.OPEN
  Num entries = 16
  {'ExploreSh_17', 'ExploreSh_6', 'ExploreSh_24', 'ExploreSh_19', 'ExploreSh_10', 'ExploreSh_20', 'ExploreSh_46', 'ExploreSh_33', 'ExploreSh_14', 'ExploreSh_3', 'ExploreSh_39', 'ExploreSh_7', 'ExploreSh_45'}

• Network Dynamics and Simulation Science Laboratory – need to go through publications and venues for these folks
• Dynamic Spirals Put to Test: An Agent-Based Model of Reinforcing Spirals Between Selective Exposure, Interpersonal Networks, and Attitude Polarization
  • Within the context of partisan selective exposure and attitude polarization, this study investigates a mutually reinforcing spiral model, aiming to clarify mechanisms and boundary conditions that affect spiral processes—interpersonal agreement and disagreement, and the ebb and flow of message receptions. Utilizing agent-based modeling (ABM) simulations, the study formally models endogenous dynamics of cumulative processes and its reciprocal effect of media choice behavior over extended periods of time. Our results suggest that interpersonal discussion networks, in conjunction with election contexts, condition the reciprocal effect of selective media exposure and its attitudinal consequences. Methodologically, results also highlight the analytical utility of computational social science approaches in overcoming the limitations of typical experimental and observations studies.

8:30 – 5:30 BRC

• Logical Graphs: Control Flow in TensorFlow – Sam Abrahams (slides)
• Went digging through the input data. It is *not* the same. Generated lots of data
• When checking, we can tell that the data in the database is the same as it was in January
• Now looking to see if the data on CI is good

7:00 – 8:00 Research

• INTEL-SA-00075 vulnerability! Download and run Intel-SA-00075-GUI!
• A good weekend off. Big, cathartic 88 mile ride on Sunday, and the Kinetic Sculpture race on Saturday
• Working on the cluster visualization. Updating Intellij at home first
  • installed networkx
  • networkx_tutorial (Code from this post)is working
  • installed xlrd
  • membership_history_builder is working
  • Working on printing out the memberships, then I’ll start diagramming
• Thinking about how to start Thursday. I think I’ll try reading in blogs to LMN and show differences between students. then bring up flocking, then go into the material

8:30 – 4:00 BRC

• Analyzing data
• Showed Aaron the results on the generated and actual data. He’s pretty happy
  • Column mismatches between January and current data
  • Present in Jan data, but not in May:
    • First Excel crash of the day
    • Got the column difference working. It’s pretty sweet, actually:

      df1_cols = set(df1.columns.values)
      df2_cols = set(df2.columns.values)
      
      diff_cols = df2_cols ^ df1_cols

      That’s it.

    • Generated a report on different columns. Tomorrow I need to build a reduced DataFrame that has only the common columns, sort both on column names and then iterate to find the level of similarity.
  • Something’s wrong with?

    calc_naive_fitness_landscape()

Research 7:00 – 8:00

• Some interesting books:
  • Facing the Planetary: Entangled Humanism and the Politics of Swarming: Connolly focuses on the gap between those regions creating the most climate change and those suffering most from it. He addresses the creative potential of a “politics of swarming” by which people in different regions and social positions coalesce to reshape dominant priorities.
  • Medialogies: Reading Reality in the Age of Inflationary Media:  The book invites us to reconsider the way reality is constructed, and how truth, sovereignty, agency, and authority are understood from the everyday, philosophical, and political points of view.
  • At the Crossroads: Lessons and Challenges in Computational Social Science With tools borrowed from Statistical Physics and Complexity, this new area of study have already made important contributions, which in turn have fostered the development of novel theoretical foundations in Social Science and Economics, via mathematical approaches, agent-based modelling and numerical simulations. [free download!]
• Finished Online clustering, fear and uncertainty in Egypt’s transition. Notes are here
• The compass within Head direction cells have been hypothesized to form representations of an animal’s spatial orientation through internal network interactions. New data from mice show the predicted signatures of these internal dynamics. 
  • I wonder if these neurons are fired when information orientation changes?

8:30 – 3:00 BRC

• Giving up on graph-tool since I can’t get it installed. Trying plotly next. Nope. Expensive and too html-y. Networkx for the win? Starting the tutorial
  • Well this is really cool: You might notice that nodes and edges are not specified as NetworkX objects. This leaves you free to use meaningful items as nodes and edges. The most common choices are numbers or strings, but a node can be any hashable object (except None), and an edge can be associated with any object x using G.add_edge(n1,n2,object=x).
  • Very nice. And with this, I am *done* for the week:

    import networkx as nx
    import matplotlib.pyplot as plt
    
    #  Create the graph
    G=nx.Graph(name="test", creator="Phil")
    
    #  Create the nodes. Can be anything but None
    G.add_node("foo")
    G.add_node("bar")
    G.add_node("baz")
    
    #  Link edges to nodes
    G.add_edge("foo", "bar")
    G.add_edge("foo", "baz")
    G.add_edge("bar", "baz")
    
    #  Draw
    #  Set the positions using a layout
    pos=nx.circular_layout(G) # positions for all nodes
    
    #  Draw the nodes, setting size transparancy and color explicitly
    nx.draw_networkx_nodes(G, pos,
                    nodelist=["foo", "bar"],
                    node_color='g',
                    node_size=300,
                    alpha=0.5)
    nx.draw_networkx_nodes(G, pos,
                    nodelist=["baz"],
                    node_color='b',
                    node_size=600,
                    alpha=0.5)
    
    #  Draw edges and labels using defaults
    nx.draw_networkx_edges(G,pos)
    nx.draw_networkx_labels(G,pos)
    
    #  Render to pyplot
    plt.show()
    
    print("G.graph = {0}".format(G.graph))
    print("G.number_of_nodes() = {0}".format(G.number_of_nodes()))
    print("G.number_of_edges() = {0}".format(G.number_of_edges()))
    print("G.adjacency_list() = {0}".format(G.adjacency_list()))

  • 
• Short term goals
  • Show that it works in reasonable ways on our well characterized test data
  • See how much clustering changes from run to run
  • Compare differences between manifold learning techniques
  • Examine how it maps to the individual user data

Star wars day

7:00 – 8:00, 4:00 – 6:00 Research

• Continuing Online clustering, fear and uncertainty in Egypt’s transition. Notes are here
• Meeting with Wayne
  • Current trajectory is good
    • HCIC poster with clusters
    • What to do July+? Build ResearchBrowser. Anything else?
    • Also, try to put together a summary in the blog before each meeting
  • Add Wayne as coauthor if we get through the next gate
  • Got to talk about the future of work. My perspective is that machines will be able to meet all needs essentially for free, so we need to build an economy on a human value-add, like the Bugatti Veyron. The goal is to support the creation of items/experiences with a human value-add and an economy built around that.

8:30 – 3:30

• Fixing all of the broken code on CI
• Migrated all the machine-learning python code so everything matches
• changed the algorithm from subdivision to naive
• Working on CI

  t-SNE: 15 sec
  New best cluster: EPS = 0.1, Cluster size = 3.0
  clusters = 17
  Total  = 1179
  clustered = 1042
  unclustered = 137
  
  Algorithm naive took 17.68 seconds to execute and found 17.0 clusters
  

• Fixed the cluster output so that it won’t save clusters that have impossible names

7:00 – 8:00 Research

• Still working on building a population map. I downloaded the population density from here, and a python library for looking at GeoTiff files (georasters). And the Wilbur terrain generator
• I think I’ll have the time to work with network graphs based on the temporal coherence work using the libraries mentioned in this post
• Continuing Online clustering, fear and uncertainty in Egypt’s transition. Notes are here
• Homophilic Clustering by Locally Asymmetric Geometry
• Social Identity Theory – provides grounding for antibelief and other flocking characteristics
• Interactive Narrative: An Intelligent Systems Approach 
  • Citations from Google Scholar

8:30 – 6:30 BRC

• Workshop on deep learning
• I think I’ll have the time to work with network graphs based on the temporal coherence work using the libraries mentioned in this post
  • Looking through graph-tool‘s documentation
  • First, add all the vertices, which are all the clusters and all the agents:

    v1 = g.add_vertex()

    Then, connect each agent to its clusters:

    e = g.add_edge(v1, v2)

    Then draw:

    graph_draw(g, vertex_text=g.vertex_index,output_size=(200, 200), output="two-nodes.png")

    After that, there seem to be all kinds of analytics

  • Aaron didn’t got to the conference, so we worked on rolling in all the chnges. The reducers work fantastically well, though there is a pile of testing that needs to be done.
  • And I learned that to get a row out of n numpy matrix, you do mat[row], rather than mat[row:]
  • Pretty pictures for the isomap run

    

    

    

    
    

7:00 – 8:00 Research

• Continuing Online clustering, fear and uncertainty in Egypt’s transition. Notes are here
• Transitions from Authoritarian Rule
  • Political science scholars consider the four-volume work Transitions from Authoritarian Rule to be a foundational text for studying the process of democratization, specifically in those cases where an authoritarian regime is giving way to some form of democratic government. The most important of the four books is without a doubt the fourth volume, Tentative Conclusions about Uncertain Democracies, also known as “the little green book.”Transitions from Authoritarian Rule was the first book in any language to systematically compare the process of transition from authoritarianism across a broad range of countries. Political democracy is not the only possible outcome. Guillermo O’Donnell, Philippe C. Schmitter, and Laurence Whitehead emphasize that it’s not the revolution but the transition that is critical to the growth of a democratic state. This ground-breaking insight remains highly relevant as the ramifications of the Arab Spring continue to play out.
• The Politics of Uncertainty: Sustaining and Subverting Electoral Authoritarianism (Oxford Studies in Democratization)
• HR-diagrams are a great example of a behavior map. I think I would even contend that a map can be defined as a diagram that supports the plotting of trajectories
• A self-organizing semantic map for information retrieval
• Computer method and system for topographical representation of textual data (patent – could be a problem for the ResearchBrowser?)
• When Deceitful Chats Look Truthful (Self organizing maps!)
• Fear and Loathing across Party Lines: New Evidence on Group Polarization
• Prior Exposure Increases Perceived Accuracy of Fake News

8:30 – 2:30 BRC

• Got the reducers in, now I need to colorize the original df for display. Done. The results aren’t great though. Below are results for isomap:

  

  

  

  
  

  

  
  

  The images show

  • The XY positions of the reduced data. I’ve added a bit of jitter so it’s possible to see all the points. They should be pretty evenly distributed, but as you can see, the lower right has a much greater population.
  • This is backed up by the color mapped images of the original clusters, were the majority of the rows are black, and the other values are all in the bottom-right square
  • The 3D fitness landscape made via subsurface shown in 3D
  • and in 2D
• A roughly similar run (and yes, they vary a lot!) is shown with a brute-force (naive)surfacer. Actually, it may makes sense to use the naive surfacer on the reduced data since it’s so much faster:

  

  

  

  
  

  

  
  

7:00 – 8:00, 3:00 – 4:00 Research

• Rita Allen Foundation – June 30 deadline for proposals
• On the power of maps: Electoral map spurred Trump’s NAFTA change of heart
• The Neural Basis of Map Comprehension and Spatial Abilities
• Neurobiological bases of reading comprehension: Insights from neuroimaging studies of word level and text level processing in skilled and impaired readers
• Reading Online clustering, fear and uncertainty in Egypt’s transition
  • Marc Lynch (webpage),
  • Deen Freelon (webpage) associate professor in the School of Communication at American University in Washington, DC. My primary research interests lie in the changing relationships between technology and politics, and encompass the study of weblogs, online forums, social media, and other forms of interactive media with political applications. Collecting and analyzing large amounts of such data (i.e. millions of tweets, Facebook wall posts, etc.) require methods drawn from the fields of computer science and information science, which I am helping to adapt to the long-standing interests of political communication research.
    • Social media data collection tools
  • Sean Aday (From GWU) focuses on the intersection of the press, politics, and public opinion, especially in relation to war and foreign policy. He has published widely on subjects ranging from the effects of watching local television news to coverage of Elizabeth Dole’s presidential run to media coverage of the wars in Iraq and Afghanistan.Before entering academia, Dr. Aday served as a general assignment reporter for the Kansas City Star in Kansas City, MO; the Milwaukee Journal in Milwaukee, WI; and the Greenville News in Greenville, SC. He graduated from the Medill School of Journalism at Northwestern University in 1990.
  • …research has demonstrated the role played by social media in overcoming the transaction costs associated with organizing collective action against authoritarian regimes, in temporarily empowering activists against state violence, in transmitting images and ideas to the international media, and in intensifying the dynamics of social mobilization.
    • There is some kind of relationship between frictionlessness and credibility. Disbelief is a form of friction that needs to be overcome.
  • We argue that social media tends to exacerbate and intensify those factors which make failure more likely than in comparable cases which did not feature high levels of social media usage. Social media promotes the clustering of individuals into communities of the likeminded, and that these clusters have distinctly damaging implications during uncertain transitions.
    • I would add “as designed”, but uncertainty sets up an entirely different dynamic, which I doubt the designers took into account.
  • Users within these clusters tend to be exposed primarily to identity-confirming and enemy-denying information and rhetoric, which encourages the consolidation of in-group solidarity and out-group demonization. The speed, intensity, and intimacy of social media tends to exacerbate polarization during moments of crisis, and then to entrench very different narratives about those events in the aftermath.

8:30 – 2:30 BRC

• Aaron’s and Bob’s grandmother’s passed away on Saturday. Aside from the important stuff which I can’t do anything about, there is the urgent issue about how to deal with the sprint impacts
• HIPAA training!
• Which machine learning algorithm should I use? 
• Social media data collection tools
• I got blindsided by reference rather than value. I built a dictionary that contained all the information about an attempt, but it was saving the references, which meant all the entries were the same, so no performance data! So, to ‘update’ an array in a way that maintains a reference to the old data, you need to do it like this:

  min_max_c = [min_max_c[MIN], mid_c]

• And we get some nice pictures. The fit is better too:

  

  

  

  
  

  Results!

  256x256
  Algorithm subdivision took 3.12 seconds to execute and found 9.0 clusters
  Algorithm naive took 7.00seconds to execute and found 8.0 clusters
  
  512x512
  Algorithm subdivision took 12.17seconds to execute and found 16.0 clusters
  Algorithm naive took 30.15 seconds to execute and found 13.0 clusters

• Starting to fold in Aaron’s code

  if args.reducer:
      lm = ManifoldLearning()
      if args.reducer == 'lle':
          mat = lm.lle(df.as_matrix())
      elif args.reducer == 'isomap':
          mat = lm.isomap(df.as_matrix())
      elif args.reducer == 'mds':
          mat = lm.mds(df.as_matrix())
      elif args.reducer == 'spectral':
          mat = lm.spectral_embedding(df.as_matrix())
      elif args.reducer == 'tsne':
          mat = lm.tsne(df.as_matrix())
      df = pd.DataFrame(mat, index=df.index.values, columns=['X', 'Y']) #  Assume 2D???

• Fika Ali’s presentation
  • What is O&M training?
  • Move the mechanism up front? I was wondering what the device was
  • Paraphrasing scenarios is ok
  • Example of finding? A specific error with a response and how it was coded?
  • ‘Perpetuating stigma’ text too far indented
  • Designers Should Also?
  • slide 25 are critical should be is critical
  • Same error should be some error
  • Overall
    • It’s a lot of words. More pictures?
    • The icon works, but maybe is a little confusing
    • Helena – don’t we already know this? THe contextual issues is de-emphasized
    • William, what does the literature say on adoption? Add a brief overview of previous work. Particularly in public places? The contribution is context.
    • Stacy lean heavily on facial recognition literature. THis can show why accuracy may b overweighted.
    • Amy – focus on the bigger points. Do the hook first. Scenarios that would make things obvious. Walking into the wrong bathroom.
    • Phil – figuring out context is hard! How do you do that?
    • Amy – too heavy on process, and not enough on motivations. Lean on the quotes, they tell a better story. Fewer than 5 slides are motivations. Add an outline so they know what’s coming up. So people can know howm much time to devote to emails
    • Helena ‘You can read the details in the paper’
    • Stacy – I want to hear be excited about your talk
    • Amy, Stacy – make a recording to listen to. Pay attention to pacing, pauses, etc.

7:00 – 8:00 Research

• Submitted the paper. Need to start on the HCIC poster next
• Continuing Examining the Alternative Media Ecosystem through the Production of Alternative Narratives of Mass Shooting Events on Twitter. Notes are here. Finished!
• The menace of unreality: How the Kremlin weaponizes information, culture and money

8:30 – 4:30 BRC

• Working on finding an exit condition for the subdivision surface
• I’m currently calculating the corners of a constricting rectangle that contracts towards the best point. Each iteration is saved, and I’m working on visualizing that surface, but my brain has shut down, and I can do simple math anymore.
• Had a thought for Aaron about how to visualize his dimension reduction. Turns out to do well.

7:00 – 9:00 Research

• Some more echo chamber flocking: Iran Deal Is More Popular Than Ever, Poll Shows Republicans registered the biggest uptick in support for the deal, which has been heavily criticized by GOP lawmakers since its inception in July 2015: 53 percent of Republican voters said they supported it, compared with 37 percent who backed it last summer and just 10 percent who supported it shortly after it was announced. Democratic support for the deal has been largely unchanged since August, and a larger share of independents are getting on board, from 41 percent in August to 48 percent now.
• Finishing corrections to paper
• This really is my phase I research question: If ‘laws of motion’ can indeed be ascribed to behavior, we should be able to model the effects of those laws. The question them becomes what form do these models take? Also, how do we detect these behaviors with domain independence and at scale?
• Submitted!
• The Relevance of Hannah Arendt’s Reflections on Evil:Globalization and Rightlessness

BRC 9:30 5:00

• Continuing Subdivision surfacing
• didn’t like the documentation on sortedcollections. going to try panda Series
• Allowable options in an arg:

  parser.add_argument("--algorithm", type=str, choices=['naive', 'subdivision', 'genetic'], default='naive', help="hill climbing algorithm")

  Note that range(), which returns a list should also work

• And here’s how you get the key/values from a pandas Series:

  print("calc_subdivision_fitness_landsape(): key = {0}, val = {1}".format(fitness.index[0], fitness.values[0]))

• Looks like it’s working. I think I should be using the average of the 4 fitnesses to decide if I’m done

  calc_subdivision_fitness_landsape(): fitness = 
  1    10.0
  0     7.0
  3     6.0
  2     6.0
  dtype: float64
  calc_subdivision_fitness_landsape(): fitness = 
  1    10.0
  0     7.0
  3     6.0
  2     6.0
  dtype: float64
  calc_subdivision_fitness_landsape(): fitness = 
  1    10.0
  0     7.0
  3     6.0
  2     6.0
  dtype: float64
  done calc_subdivision_fitness_landsape

7:00 – 8:30 Research

• Proofreading and tweaking the CSCW paper.
• Finished the paper edit. Started to roll in the changes
• Made a 10D chart of the explorer probability distribution. I think it tells the story better:
•  
• Had to install a dictionary in TexStudio. This helped a lot.
• Started rolling in the changes to the tex file

BRC 9:00 – 4:30

• Looks like the sort changes to the data_generator.py code haven’t been pushed yet
• Starting on subdivision surfacing

  def calc_subdivision_fitness_landsape(self, eps_step: float, min_cluster: int) -> pandas.DataFrame:
      # create the four extreme corners. These will work their way in
      # calculate halfway points
      # keep the square with the greatest (single? average?) value
      # repeat until an epsilon, max value, or max iterations are reached
      # construct a sparse matrix with spacing equal to the smallest spacing
      # fill in the values that have been calculated
      # build a dataframe and return it for visualization

• I need to sort a dict, so I’m trying SortedContainers.
• Then things went off the hails a bit, and I wrote a haiku program as a haiku that prints itself:

  def haiku(sequence):
      this_is_not_needed = ""
      return "".join(sequence)
  
  if __name__ == "__main__":
      f = open('haiku.py')
      print(haiku(f.readlines()[:3]))

7:00 – 8:30 Research

• Wikipedia founder Jimmy Wales launches Wikitribune, a large-scale attempt to combat fake news
• Listening to the BBC Business Daily on Machine Learning. They had an interview with Joanna J Bryson (Scholar). She has an approach for explaining the behavior of AI that seems to involve simulation? Here are some papers that look interesting:
  • Behavior Oriented Design (MIT Dissertation: Intelligence by Design: Principles of Modularity and Coordination for Engineering Complex Adaptive Agents)
  • Learning from Play: Facilitating character design through genetic programming and human mimicry
    • Mimicry and play are fundamental learning processes by which individuals can acquire behaviours, skills and norms. In this paper we utilise these two processes to create new game characters by mimicking and learning from actual human players. We present our approach towards aiding the design process of game characters through the use of genetic programming. The current state of the art in game character design relies heavily on human designers to manually create and edit scripts and rules for game characters. Computational creativity approaches this issue with fully autonomous character generators, replacing most of the design process using black box solutions such as neural networks. Our GP approach to this problem not only mimics actual human play but creates character controllers which can be further authored and developed by a designer. This keeps the designer in the loop while reducing repetitive labour. Our system also provides insights into how players express themselves in games and into deriving appropriate models for representing those insights. We present our framework and preliminary results supporting our claim.
  • Replicators, Lineages and Interactors: One page note on cultural evolution
    • If we adopt the other option and refer to culture itself is the lineage, then the culture itself can evolve since the replicators are the ideas and practices that exist within that culture. However, if it is the culture that is the lineage, we cannot say that it evolves when it takes more territory, in the same way that a species does not evolve with more individuals. Adaptation is presently understood to be about changes in the frequency of replicators, not about absolute numbers of interactors. In sum, cultural evolution (changes of practices within a group) is necessarily a separate process from cultural group selection (changes of the frequency of group-types at a specific location).
  • The behavior-oriented design of modular agent intelligence
  • Should probably cite some of these and a reference to Behavior-Oriented Design in the conclusions section of the paper
• Continuing Examining the Alternative Media Ecosystem through the Production of Alternative Narratives of Mass Shooting Events on Twitter
  • We collected data using the Twitter Streaming API, tracking on the following terms (shooter, shooting, gunman, gunmen, gunshot, gunshots, shooters, gun shot, gun shots, shootings) for a ten-month period between January 1 and October 5, 2016. This collection resulted in 58M total tweets. We then scoped that data to include only tweets related to alternative narratives of the event—false flag, falseflag, crisis actor, crisisactor, staged, hoax and “1488”.
    • These keywords specify a ‘primary information space’. Bag-of-words of text correlated with each term could make this a linear axis
  • Of 15,150 users who sent at least one tweet with a link, only 1372 sent (over the course to the collection period) tweets citing more than one domain.
    • This is the difference between implicit behaviors (clicking, reading, navigating) and explicit actions. Twitter monitors what people are willing to write
  • Interestingly, the two most influential Domains in Alternative Narrative Tweets Interesting, the two most highly tweeted domains were both associated with significant automated account or “bot” activity. The Real Strategy, an alternative news site with a conspiracy theory orientation, is the most tweeted domain in our dataset (by far). The temporal signature of tweets citing this domain reveals a consistent pattern of coordinated bursts of activity at regular intervals generated by 200 accounts that appear to be connected to each other (via following relationships) and coordinated through an external tool.
    • There is clearly a desire to have a greater effect through the use of bots. Two questions: 1) How does this work? 2) How did this emerge?
  • The InfoWars domain, an alternative news website that focuses on Alt-Right and conspiracy theory themes, was the second-most tweeted domain, but as (Figure 1) shows it was only tenuously connected to one other node.
    • Why? Is InforWars more polarized? Is it using something other than Twitter?
    • Infowars Inbound links
      

      Domain score	Domain trust score	Domain	Backlinks	IP Address	Country	First seen	Last seen
      0	0	breakingnewsfeed.com	1857029	174.129.22.101	us	2015-09-28	2017-03-26
      4	4	e-graviton.com	1335835	67.210.126.35	us	2014-01-19	2017-03-21
      33	39	prisonplanet.com	648958	69.16.175.42	us	2013-06-07	2017-03-25
      1	0	nwostop.com	346153	104.28.28.16	us	2014-01-19	2017-03-21
      13	31	nwoo.org	182060	81.0.208.215	cz	2013-06-07	2017-03-26
      12	30	conservative-headlines.com	151778	104.18.50.72	us	2016-06-27	2017-03-22
      1	0	america2fear.com	92766	69.64.46.138	us	2014-11-14	2017-03-23
      4	29	subbmitt.com	49288	64.251.23.173	us	2015-02-04	2017-03-26
      14	30	anotherdotcom.com	47195	174.129.236.72	us	2014-10-02	2017-03-20
      1	0	exzacktamountas.com	43748	208.100.60.13	us	2016-06-08	2017-03-24

9:00 – 5:30 BRC

• John is having trouble getting Linux running on the laptop
  • No luck. Re-submitting for an Alienware deskside
• Back to getting the temporal coherence. last try to finish up, then switching to fitness landscape optimization, which I dreamed about last night
• Finished coherence! Had to include a state check for a timeline to see if a DIRTY state had been touched with an update. If not, then the timeline is set to CLOSED. If a new cluster appears that would have had some overlap, a new timeline is created anyway. This could be an optional behavior.
  • Still need to test rigorously across multiple data sets
• Long scrum, then ML meeting.
  • Hard tasks
    • TF server set up to work in our environment
    • Pre-calculated models to speed up training from research browser
    • T-SNE or other mapping of returned CSE text to support exploration
    • Fast, on-the-fly classification and entity extraction within the research browser framework. Plus interactive training
    • NMF (or other) topic extraction tied to human labeling and curation, plus cross-user validation of topics
• Poster with Aaron later? Yep. Couple of hours. Done?
• Oh, just why? Spent an hour on this before going brute force:

  def get_last_cluster(self) -> ClusterSample:
      # return self._cluster_dict[self._cluster_dict.keys()[-1] TODO: This should work
      toReturn = None
      for key in self._cluster_dict:
          toReturn = self._cluster_dict[key]
      return toReturn

• Walked through some gradient descent regression code with Bob. More tomorrow?
• Got the new sort working with Aaron. Much faster progress as a pair

7:00 – 8:00, 3:00 – 4:00  Research

• Continuing to tweak paper
• Starting Examining the Alternative Media Ecosystem through the Production of Alternative Narratives of Mass Shooting Events on Twitter
  • From the introduction. Do I need something like this? Our contributions include an increased understanding of the underlying nature of this subsection of alternative media — which hosts conspiratorial content and conducts various anti-globalist political agendas. Noting thematic convergence across domains, we theorize about how alternative media may contribute to conspiratorial thinking by creating a false perception of information diversity.
• Conspiracy Theories
  • Cass R. Sunstein
  • Adrian Vermeule
• A cool thing on explorers and veloviewer:  Here’s an overview of the project
• Brownbag
  • Teaching abstract concepts to children and tweens (STEM)
  • Cohesive understanding of science over time
  • Wearable technology as the gateway for elementary-school-aged kids? Research shows that they find them valuable
  • How are these attributes measured? <——-!!!!!!!
  • Live visualization sensing and visualization
  • Zephyr bioharness
  • Gender/age differences? Augmented reality? Through-phone?
  • leylanorooz.com/research/

8:30 – 2:30 BRC

• Expense report! Done? Had to get a charge number, and re enter. Took forever.
• Found out that I’m getting a laptop rather than what I asked for
  • Having John install Ubuntu and verify that multiple monitors work in Linux
• Helped Bob set up Git repo
• Still working on temporal coherence. Think I’ve figured out the logic. Now I need to set clusters in ClusterTimelines
• Learned how to do Enums in Python