Vacation (Strava links below)

1. Day 1 – 37 Miles (To Forest Grove)
2. Day 2 – 83 Miles, parts one and two (To Astoria)
3. Day 3 – 69 Miles to Tillamook
4. Day 4 – 81 Miles to Newport
5. Day 5 – 58 Miles to Florence
6. Day 6 – 79 Miles to Bandon
7. Day 7 – 80 Miles to Roseburg, parts one and two
8. Day 8 – 102 Miles to Eugene
9. Day 10 – 83 Miles to Willamette pass. Now that was a big hill
10. Day 11 – 79 Miles to Bend, parts one and two
11. Day 12 – 80 Miles from Bend to Detroit Lake
12. Day 13 – 93 Miles to Troutdale
13. Day 14 – 23 Miles to Portland

A grand total of about 947 Miles, or an average of 72 miles a day. Fun!

Also other interesting things:

• Exploring the Friendships, Rivalries, and Trysts between Ideas in Texts
• Also, and interesting quick take blog from Matt Kiser: whatthefuckjusthappenedtoday.com
• Google Scholar search on primate collective movement
• Coordination in Human and Primate Groups
• Decision-making processes: The case of collective movements
• Decision-Making in Przewalski Horses (Equus ferus przewalskii) is Driven by the Ecological Contexts of Collective Movements Via Ethology
• The Lens of time – looks at flocking behavior using high-speed cameras. Ian Couzin is involved

Thought from rest day one:

• It mak not be a map in the classical sense of a representation of fixed points. The representation may more resemble the detailed pictures that have been coming in from Cassinii recently. There is no ground, but there are large scale recognisable features (the Red Spot, bands, hexagonal poles, etc). The relationships between the features can change, but the patterns could be stochastic in the sense that there is a sustaining pattern, like a candle flame. From C&C:
  • Every kind of intellectual and affective means come into play as soon as the slightest hint of difference arises, so that arguments are invented, opposing views interpreted, to find the grounds for disagreement, but, above all to find a way out of them. [p 173]  
• For the simulation
  • The agents contribution is the heading and speed
  • The UI is the horizon
  • The IR is the stage
  • An additional part might be to add the ability to store data in the space. Then the behavior of the IR (e.g. empty areas) would b more apparent, as would the effects of UI (only certain data is visible, or maybe only nearby data is visible) Data could be a vector field in Hilbert space, and visualized as color.

Rest Day 2

• C&C
  • By causing the majority to converge towards a social representation, the multiplicity of decisions that lead to a consensus do more than draw viewpoints closer: they initiate or reinforce social ties. Beyond the specific dilemmas that each decision resolves, they correspond to this general aim. They produce a mass effect in the network of groups that choose and discuss, creating and re-creating the bonds in our society by a common action, just as at one time public opinion originated in the market-places and cafes, and from drawing-room conversations. It is the surplus value that we extract from this task of collective decision-making, the diversity and multiplicity of which have reached so high a level that it has become a profession -we speak of decision-makers – and a significant factor in our social and moral world. [p 174]
    • Is community building inherently an act of dimension reduction? We find commonalities and focus on those, while excluding other areas. It’s not necessarily about agree/disagree, but more like areas that we can create norms of behavior in. It’s easier to create norms in homogeneous, closed cultures, and harder in heterogeneous cultures. But though homogeneous cultures may be faster to respond, they are also less resilient. I’m thinking in particular of the Aspen, or any plant that reproduces asexually. The there is an initial advantage, but if conditions change, the plant goes extinct quickly.
  • The socio-cognitive conflict inherent in any decision taken in common combines together two permanent tendencies. The one aims at maintaining existing uniformity and agreement, the other at changing them by imparting an original form to things and ideas. The choices that are made usually express a balance of forces between the two because, without any element of novelty, they are mere stereotypes or ritual, and, without a dose of conformity, they become fancies and fluctuations that lead to disorder. It is of real theoretical interest to recognize at work in the socio-cognitive conflict a dual process of social thought. It can only be dual, in.view of the opposing functions that it fulfils and of the simultaneous use it makes of divergent and convergent thinking, the one the badge of innovation and the other of uniformity [p 174]
    • This is a very good example of the explore/exploit condition as it relates to social cognition. The same for sea turtles and people
  • To resolve the conflict by eliminating convergent thinking would be to abandon discussion and any choice made in common. To resolve the conflict by censoring divergent thinking would condemn participants to routine, to stereotyping, to what is termed ‘groupthink’. On the other hand, to negotiate this conflict, which is both social and cognitive, is an arduous task. [p 175]
    • There should be a way of factoring these values into the model. Making it easier or harder to ‘negotiate’ between explore and exploit. Need to look into the closed form solution to the two-armed bandit problem
  • The effects of a normative intervention on group decision-making performance
    • A space ship having crashed on the moon, a team of astronauts has to cover a distance of some 300 kilometres in order to reach the spot where they have a rendezvous with another team. Before embarking on this perilous undertaking, the members of the team have to decide which of the fifteen objects necessary for survival – oxygen reserves, concentrated food, signalling equipment, heating requisites, etc. – they will take with them. Those participating in the study were asked to draw up a list of priorities for these objects, first separately as individuals, and then in groups, by arriving at a consensus. Half of the groups received no special instructions for this common task of decision-taking. The other half were instructed to confront other members of the group and resolve the differences between them. They were informed that they had to set out their arguments lucidly and were not to change their opinion with the sole aim of avoiding conflict, nor to seek agreement using procedures such as a majority vote, the establishment of a mean position, bargaining, tossing a coin, or in other ways. Moreover, the instructions emphasized the need to look upon differences of opinion as both natural and useful, so that any precipitous agreement would be suspect, so long as the reasons for it had not been gone into thoroughly. The main thing was to resist group pressures that tempted one to yield to others without sound reasons, just in order to attain a consensus, which would lack any guarantee of success. [p 176]
      • Good game for study design?
    • Hall and Watson were persuaded that by weakening such pressures they would encourage divergent thinking to be displayed. This would lead groups first to produce solutions of a superior quality, and then to make better use of the resources of each member, so that overall performance would exceed even that of the group’s cleverest members, and finally to discover novel solutions. Most of these hypotheses were verified.
  • The other means of encouraging divergent thinking is by the presence of a minority participating actively in the group’s discussions. This is a comparatively natural means for use in decisions leading to a consensus, since it arises solely from the obligation to respect one of the essential conditions. These assume in fact a state of equality between the members of the group. This means the majority recognizes the right of the minority to express itself, and will set very great store by its opinions; otherwise the agreement arrived at would be worthless. [p 177]
  • (Moscovici and Zavalloni, 1969) looked for indications of this cognitive transformation of individuals in a group. It is no exaggeration to state that the decisions, in the broadest sense, prepare the way for it and make it their prime target. The students who participated, it will be recalled, had to adopt a common attitude towards de Gaulle and the Americans. In order to arrive at this, particularly if the attitude were unfavourable, they had to acknowledge a common set of values and a common code. This most possibly assumed , among the categories available to each individual, the use solely of those that corresponded to that code. These were the ones that were retained, that were used frequently, and towards which individuals would converge. In order to verify this we calculated an entropy index (H) of the distribution of categories, borrowed from information theory. [p180]
    • This looks like dimension reduction to me. It also implies a way of testing for it. The description for how they calculated entropy isn’t that clear from the book, but I’m guessing that it has something to do in the variability of the terms used. Combined with an awareness of sentiment, this might be a reasonable way to determine relevant axis of discussion

7:00 – 8:00 Research

• A quick thought about the overlapping of spatial and conceptual ‘space’ concepts. Dug up the studies that showed how physical warmth can cause social ‘warmth’ and some of the underlying neural structures
  • Experiencing Physical Warmth Promotes Interpersonal Warmth
    • “Warmth” is the most powerful personality trait in social judgment, and attachment theorists have stressed the importance of warm physical contact with caregivers during infancy for healthy relationships in adulthood. Intriguingly, recent research in humans points to the involvement of the insula in the processing of both physical temperature and interpersonal warmth (trust) information. Accordingly, we hypothesized that experiences of physical warmth (or coldness) would increase feelings of interpersonal warmth (or coldness), without the person’s awareness of this influence. In study 1, participants who briefly held a cup of hot (versus iced) coffee judged a target person as having a “warmer” personality (generous, caring); in study 2, participants holding a hot (versus cold) therapeutic pad were more likely to choose a gift for a friend instead of for themselves.
  • Shared neural mechanisms underlying social warmth and physical warmth.
    • Many of people’s closest bonds grow out of socially warm exchanges and the warm feelings associated with being socially connected. Indeed, the neurobiological mechanisms underlying thermoregulation may be shared by those that regulate social warmth, the experience of feeling connected to other people. To test this possibility, we placed participants in a functional MRI scanner and asked them to (a) read socially warm and neutral messages from friends and family and (b) hold warm and neutral-temperature objects (a warm pack and a ball, respectively). Findings showed an overlap between physical and social warmth: Participants felt warmer after reading the positive (compared with neutral) messages and more connected after holding the warm pack (compared with the ball). In addition, neural activity during social warmth overlapped with neural activity during physical warmth in the ventral striatum and middle insula, but neural activity did not overlap during another pleasant task (soft touch). Together, these results suggest that a common neural mechanism underlies physical and social warmth.
• Looked more closely at the NSF grants. I think the mapping fits the following:
• Information Integration and Informatics (III)
  • Technological advances have resulted in accelerating increases in size, diversity, and complexity of data in virtually all aspects of human endeavor. Our ability to gather data of all types greatly outstrips our cognitive capacity to use it,while scientific, technical, and societal advances are increasingly dependent on new insights, theories, and tools to exploit data effectively for timely delivery of relevant and accurate information and for knowledge discovery. The Information Integration and Informatics (III) program supports research to realize the full transformative potential of data, information, and knowledge in this increasingly digital and interconnected world.
  • III-funded projects may address data of unprecedented scale, complexity, and rate of acquisition, as well as issues of heterogeneity and complexity with innovative approaches and deep insights. Projects may support the diverse functionalities and processing needs for data, information, and knowledge from disparate and uncoordinated sources,or cope with the changing landscape of computing platforms at scales ranging from small mobile devices to potentially global-scale cloud and networked computing resources. Successful proposals should demonstrate effectiveness in dimensions such as scalability, interactivity, or scientific, technological, or societal impact.
  • III-funded projects should address contemporary applications of societal importance through advances in information integration and informatics. Projects may deal with one or more facets of the full knowledge life-cycle, including creation, acquisition, selection, storage, display, and preservation, use, and re-use of data, information, and knowledge for decision-making and action. Ultimately, the deep scientific insights and advanced technologies resulting from III-funded projects will transform the functions and uses of data, information, and knowledge in society.
  • More information on topics of interest to the III program is available at: https://www.nsf.gov/cise/iis/iii_pgm12.jsp.
• More C&C
  • …certain aspects of a situation that bear no relationship to one another, such as the dimensions of a room and the decisions taken in it, that is, aspects deemed to be ancillary and which apparently should not be significant, may in reality be determining factors. In other words, what one considers to be a factor of the world external to social relationships has indeed an action within them. [p 130]
    • Design matters in all kinds of ways, and can affect the outcome and quality of the result. Consider polling – focus group, personal home interview, phone, online. What kind of bias do these modalities introduce? There was more detail on p 131 as well
  • In the case where different opinions can be freely expressed, discussion allows each individual better to defend his own viewpoint and to become more involved in the activity of the group. Thus we expected that the consensus would stress the dominant tendency among its members – in short, be polarized. On the other hand, when the participants placed a distance between one another and paid attention to the manner in which they arrived at a decision, their attention was concentrated upon positional differences. The discussion of the latter could not involve them greatly. Moreover, they had no means of justifying or forcibly making their position prevail, and thus were thrown back on resolving their differences by a compromise. This is indeed what we observed. Consequently attention to rules of procedure, which direct discussion towards information regarding the responses, leads towards a decision of ‘the just mean’. Discussion directed towards content produces the opposite effect. [ p 136]
    • This makes me a little nervous about using Pro Publica congressional data, since the law is very process oriented. Still, the body does seem to be polarizing, and potentially in different ways. Republicans are moving to the right and also disrupting process. Democrats appear to be moving less and have been more focused on ‘standard procedure’. Something to consider is if this is a function of being the opposition party of the party in power. The incentive when in power is to do things, which may require more compromise, as opposed to being the opposition party, which is almost by definition, uncompromising.
  • Imposing time limits on discussions also leads to compromise, while having open-ended discussions tend to polarize. The time constraint acts as a limiter of possible options? [p 136-137]
  • …groups that had to deal with information according to a set method, following directions, took compromise decisions. By contrast, with groups who could deal with the information without being subject to any constraints, the decisions were more extreme than the individual decisions. In principle we have here a confirmation of what we had always supposed to be the case: a normalized form of participation, which involves individuals less, leads to a moderate decision. [p 138]

8:30 – 4:30 BRI

• Finished stories
• Built and uploaded MDS, NLP, Geocoder and GeoIngest
• Had a variety of problems getting GeoIngest tests to behave. I need to come up to speed on Spring, I think. At least if this is going to be a regular thing.

Finally a cool, lower humidity day! The windows are open and the AC is off.

7:00 – 8:00 Research

• The Ring Theory of Kvetching has a different, “kvetcher-centered” map
• Inglehart–Welzel cultural map of the world  
• The spread of fake news by social bots
  • The massive spread of fake news has been identified as a major global risk and has been alleged to influence elections and threaten democracies. Communication, cognitive, social, and computer scientists are engaged in efforts to study the complex causes for the viral diffusion of digital misinformation and to develop solutions, while search and social media platforms are beginning to deploy countermeasures. However, to date, these efforts have been mainly informed by anecdotal evidence rather than systematic data. Here we analyze 14 million messages spreading 400 thousand claims on Twitter during and following the 2016 U.S. presidential campaign and election. We find evidence that social bots play a key role in the spread of fake news. Accounts that actively spread misinformation are significantly more likely to be bots. Automated accounts are particularly active in the early spreading phases of viral claims, and tend to target influential users. Humans are vulnerable to this manipulation, retweeting bots who post false news. Successful sources of false and biased claims are heavily supported by social bots. These results suggests that curbing social bots may be an effective strategy for mitigating the spread of online misinformation.
• The Age of the Crowd: A Historical Treatise on Mass Psychology Looks like a mass audience book by Moscovici. The table of contents is very compelling:
  • Part I. The Study of the Masses:
    1. The individual and the masses;
    2. The revolt of the masses;
    3. What do we do when faced with the masses?;
    4. Eastern and western varieties of despotism;
  • Part II. Le Bon and the Fear of Cowards:
    1. Who was Gustave Le Bon?;
    2. The Machiavelli of mass societies;
    3. Four reasons for saying nothing;
    4. The discovery of the masses;
    5. Mass hypnosis;
    6. The mental life of crowds
  • Part III. The Crowd, Women and Madness:
    1. Collective matter: the impulsive and conservative crowd;
    2. Collective form: the dogmatic and utopian crowd;
    3. The leaders of the crowd;
    4. Charisma;
    5. The strategies of propaganda and mass suggestion;
    6. Conclusion;
  • Part IV. The Leader Principle:
    1. The paradox of mass psychology;
    2. Natural crowds and artificial crowds;
    3. The leader principle;
  • Part V. Opinion and the Crowd:
    1. Communication is the valium of the people;
    2. Opinion, the public and the crowd;
    3. The law of the polarisation of prestige;
    4. The Republic in France: from a democracy of the masses to a democracy of publics
  • Part VI. The Best Disciple of Le Bon and Tarde: Sigmund Freud:
    1. The black books of Dr Freud;
    2. From classical to revolutionary mass psychology;
    3. The three questions of mass psychology;
    4. Crowds and the libido;
    5. The origin of affective attachments in society;
    6. Eros and mimesis;
    7. The end of hypnosis;
  • Part VII. The Psychology of the Charismatic Leader:
    1. Prestige and charisma;
    2. The postulate of mass psychology;
    3. The primal secret;
  • Part VIII. Hypotheses About Great Men:
    1. ‘The man Moses’;
    2. The family romances of great men;
    3. Creating a people;
    4. Mosaic and totemic leaders;
  • Part IX. Secular Religions:
    1. The secret of a religion;
    2. The prohibition of thought;
    3. The cult of the father;
  • Conclusion: the planetary age of the crowd;
• More C&C
  • Inevitably we will end up by acknowledging the place of values in a world of information. Indeed the problem is to know which items of information to use ‘and which to discard, in order to arrive at a particular agreement. [p 122]
    • To restate, how to perform manifold reduction from a high dimensional, incomprehensible problem to something that is a more manageable size. This can include combining dimensions, deleting dimensions and synthesizing new dimensions that may or may not have a direct connection to the problem but seem relevant (Blindly following a leader, for example).
  • We may say that values are their vertical and horizontal coordinates: Even if they are not rational, they make reason possible, reason which for us has – and we must not forget it- the meaning of a value or norm, the highest norm to which our society pays homage. Hence the requirement that is emphasized by two specialists in decision-making; it is ‘a theory of rational consensus and commitment when, in fact there is a dissensus‘ (Lehrer and Wagner, 1982: 4). To fulfil this need, such a theory, as we have demonstrated, must establish a relationship between these ideas. [p 122]
    • Self-organizing value maps, with axis based on dissensus measures. I think this is what I’m aiming for. And I think that dissensus could be measured by looking for the broadest range of sentiment applied to words, phrases or entities. These axis may be organizable using betweenness centrality measures that keep like axis together.

8:30 – 5:00 BRI

• So the problem I’ve been having with the GEM was that JsonNode.asText() returned an empty String. JsonNode.toSting() does what asText is supposed to do as I understand it… Anyway, I am now able to generate a json message that can be used to test the GeoMesaIngestService.
• Finished out the Web ingest test
• Started the service ingest test
• Need to enter stories and flesh out queries
• Need to push NLP, Geocoder and Ingest before heading out today
• Timesheet!

I seem to be making $800 mistakes these days. Got the flight information wrong on the flights to HCIC, and this past Saturday I crashed my bike on wet roads, which destroyed the camera I was carrying and my front brake. Here’s hoping that these things don’t happen in threes…

7:00 – 8:00 Research

• Codev2 by Lawrence Lessig:Lessig’s “Code and Other Laws of Cyberspace” was published in 1999. The book quickly began to define a certain vocabulary for thinking about the regulation of cyberspace. More than any other social space, cyberspace would be controlled or not depending upon the architecture, or “code,” of that space. And that meant regulators, and those seeking to protect cyberspace from at least some forms of regulation, needed to focus not just upon the work of legislators, but also the work of technologists.Code v2 updates the original work. It is not, as Lessig writes in the preface, a “new work.” Written in part collectively, through a Wiki hosted by JotSpot, the aim of the update was to recast the argument in the current context, and to clarify the argument where necessary.
• More C&C
• This is tantamount to stating that the social universe resembles the organic universe of Aristotle, in which there can be distinguished a centre and a periphery, a top and a bottom, and a high and a low, rather than the mechanical universe of Newton, homogeneous and lacking any one favoured direction. Without wasting words, we may state that in decision making there is no tabula rasa, any more than there are many decisions that are disputed once they have been taken. [p 113]
  • To Plato and Aristotle alike the universe is spatially finite; it is a sphere, of which the outer portion is divine in nature, the central human and imperfect, the former through a descending series giving the law to the latter.
  • Using the geocentric approach (Normocentric?  Valocentric?) to display/mapping could certainly be a mapping paradigm that could work. Looking at interactions as a sphere allows for multiple perspectives, multiple poles, and interesting projections down to 2D. And from a belief space perspective, people can take their rightful place near the center. And as well, someone who is near the periphery of belief can know that too.
• …consensus polarized in the direction of emerging norms [p 114]
  • Are norms poles that have a position and velocity in belief space? Or are they a manifestation of the group behavior, sort of the position of the average future center (position and variance?) of the group. In other words, do norms have an attraction or are they an implicitly agreed on, emergent, set of beliefs that exist at a certain time? And along these lines, are there patterns that persist over different time spans? I would bet that a circling flock has a centroid of (persistent beliefs), while a stampede doesn’t. This ties back to Arednt’s description of totalitarianism and terror where change is the only constant.
• On the whole here we are looking at, broadly sketched out, the picture of what must occur when a problem evokes a large-scale movement of opinion. People participate in the debate frequently and with intensity. The series of decisions leading to consensus polarizes towards the norm that is emerging and, by this very fact, emphasizes it. Thus these decisions cause the norm to crystallize and facilitate its being embraced fully by each individual, who feels himself a little its creator. Therefore no coercion or forced consensus should enter into it. [p 117]
  • Support for the idea that the norms are not a pole, but a projection of the future?
• Attitude polarization, familiarization and group process
• What happened where a feminist or anti-feminist confederate was present? Of course both were confronting an attitude that had crystallized, had become solidly fixed, and was almost a cultural cliche. The feminist confederate was no longer proposing anything new, but was defending what had become a norm and his or her influence was reinforcing conformity, whereas the anti-feminist confederate seemed to be utterly conservative, a reactionary deviant. Upon examining the data, it was found that their influence had little room in which to be exerted, and that it was weak. Thus the feminist confederates succeeded in polarizing somewhat during the discussion, but to a significant degree (F = 6.56;p < .01), the consensus. On the other hand, the confederate defending an anti-feminist position brought about no reaction, as if this was already ruled out (see Table 5.4). This is why one can no longer observe the former bi-polarization, when the discussions on the status of women set pro-feminists against their adversaries. At any rate the heated atmosphere had cooled down. The researchers noted that there were fewer arguments, and that the discussions were flat and unenthusiastic. An air of nostalgia hung over the groups, who seemed to be asking themselves, ‘But where are the debates of yesteryear?’, just as the poet Villon had once asked, ‘Where are the snows of yesteryear?’ They knew they were waging a fight that had been won by others quite a time ago. [p 120]
  • This would be the old study, where norms are emerging (beliefs in collision):  This would be stable, evolving norms:  And this would be something like a fixed ideology 
  • Here’s some charts of ideology in Congress. Picture links to article. The variance is interesting – it implies in the chart on the left that democrats are becoming less diverse, while republicans seem to be breaking into sub-populations(?). 

9:00 – 5:00 BRI

• Need to write up stories
• Need to discuss refactoring GeocoderService
• Stories
  • Mock tests for the interface 3 points (fix unit tests)
  • Modify GeoIngestService for document-centric storage of GEM Facts – 5 points
  • Refactor GeoMesaIngestService to GeoMesaDatabaseService – 5 points
  • Business logic for publicly accessible methods
    • Store (produce stunt GEM Json) – 5 points
    • Queries (8 points)
      • By geo coordinate
      • By document and geo coordinate
      • By location name and geo coordinate
    • Deploy – 3 points
• Wrote teh above up as an email and sent off to Matt
• Working on getting the json assembled but the serializer is choking on a null elevation. There is a way to get elevation data from Google, but it requires a separate call. Stuffing in zeros for the time being. Oops:

• private double latitude; private double longitude; private Double elevation;

• No, that’s a feature, but the serializer chokes. Not sure what to do and it looks like everyone has left?

• {  "name": "elevation",  "type": ["null", "double"], <- serializer does not like this  "default": null,  "doc": "The elevation in meters. May be negative, 0, positive, or null, if unknown."  }

7:00 – 8:00 Research

• Continuing C&C
• … to establish, on the one hand, the existence of a positive correlation between the average of individual positions and the shift towards the mean of positions around which a consensus is formed, and, on the other hand, the reduction in variance between the different positions after discussion in groups. [p 106]
  •  My model, All Exploit, stampede settings
• polarization depended not upon the inclination of individuals to take risks or to remain prudent, but indeed on the form in which dilemmas are couched and debated, which causes the balance to swing one way or another. [p 106]
  • This is an example of the presentation affecting the outcome. If this were a gatekeeper issue, the information presented would have been different
• Cvetkovitch and Baumgardner show that “group interaction will increase the participants’ involvement with the discussed topic and eventuate in individual opinion and group consensus that is more extreme than pre-discussion attitude. Additionally, the direction of polarization will be towards the naturally occurring majority opinion of the salient reference group and not toward the average attitude within the discussion groups. (1973: 161)” [p 107]
• Main and Walker (1973) noted that in this respect the decisions were more liberal when the judges decided as a body (55 per cent as against 45 per cent). Knowing that only a minority of decisions taken by a judge presiding alone were of a liberal kind, the authors hypothesized that judges nevertheless had a liberal code of values. When they decided alone, the pressure of anti-liberal public opinion led them to a compromise. On the other hand, when they judged collegially their personal values came out and became more radical during discussion with their colleagues, who were of the same persuasion. [p 111]
  • A thought about how interacting with different groups with different norms might help reduce polarization. The flip side of this is that social media allows us to continually be present in a group online. We don’t visit in the same way as going to a meeting, social hour, gathering of friends, etc.

9:00 – 5:00 BRI

• Aaron is still beating on the @AutoMiswired issue. It turns out that the GeoMesaIngest didn’t have the xxxService.java (interface) xxxServiceImpl.java (method), which was causing Spring to point at the wrong (main) instance. Fixed!
• Sprint review at 11:00. Looking for ways to present screenshots in rocket
• Wound up not presenting anything. It looks like you have to fill out an agenda on Confluence
• Sprint planning meeting. Whoops, didn’t make the tickets that describe the remaining work.
  • Create stories for deployment by DevOps (Heath) of GeoCoder and GeoMesaIngest
• Something is wrong with the clicking on the document line – I’m getting an ‘IndexOutOfBoundsException’…

Dentist!

7:00 – 8:00 Research

• Hybrid Forecasting Challenge from IAPA looks like something that the Research Browser might be good for. Lots of good proposer material here as well
• Back to C&C
• risky shift
• they demonstrated this in a very simple way. On the one hand, they devised a questionnaire with choice dilemmas analogous to that used by the American social psychologists. Four dilemmas brought into play risk values; four others called for prudence values. But they changed the method of answering by using – as we did – a graduated seven-point scale. This read as follows: 1: strongly recommend x, the risky solution; up to 7: strongly recommend the prudent solution; with 4 as the neutral point at which both solutions appear of equal value.
• The scale provides a measure of the tendency of the group, indicating unambiguously whether the average of the attitudes of its members at the beginning was located on the side of prudence or of risk. In order to verify the hypothesis, consensuses must be polarized in the direction of the initial values prevailing before the discussion, that is, they should not go beyond the neutral point, crossing the invisible Rubicon of risk towards prudence, or vice versa. The answer can therefore easily be given, since this neutral point expresses indecision or psychological indifference. It is clear that in most cases the groups have polarized, if one makes the comparison between the consensuses or final decisions and the initial decisions. Very rarely did they go beyond this neutral point in the opposite direction to that which their members had tended at the outset. If these members were bold, the shift took place in the direction of risk, and if they were prudent at the beginning, they became even more so. [p 101 -102]
• Gouge and Fraser (1972) instead of choice dilemmas, proposed to the groups that they debate a great variety of problems ranging from drugs to sexuality, and including racism, suicide, etc. With one single exception, consensus accentuated the initial tendency of attitudes and judgements. Those propositions about which participants were already in agreement separately secured even greater agreement after discussion. Those that met with moderate agreement from individuals separately produced a more extreme agreement when they assembled in a group. [p 102]
  • There is a selection of the dimension(s) that the discussion will take place over.
  • There has to be a level of agreement along those dimensions
  • Closely aligned dimensions are therefore more likely to be in agreement, while further ones are less likely, since the ‘motion’ has less (implicit?) correlation.
  • Collapsing dimensions makes for easier discussion. If everything could be collapsed to one dimension, then it’s trivial (Arednt suggests that this is what totalitarianism is)
  • In the absence of external influence, with a sufficiently small number of dimensions, direction stays fixed, though I’m not sure about velocity. But this is echo chamber/stampede space
  • Diversity can be the presence of different headings, opposing velocity or additional dimensions (enlarging the information horizon)
• depending on whether individuals are nearer or further away from the dominant pole of the scale, things proceed differently. When nearer to it, the extremists maintain their position, shifting less than do the moderates. This arises because the extremists can move only in a direction running counter to the norms, which is an eventuality ruled out, whereas the moderates can shift closer to this pole. As regards the other pole, it is the extremists who change more than do the moderates (p <-.001). They are comparatively more numerous than the latter (70 per cent as against 59 per cent) in linking up with the predominant norm in the population. Moreover, their greater distance from the norm has the result that after the discussion not only do they change in greater proportion, but also this change occurs to a significant degree. [p 104]
  • This implies that there is a physics model with respect to normative poles, though it may not fall off with distance.

10:30 – 5:00 BRI

• Flailing at reworking GeoMesaIngest. Starting at the web test that fires the GeoMesaController with  mocked GeoMesaIngest. Except that’s throwing errors. Aaron is looking into it, says it seems familiar. Essentially, Spring needs to scan the framework to find the annotations, but the the test code is not being scanned properly, so code from the main is being injected into the test, which is breaking things.
• Helped Aaron out a bit looking for projects that would serve as good frameworks to hang our research and demo work on

7:00 – 8:00 Research

• Never re-OCR a PDF text in Acrobat. It screws up all the highlighting, because Adobe.
• Based on what I’m reading in C&C, I think the next experiment might be to look at how to facilitate and map discussion groups. Roughly the same research browser concept – provide support for search in a variety of affordances, but also have a chat room where the discussions take place. Supporting points from user searches could be shared with other discussants. A copy of the annotated web page (highlights, etc) would appear as a new tab in the other browsers as a ‘shared from’. In this way, the important axis of the discussion could emerge and be factored into the map.
• Ok, back to the reading. These were an important couple of pages:
• It justifies our emphasizing how obsolete are the distinctions between factual and value judgments, the one exercising an informative influence, and the other a normative influence. Yet others continue to adhere to these distinctions. [p 97]
• Each time the assessment is arrived at by ordering the terms on a scale, one of whose poles has in our eyes greater importance than the other. It is in relation to this pole that we place persons, things or ideas. [p 98]
  • This is what I was discussing above. It should be possible to determine what these poles are, and create maps using them. These maps will have utility, because they reflect what is important in the discussion.
  • A second issue is to be able to show the pattern where what is important changes. A timeline of pole eigenvectors might be helpful.
• In a social environment, as soon as this hierarchy becomes explicit, individuals and groups clearly move towards the dominant pole. They seek to become more than they were, and this more so than others: more loyal, more courageous, more tolerant, more patriotic, more modern, and so on. This occurs particularly in novel circumstances, where experience does not relativize values or the image individuals wish to have of themselves. [p 98]
  • So, what does explicit mean here?
• This is most certainly very apparent nowadays, where the value assigned to novelty, to being avant-garde, to the fact of being different, is very high. [p 98]
  • I think this is the motion part that leads to flocking and other dynamic behavior (e.g. fashion trends). Novelty is always attractive, because as animals that had to hunt and gather, we understand at a genetic level that stasis means that we starve.
• the normative hypothesis of the theory: the tendency defined by the dominant values and attitudes is accentuated during the discussions, meetings, etc., and determines the directional shift of the decisions that lead to group consensus. [p 98]
• To sum up: it may seem strange that groups spontaneously swing away from the just mean and the conformity they should adhere to. But they do not do so regardless of the direction, which is towards the norm to which they all adhere. This is why one can scarcely ask whether a consensus is going in the right or the wrong direction, without asking on what basis it has been established and by whom. Stated in statistical terms, this hypothesis predicts that the mean of the choices on which the group members reach agreement is closer to the dominant pole of its scale of values than the mean of the initial choices made by each one of them separately. [p 99]
• In many respects this hypothesis is the most important one. It indicates how greatly the direction taken by collective opinions and judgements is predetermined, no matter what one does. Thus they are predetermined by the store of previous knowledge and values, and up to a certain point by the collective memory that all share before they meet, and which are all ingrained in them. [p 99]
  • This is the inertia of the group moving across belief (value????) space. And this is why the flocking algorithm, based only on heading and velocity is appropriate.

9:00 – BRI

• See if GeoMesa example works again
• Unable to run hdfs dfs -mkdir /hbase/lib because safe mode was on. Went on a disk-cleaning rampage, including removing libre office and got safe mode off
• And because I was on a roll, installed Bleachbit and gave it a shot
• Still fails the GeoMesaIngest tests due to a null pointer. Going back to the HBaseQuickStart program.
• My new IntelliJ needed to download many things
• Quickstart works! So we’re back to the starting gate. Now why is GeoMesaIngest blowing up?

  org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.NullPointerException
  
  	at com.vistronix.geomesaingest.web.GeoMesaIngestControllerTest.testIngestJson(GeoMesaIngestControllerTest.java:80)
  Caused by: java.lang.NullPointerException
  	at com.vistronix.geomesaingest.service.GeoMesaIngestor.ingestJsonData(GeoMesaIngestor.java:60)
  	at com.vistronix.geomesaingest.web.GeoMesaIngestController.ingestJson(GeoMesaIngestController.java:44)
  

• Ok, it turns out that the Mock tests are accessing the actual controller that is in turn accessing the db, which not right at all. Need to rewrite the current tests and add some more, as well as change the schema so that it works with documents and not patients
• Also need to be able to handle queries? A person would bridge several documents. How should that work? What does the test look like
• Realized that -1.0 was a bad flag for no geocoordinate. Changed it to 999 as a static final double

7:00 – 8:00 Research

• Feeling blue and it’s hard to focus.
• More C&C
  • We would have, on the one hand, a mimetic sociability, and, on the other, a cathartic sociability. But once more this is an antimony that depends upon circumstances and upon human nature. If we mention here both lines of thinking it is not in order to plead for one rather than the other, but to put both in perspective. [p 92]
  • Ratio decidendi [p 95]
  • At the same time, the more precise the information, the less the field is left free for differences and individual positions. Gradually these are supplanted by collective positions in the consciousness of group members. If the members continue for long enough, the consensus approximates to these values, just as a house under construction does to the architect’s plan. [p 97]
    • I think that I see this as a dimension reduction issue. Too many dimensions, and the distances are too great to come to a consensus, average or extreme. But some dimensions have very little variability, and can be discarded. Some dimensions can be placed outside of the scope of the discussion while others are included by mutual consent. The process of interactive dimension reduction continues until there is enough range of opinion across a sufficiently small set of dimensions that it is possible for a group opinion to emerge. The qualities would be different for different groups – humans, AI, or genes.
• Chat with Stacy on messenger this morning:
• Phil, this made me think of your comment about how robots may be changing humans for the worse:
  Opinion | The Trouble With Sex Robots
  Sexual consent is a serious enough problem without bots encouraging men to think it’s unnecessary.
  nytimes.com
  5:30AM
  Thanks – I downloaded and scanned the source paper. I think my basic concern is that ‘power corrupts’, and the moment our machines afford a power dynamic similar to a human one, there are many opportunities for things to go badly.
  7:40AMInteresting. Can a toaster afford a power dynamic (“Make my toast!”)? What about a life-size blow-up female doll? What about a male real doll?
  I think it’s in the eye of the beholder. I have ‘relationships’ with some of my bicycles that are strong enough that I can’t sell them. But an old phone, arguably something that I spend more time with, I can dispose of easily.
  With respect to the toaster, I think there is a continuum? Imagine at one end, a purely mechanical toaster. Push lever, pause, toast pops up.
  On the other end is a human butler, who you ask to make toast. Machines are now traversing that range. It wouldn’t be hard, for example to have Alexa interact with an IoT toaster to make toast on request.
  And Alexa is designed for you to be curt with it.
  I think I heard on NPR that there are researchers looking into how our interactions with voice-activated technology will impact our interactions with human beings, since the same affordance (speech) is being used on entities place in different power dynamics.
  Pulling this around to my research (Because why wouldn’t you?), these sorts of novel inputs may affect how we flock and move through belief space.

9:00 – 5:30 – BRI

• Generating the message from GeocoderService, but GeoIngestService fails.
• Going back to my geomesa-tutorials. That fails too: Exception in thread “main” java.io.IOException: java.util.concurrent.ExecutionException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/data/default/geomesa4/d7c3215c807b2ad9bb32555efbec133a/.regioninfo could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation.
  • It’s possible the node is full. Trying that first
  • Stopping everything and reformatted.
  • Can’t even copy in the GeoMesa jar:

    ~/hadoop-2.7.3$ bin/hadoop fs -put ${GEOMESA_HOME}dist/hbase/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar hdfs://localhost:9000/hbase/lib/
    Picked up JAVA_TOOL_OPTIONS: -Dgeomesa.hbase.coprocessor.path=hdfs://localhost:9000/hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar
    17/07/18 10:55:05 WARN hdfs.DFSClient: DataStreamer Exception
    org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
    	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1571)
    	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3107)
    	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3031)
    	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
    	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
    	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
    	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
    	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
    	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
    	at java.security.AccessController.doPrivileged(Native Method)
    	at javax.security.auth.Subject.doAs(Subject.java:422)
    	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
    
    	at org.apache.hadoop.ipc.Client.call(Client.java:1475)
    	at org.apache.hadoop.ipc.Client.call(Client.java:1412)
    	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
    	at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
    	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
    	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    	at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
    	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1455)
    	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1251)
    	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)
    put: File /hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.

  • Looks like the problem was that you should not format the dfs multiple times. Hadoop gets out of sync. Deleted the /tmp/hadoop-pfeldman directory and files, then reformatted and restarted. That appears to be working.

Big, tough ride on Saturday. Still tired!

7:00 – 8:00 Research

• Is this an AI stampede/echo chamber??
  • “There was no reward to sticking to English language,” says Dhruv Batra, visiting research scientist from Georgia Tech at Facebook AI Research (FAIR). As these two agents competed to get the best deal–a very effective bit of AI vs. AI dogfighting researchers have dubbed a “generative adversarial network”–neither was offered any sort of incentive for speaking as a normal person would. So they began to diverge, eventually rearranging legible words into seemingly nonsensical sentences. Supporting Facebook papers
• More C&C:
• In general, no one remains entirely passive when faced with what emanates from other people. They are approved of, or argued with in one of those interior dialogues, those silent conversations, in which the group with whom we are communicating is no longer outside us but within us – which is what ‘thinking’ means. Arguments are adopted because they are better formulated, or because we believe we have discovered them ourselves, although we are often repeating, without being aware of it, those we have heard at the time. And when we exclaim ‘I’ve always thought so, but I didn’t dare say it,’ or, ‘I’ve always said so, that’s plain,’ it matters little whether we are sincere or not. It is a cry for recognition by the group.
• Thus the consensus of the great majority of the groups undergoes a polarization effect. The effect is weak when communication is carried on passively and impersonally, but grows stronger as soon as communication becomes intense and touches people personally. This signifies that the convergence observable in a group depends more on the level of participation and on reciprocal action between its members than on their individual qualities.  [p 87]
  • This also explains why the explore <-> flocking <-> stampede spectrum can be modeled by so few variables (heading, speed, and influence radius), as processed by the agent. This is a personal process with global effects.
• Thurstone scale  A Thurstone scale has a number of “agree” or “disagree” statements. It is a unidimensional scale to measure attitudes towards people.
  • This also could be a way of determining dimensions that have large ranges as opposed to highly constrained ones
• It is revealing that a situation in which one has to choose in a personal fashion renders the judgments and choices more extreme, whereas a situation demanding an impersonal choice favors compromise, or almost does. [p 91]
• We should bear in mind that all this related to the portrait of no one young man in particular. What would happen now if the participants were presented with photographs of familiar people, socially typical, such as workers or intellectuals? Inasmuch as their characteristics stand out more and attitudes towards them are more marked, it might be expected that the results might be more extreme. This was indeed the case. After the group discussion. it turned out that judgments on the characteristics became more extreme, even on the less important ones. [p 91]
  • So does it follow that online forums have this effect? Dose this create new norms after some period of time? Here’s the list from Wikipedia of controversial Reddit communities.
    • Tools for accessing Reddit data: http://files.pushshift.io/reddit/,  https://www.reddit.com/dev/api/ (more on pushshift.io)

8:45 – 5:00 BRI

• Working out how to fit the location/document message into the GEM and GeoMesa
• Using a HashMap to dedupe locations from the document
• Think I got it built. My mistake was thinking that the GEM had an inheritance structure. Next thing is to send the message to GeoMesa
• Off into the rain!

7:00 – 8:00 Research

• Wrote up some notes from meeting with Aaron
• More C&C
  • The conflicts or differences between members of the group are normally resolved by convergence towards an extreme position. Yet, depending on whether the discussion is public or private, or the dialogue exterior or interior, the convergence will be more, or less, close to that position. In other words, discussion, in its current meaning, depending on whether the individuals involved are active or passive, determines the extent to which the decision will become polarized. [p 81]
    • The book doesn’t cover CMC discussion, but the following two papers appear to perform similar experiments to the Moscovici work
  • Group and computer-mediated discussion effects in risk decision making
    • Managers individually and in 3-person groups made multiattribute risk choices (two investment alternatives, each with multiple outcomes). Two group decisions were reached during face-to-face discussion, and two were reached during (real-time) computer-mediated discussion. In comparison with prediscussion individual preferences, groups’ multiattribute risk choices and attitudes after face-to-face discussion were risk averse for gains and risk seeking for losses, a tendency predicted by prospect theory and consistent with choice shift and other group extremitization research. By contrast, group decisions during computer-mediated discussion did not shift in the direction of prospect theory predictions. The results are consistent with persuasive-arguments theory, in that computer-mediated discussion contained less argumentation than face-to-face discussion. Social decision schemes were used to evaluate alternative assumptions about the group process. A “(prospect-theory) norm-wins” decision scheme described group choice well in the face-to-face discussion condition, but not in the computer-mediated discussion condition. Another decision scheme, first-advocate wins, which described choices well in both face-to-face and computer-mediated discussions, was explored in a discussion of the role of communication in group decision making.
  • Group Polarization and Computer-Mediated Communication
    • Group polarization is the tendency of people to become more extreme in their thinking following group discussion. It may be beneficial to some, but detrimental to other, organizational decisions. This study examines how computer-mediated communication (CMC) may be associated with group polarization. Two laboratory experiments were carried out. The first experiment, conducted in an identified setting, demonstrated that removal of verbal cues might not have reduced social presence sufficiently to impact group polarization, but removal of visual cues might have reduced social presence sufficiently to raise group polarization. Besides confirming the results of the first experiment, the second experiment showed that the provision of anonymity might also have reduced social presence sufficiently to raise group polarization. Analyses of process data from both experiments indicated that the reduction in social presence might have increased group polarization by causing people to generate more novel arguments and engage in more one-upmanship behavior. Collectively, process and outcome data from both experiments reveal how group polarization might be affected by level of social presence. Specifically, group discussion carried out in an unsupported setting or an identified face-to-face CMC setting tends to result in weaker group polarization. Conversely, group discussion conducted in an anonymous face-to-face CMC setting or a dispersed CMC setting (with or without anonymity) tends to lead to stronger group polarization. Implications of these results for further research and practice are provided

8:30 – BRI

• Added publishers and subscribers to NLP, Gecoder, and Crawl. This is how I think it should work:
• Publisher (NLP):

  <publishers>
      <publisher id="masterdata-nlp" name="masterdata-nlp" default="true">
          <exchange>eip_exchange</exchange>
          <routingKey>eip.masterdata.nlp</routingKey>
      </publisher>
  </publishers>

• Subscriber (Geocoder, single channel):

  <subscribers>
      <subscriber name="subscriber-masterdata-nlp">
          <exchange>eip_exchange</exchange>
          <routingKey>eip.masterdata.nlp.#</routingKey>
      </subscriber>
  </subscribers>

• Subscriber (MDS, two channels):

  <subscribers>
      <subscriber name="subscriber-masterdata">
          <exchange>eip_exchange</exchange>
          <routingKey>eip.masterdata.#</routingKey>
      </subscriber>
      <subscriber name="subscriber-masterdata-nlp">
          <exchange>eip_exchange</exchange>
          <routingKey>eip.masterdata.nlp.#</routingKey>
      </subscriber>
  </subscribers>

• That seems to be working fine. Now I need to parse out the LOCATION facts. Here’s the loop that gets the document ID and all the locations:

  String idString = event.getId();
  List<Fact> fList = event.getFacts();
  for(Fact f : fList){
      Result r = (Result)(f.getValue());
      Map<String, List<NamedEntity>> neMap = r.getNamedEntities();
      if(neMap.containsKey("LOCATION")) {
          List<NamedEntity> locList = neMap.get("LOCATION");
          System.out.println(locList.toString());
      }
  }

• Now I need to assemble a message with document ID and all the locations that have lat/longs. I think the way to do this is to build an event that contains a set of geo-coordinate facts.
  • From Matt:
    • you send an Event with the same metadata fields (domain, id, tag, etc…) as the incoming event. Your event has a facts field. In that field you’ll add the Location entity
    • yes. Fact.value is the object and Fact.index is its index
• Started to build the event in AMQPMessageListener.onMessage():

  Event gcDocEvent = new Event(event.getId(), event.getType(), event.getDomain(), event.getTag(),
          System.currentTimeMillis(), "MessagingService", "event containing geolocation facts",geoFactList.toString(),geoFactList,
          "1.2.0-SNAPSHOT");

7:00 – 8:00, 4:00 – 6:00 Research

• Nice (and tasty!) meeting with Wayne
  • Put together outlines of papers by September 16th(?)
    • Precision and recall considered harmful (started, targeted at CHIIR)
    • Gatekeepers and information diversity (IR and UI) (This could be the above paper, or it might be more generalized)
    • Lone wolves, flocks, and stampedes in information space: (human, genetic, and AI)
    • Maps as mediating objects between human and machine knowledge
• Meeting today with Aaron
  • Using the ProPublica API seems like a good choice for quality data
  • Much discussion on how projections should be chosen and how much bias is introduced, particularly WRT axis choice. My thoughts are that the axis should be the areas that have highest variance, but there may be better options…
• More C&C
  • Showing the relationship that exists between , on the one hand, the conflict of opinion and differences in information and, on the other , the eyeball confrontations that lead to consensus. [p 79]
    • There was a presentation at Collective Intelligence 2017 that talked about how the ordering of results would affect the ‘quality’ of downloaded  (vs ‘liked’) items. Random ordering (with no visible rating) of results with no rating provided the most consistent results. Ordering based on visible ratings led to first-mover cascades, regardless of ‘quality’. Ratings and order do seem to behave in some ways for proxies for ‘eyeball confrontations’?
  • The mere presence of other people already produces a movement in this direction. It is hardly surprising that, through looking at and listening to them, one becomes a participant in the dialogue, engaging within oneself in one of those imaginary conversations with which we are all familiar. This is sufficient to spark off a ‘fictitious polemic’ with our friends or superiors, in which we argue with them, and which on occasion leads us to modify our attitudes or choices. [p 80]
    • I think forum lurking is an example of this, though we may seek forums where our imaginary conversations are in line with the crowd.
  • Hannah Arendt’s take: The power of judgment rests on a potential agreement with others, and the thinking process which is active in judging something is not, like the thought process of our reasoning, a dialogue between me and myself, but finds itself, always and primarily, even if I am quite alone in making up my mind, in an anticipated communication with others with whom I know I must finally come to some agreement [ p 80]

8:30 – 3:30 BRI

• Got Hbase working, now working on NLP
• NLP is running fine, generting locations. Now I need to catch those in GeoCoder and see if I can get coordinates
  • The Geocoder test failed because “lon”: “-122.0850862” != “lon”: “-122.0850861”
• Adding in <publisher> role for NLP and reading in with MDS and GeoCoder

7:00 – 8:00 Research

• Continuing C&C
  • The authors of this study therefore varied the degree of cohesion in the groups formed in their laboratory, whose task was to reach an agreement on the risks to be recommended to the fictitious persons of the questionnaire with which we are now very familiar. The results obtained were in conformity with expectations. It turned out that groups having less cohesion recommended daring options, and groups having more cohesion prudent ones. According to the former groups, the fictitious persons would jump at the chance of changing their job and lifestyle; according to the latter groups, they would be content with their present lot. At the same time it was discovered that groups conscious of their cohesion were little subject to tensions and contradictions; this means that their members showed more esteem for their group and desired more strongly to be together than did members of groups possessing less cohesion. They also declared that agreement was reached with their fellows in greater personal intimacy and in a more favourable atmosphere. These are indications that they have done everything to maintain harmony and minimize the differences between them by avoiding factors leading to discord. In short, as the long-standing theory of Festinger (1950) had predicted, cohesion increases the pressure to conform and leads to the search for a compromise in the group. [p 73]
    • In reading this, I think that there may be a pattern where large, diverse groups split into progressively smaller, more cohesive groups, each on their own trajectory. An example of this could be the pattern of schism (and to a lesser degree union) in Christianity
  • Clearly, by favouring divergence, and then debate, through the heterogeneous nature of individuals, through their belonging to different professions, through the distance between individual positions, through a lesser cohesiveness in groups or increased trust among their members, consensus is polarized. Moreover, is it not characteristic of such a consensus for common choices not to be decided in advance by a majority rule or compromise, but discovered during adequate discussion? With this as a basis they are rooted in the collectivity as much as in individuals. This is why those who meet together have an interest in not resembling one another. And yet it is true that birds of a feather flock together. All our collective relationships hinge on this paradox. [p 76]
    • Is this a manifestation of explore/exploit? I think so.
  • Thus it is knowledge gleaned from several sources that fuels discussion among them. They are the cornerstones of a well-informed society, a collective organism that is endowed with the power of thought. But the organism shares out among individuals the task of selecting and exploiting the various kinds of knowledge, as well as the job of imparting meaning to words (Putnam, 1979). [p 76]
    • Looks like the authors may think this too
  • In half the groups all their members listened to the proofs in the same order; in the other half, each member listened to them set out in a special order that differed for each member. The first set of information was homogeneous, the second heterogeneous. Moreover, twelve juries listened to proofs that inculpated the accused, and the twelve others to proofs that exculpated him. According to the usual procedure, after listening in court to the facts presented, the jurors assessed separately the degree of guilt of the accused. Then, meeting together as a jury, they discussed the case before evaluating once more separately the degree of guilt. Here we are very close to a real life situation; hence the great significance of the findings.The following is what emerged: consultation together, yet again, led to more decisive verdicts. The difference was even more marked in the groups where each juror heard the proofs in a different order than in those which listened to them in the same order. In other words, when the task of cognition is divided up, the groups polarize more than when the task is uniform. One consequence among others is the following. It is often recommended that jurors should be selected from people whose social origins and intellectual training are as diverse as possible, that is, based on reality, in order to ensure fairer verdicts. The suspicion is that in this way they may be either more clement or more severe. In any case, the analysis of the discussions themselves showed that those jurors who had listened to the proofs presented in a different order mentioned a wider variety of facts than did the others, particularly towards the end of their discussion. [p 77]
    • This is near the core of the Precision and Recall Considered Harmful argument.

9:00 – 4:00 BRI

• Working on getting all of my pieces working.
• Got CrawlService running in IntelliJ!
• Payload is coming in just fine using:

  {
  "query": "Illinois&exactTerms=William Malik&orTerms=police arrest officer charge report",
  "requestId": "IntegrationTestWilliamMalik"
  }

• However, nothing is going to the MDS or NLP. It looks like no crawl is happening?
• Found out why. Here’s the exception:

  org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'crawler4jCrawlController' defined in class path resource [com/vistronix/crawlservice/config/CrawlerConfig.class]: 
  Bean instantiation via factory method failed; 
  nested exception is org.springframework.beans.BeanInstantiationException: 
  Failed to instantiate [edu.uci.ics.crawler4j.crawler.CrawlController]: 
  Factory method 'crawler4jCrawlController' threw exception; 
  nested exception is java.lang.Exception: couldn't create the storage folder: /data/crawl/1499883629556 does it already exist ?

• Created /data/crawl folder with rwx permissions
• Working on getting MDS working. Hbase is not the default, so the following args have to be added to the VM: -DdataStore.type=hbase -DdataStore.host=localhost -DdataStore.port=2181

7:00 – 8:00 Research

• Played around with the lit review section using LMN. Here’s a screenshot of the docs as of yesterday: 
• Continuing C&C
  • Thus, as a hypothesis concerning the polarization of groups, it may be concluded that the consensus reached will be the more extreme:
    (a) when individuals participate more directly in the discussions;
    (b) when the differences between them, their knowledge and their opinions are more marked;
    (c) when what is at stake in the discussions is perceived by them as valuable. [p 67]
    • What interests me here is (b). I think that there are several measures of difference that matter
      1. The information distance, as determined by amplitude and variance. There is a difference between agreement about two extreme positions that are broadly based and two narrow positions.
      2. The heading alignment. It is possible to arrive at a position from different directions. Is it easier if the headings are similar?
      3. Velocity. Is there a situation where one piece of information is held fixed and everything else is allowed to change? (e.g. The Leader is always right, though the position is in constant flux [Trump supporters know Trump lies. They just don’t care.])
      4. Exogenous visibility. What does the information horizon look like to the discussants? Do they feel as though they are relatively close or far apart? The VI/Emacs disagreements seem both vast and trivial, depending on framing, for example.
  • Plainly, there is no halo effect on questions that are not included in it (the discussion) [p 69]
    • So in emergent groups, what is the discussion? Or do we look for polarizing behavior and infer the point of discussion from that? I think that this implies axis on a dimension-reduced map that might make sense. 
  • we assumed that where discussion had created tension, shifts in the direction of a consensus should be more frequent. To verify this, the distance was measured between the two individuals whose opinions diverged most before the beginning of the discussion. A distance of 1 meant that these two opinions were separated by one point on a seven-point attitude scale. The opinions of the others, whether identical or not among themselves, were located between those of the two individuals who differed most. In the same way a distance of 6 meant that one of the individuals in the group was located at the favourable pole and another at the unfavourable pole. Thus they were opposites; the opinions of the rest were distributed between these poles. It is here, where conflict was greatest, that the maximum polarization should be recorded. In fact, the shifts towards an extreme consensus turned out to be more frequent in the groups when the gap was more than three points than when it was below that figure. [p 69]
  • The finding was simple: the common choices were much more extreme in the groups of five than in those of four, which themselves were more extreme than those in the groups of three. Moreover, they polarized more when they engaged in discussion among themselves than when they proceeded to a silent exchange of notes [p 70]

9:00 – 4:30 BRC

• Setting up test instance
  • CrawlService – Cloned
  • NLPService – Cloned
  • MasterDataService – Cloned
  • gtc-test-fixtures – Cloned
  • Added Chrome and Postman
  • Added the rmq dash: sudo rabbitmq-plugins enable rabbitmq_management, which runs at http://localhost:15672 with default login and password of guest
  • MasterDataService runs and is visible here: http://localhost:8890/masterdataservice/events
  • To make things easier to run, added a slew of aliases:

    alias mrmq='sudo rabbitmq-plugins enable rabbitmq_management'
    alias ovpn='cd /home/pfeldman/openVPNconfig && sudo openvpn --config bellrock.ovpn && cd ..'
    alias run_crawl='java -jar /home/pfeldman/IdeaProjects/CrawlService/build/libs/crawlservice.war &'
    alias run_geocoder='java -jar /home/pfeldman/IdeaProjects/GeoCoderService/build/libs/geocoderservice.war &'
    alias run_mds='java -jar /home/pfeldman/IdeaProjects/MasterDataService/build/libs/masterdataservice.war &'
    alias run_nlp='java -jar /home/pfeldman/IdeaProjects/NLPService/build/libs/nlpservice.war &'
    alias startdfs='/home/pfeldman/hadoop-2.7.3/sbin/start-dfs.sh'
    alias starthbase='/home/pfeldman/hbase-1.3.1/bin/start-hbase.sh'
    alias startrmq='service rabbitmq-server start'
    alias stopdfs='/home/pfeldman/hadoop-2.7.3/sbin/stop-dfs.sh'
    alias stophbase='/home/pfeldman/hbase-1.3.1/bin/stop-hbase.sh'
    alias stoprmq='service rabbitmq-server start'
    

  • Have the full stack running, but was unable to get crawlservice running with this payload from scheduling service:

7:00 – 8:00

• Social Media and News Sources during the 2017 UK General Election
  • Platforms like Twitter and sources like Wikipedia are important parts of the information diet for many citizens. In this data memo, we analyse Twitter data on bot activity and junk news for a week in the final stages of campaigning of the 2017 UK General Election and also present data on Wikipedia page consultations about those parties and leaders. (1) Content about the Labour Party strongly dominated Twitter traffic in this period. (2) Social media users in the UK shared five links to professional news and information for every one link to junk news. (3) Wikipedia queries have gone from being mostly about the Conservative Party and Prime Minister Theresa May to being mostly about the Labour Party and the Labour leader Jeremy Corbyn. (4) In comparison to the first week of the campaign period, we find that users are sharing slightly better quality news content, that automated accounts are generating more traffic about the election, and that more of the automation uses Labour-related hashtags (though may not be from the Labour Party itself). (5) In comparison to trends in other countries, we find that UK users shared better quality information than that which many US users shared during the 2016 US election, but worse quality news and information than was shared during the French 2017 election.
• Recognizing safety and liveness
  • Informally, a safety property stipulates that “bad things” do not happen during execution of a program and a liveness property stipulates that “good things” do happen (eventually) (Lamport 1977). Distinguishing between safety and liveness properties is useful because proving that a program satisfies a safety property involves an invariance argument while proving that a program satisfies a liveness property involves a well-foundedness argument. Thus, knowing whether a property is safety or liveness helps when deciding how to prove that the property holds.
    • Read about this in this month’s Communications of the ACM. I wonder if it could be applied to the types of social models I’m building and trying to trace in user data.
• Continuing C&C
  • In contrast to the consensual form, we can understand that the normalized form, which gives only a subordinate role to some members of the group, creates a certain distance, causing the group not to loom so large in the life and consciousness of individuals, so that in the end it appears strange and abstract. Immediately the participants become detached from one another, and instead of being actors become mere spectators in the discussions. [p 62]
  • Since controversy is in proportion to the participation of members, few conflicts are observed, unless it be in the ranks of the leaders. It is as if individuals tended to minimize their ‘investment’ and their attachment to the collectivity, remaining aloof from intrigues, and, so far as possible, conforming to the opinions and actions that were suggested to them. [p 62]
    • Is this what happens on forums and low-participation social systems like comments?
  • Although the one satisfies the need to participate in a more intense way, and one of which people cannot be deprived for long, the other at least provides a substitute for it. [p 62]
    • This could be another affordance of the system. Some way to grade participation and discussion as a threshold of entering?
  • There can scarcely be any doubt that, by meeting and talking together, a group’s members bring out the values predominant among them, ones to which they are attached. In some way their substance is given shape, so that what we hold in common, but is concealed, becomes manifest. [p 65]
  • …consensual participation probably has the effect of raising the level of collective involvement, whereas normative participation lowers it. One may conclude that the former polarizes the decisions leading to consensus, whereas the latter modifies them. The former causes the members of the group to converge on the pole of values already shared by them before they took part in the decision, and the latter towards the just mean. [p 65]

8:30 – 4:30 BRC

• Taking care of travel paperwork. EZpass tolls not available yet
• Got GeoMesa running after a few hiccups. Full startup process is here
• Looked at the GM integration plan
• This could be appropriate: Event Analytics for Innovation Trajectories: Understanding Inputs and Outcomes for Entrepreneurial Success
• Installed GeoMesaIngestService and GeocoderService
• Need to install RabbitMQ Server v 3.5.3 for Ubuntu as per this page
  • Done with apt-get!

6:00 – ?? Research

• Continuing C&C
  • This tension arises less from the content of the argument or the difference that exists between them than because the disagreement manifests itself through someone else who has to be faced up to [p 55]
    • So what are the implications of CMC, where the ‘distance’ can be moderated? The spectrum can run from video chat to text chat to forum, to search results. What’s the sweet, frictionless spot that creates stapedes?
  • Thus, throughout controversies and counter-arguments, which resemble body-blows, the members of the group covertly exert upon one another an influence that emphasizes what can draw them closer. Between them can be observed a synchronized, imitative process which transforms every word into a signal, every gesture into a model, and every piece of information into an argument. All the forms of the rhetoric of mind and body become maneuvers through which the distances between the participants grow smaller and the frictions between them are deadened. [p 55]
  • But most frequently, by the very fact of being called upon to discuss, each individual feels himself to be an actor in the ritual and a member of the group instituting it. In this way, group cohesion is reinforced at regular intervals. [p57]
  • Numerous studies justify the assertion that people are more disposed to start out on that painful intellectual and affective path when they have to deal with opposing arguments coming from several sources rather than from one source alone. It is as if a group speaking with several voices were more conspicuous and offered greater room for maneuver than a group with only one voice.
    • Higher dimensions == less constraint?

8:30 – 4:30 BRI

• Continuing GeoMesa with Hbase. Notes are here