5:30 – 6:30 BRI
- Catching up on email, etc.
Research
- Implementing LETHAL and RESPAWN options
- Poster presentation! See if I can get a space near a table/outlet
- Talks
- Ece Kamar Humans to the rescue
- Troubleshooting of ML systems
- What happens when systems are functioning in the wild
- Biases in ML – minority representations of well, minorities (blind spots – unknown unknown)
- Beat the machine Attenberg 2011
- Multi-armed bandit – exploration – but how? What is the source to explore
- Hard to debug. What about AI making models that are understandable and effective. E.g. build low-variable systems using GP
- Nick Ouellette – Validating Models of Collective Behavior
- Flocks and swarms
- Movement + interaction rules+interaction range -> group structure function
- Vivsek et. al. PRL (1995) alignment only
- How to benchmark. What is a good/bad model
- ‘web.stanford.edu/~nto
- Creating Collective Intelligence – Dan Weld (Relevant to curating and BRI)
- Provisioning mixed computer – human teams
- Objective -> initial workflow -> improve, repeat (Self-optimizing workflow)
- Sliders as a way of understanding the sers, then a place to clarify. IMPORTANT. Then cluster on the confusions, and update the query/ Sample from the confusion results to test improvement.
- Gold questions (flags) that are known. What percentage get inserted for optimal user response
- Partially Observable Makkov decision process (Belief state as input)/ Build a policy that provides the optimal result
- Explore/Exploit strategy since partial information
- MicroTalk Test Task <- serendipity injection
- Select for discerning worker Flesch-Kincaid model?. Need to look into that.
- Conclusions
- Tools for creating collective intelligence
- Self-Optimizing Workflows
- Getting workers to argue
- Chris Welty – Google – moderator for session 4
- Andres Abeliuk – Controlling Collective Behavior through Position Bias
- display policies affect what customers choose
- Youtube, spotify, etc use ratings/reviews, PageRank, etc form feedback loops
- How do we mitigate these biases
- Salgannik MusicLab Science (2006?) runaway feedback loops to first user advantage
- This study ranks by listens/downloads. Does this affect the result. Four ranking policies, and yesy/no social signals
- Random ordering with no social signal seems to provide most consistent ranking with least inequality
- Brent Hecht: The Role of Human Geography in Collective Intelligence
- Physical Geography
- Spatial Computing
- Human Geography
- Spatial homophily – we live and spend time near people like us. Impossible to overstate
- Worker location
- Location of the work
- Distance between these two
- If a bank account is required, for example, you will have spatial bias. Pokemon Go was influenced by a biased crowdsourced dataset
- Interaction decreases with distance
- Structured variation population density
- Highest in urban cores lower in suburbs, lowest in rural.
- Johnson Et al SIGCHI 2016
- Mental maps, region theory
- Hila Lifshitz-Assaf Delineating Role Behaviors in Wikipedia
- Emergent roles determined by clusters of activity
- Temporal distance is as influential as geographic distance
- Watching what people do over time – how much chaos, how much order
- Need to dig up these papers
- Why did you choose the motivation axis?
- Mehdi Moussaid – The Propagation of accurate judgements in Experimental Transmission Chains
- Judgement propagation – well known
- Behavioral processes affect the level of influence – poorly known
- Experiment is sequential, but each person is exposed to the judgement of the prior results.
- Visual perception task, where the user determines the overall direction of a cluster of dots
- First person has an easy task – clear movement, but the others have a noisier version of the task, but the same answer
- Propagation slows down with social distance and falls to zero around 3
- Yue Han – Collective exploration: Remixing with human-based search Algorithms
- This is mapping. Not sure how the semantic spoace is mapped WRT what. Amd what i you run it in reverse?
- Kennith Huang – Real Time On-Demand Crowd powered entity extraction
- Output agreement mechanism – agreement gets reward.
- John Harlow – Proactively Identifying and Correcting for Social Biases in Datasets Proliferating into Civic Technology
- Identify biases of the past algorithmically and initiate corrections
- Being digitally invisible
- How people in a place understand the world around them?
- Yun Huang – BandCaption: Crowdsourcing Video Caption Corrections
- John Prpic – Unpacking Blockchain
- The poster went well!
- Kate Starbird Online rumouring
- Overlapping narratives and websites supporting alternative narratives
- Shooting related search terms for 9 months of twitter
- Gun takeaway agenda.
- False Triangulation – same information distributed across different sources
- Is there some sort of evolution of low dimension attractors?
- Gamification became co opted by efficiency and lost its game-like option Efficiency destroys diversity.
The Russian “Firehose of Falsehood” Propaganda Model Why It Might Work and Options to Counter It
- Is there a difference between ‘diversity’, ‘noise’, and disinformation/alt narratives. I think there is some kind of monitoring the ‘native population’ and then figuring ways to amplify? What is the adversary signaling
- Can you build a high-engagement game that will attract those in the alt-narratives an expands their perspective? Deliberation that is not tied directly to the outcome
- The alt-narrative ecosystem is not about signalling capability. It’s about signalling reach. And through the reach to powerful players, instigating uncertainty and disruption..
viztales 
You must be logged in to post a comment.