7:00 – 5:00 ASRC Research
- Graph laplacian dissertation
- The spectrum of the normalized graph Laplacian can reveal structural properties of a network and can be an important tool to help solve the structural identification problem. From the spectrum, we attempt to develop a tool that helps us to understand the network structure on a deep level and to identify the source of the network to a greater extent. The information about different topological properties of a graph carried by the complete spectrum of the normalized graph Laplacian is explored. We investigate how and why structural properties are reflected by the spectrum and how the spectrum changes when compairing different networks from different sources.
- Universality classes in nonequilibrium lattice systems
- This article reviews our present knowledge of universality classes in nonequilibrium systems defined on regular lattices. The first section presents the most important critical exponents and relations, as well as the field-theoretical formalism used in the text. The second section briefly addresses the question of scaling behavior at first-order phase transitions. In Sec. III the author looks at dynamical extensions of basic static classes, showing the effects of mixing dynamics and of percolation. The main body of the review begins in Sec. IV, where genuine, dynamical universality classes specific to nonequilibrium systems are introduced. Section V considers such nonequilibrium classes in coupled, multicomponent systems. Most of the known nonequilibrium transition classes are explored in low dimensions between active and absorbing states of reaction-diffusion-type systems. However, by mapping they can be related to the universal behavior of interface growth models, which are treated in Sec. VI. The review ends with a summary of the classes of absorbing-state and mean-field systems and discusses some possible directions for future research.
- “The Government Spies Using Our Webcams:” The Language of Conspiracy Theories in Online Discussions
- Conspiracy theories are omnipresent in online discussions—whether to explain a late-breaking event that still lacks official report or to give voice to political dissent. Conspiracy theories evolve, multiply, and interconnect, further complicating efforts to limit their propagation. It is therefore crucial to develop scalable methods to examine the nature of conspiratorial discussions in online communities. What do users talk about when they discuss conspiracy theories online? What are the recurring elements in their discussions? What do these elements tell us about the way users think? This work answers these questions by analyzing over ten years of discussions in r/conspiracy—an online community on Reddit dedicated to conspiratorial discussions. We focus on the key elements of a conspiracy theory: the conspiratorial agents, the actions they perform, and their targets. By computationally detecting agent–action–target triplets in conspiratorial statements, and grouping them into semantically coherent clusters, we develop a notion of narrative-motif to detect recurring patterns of triplets. For example, a narrative-motif such as “governmental agency–controls–communications” appears in diverse conspiratorial statements alleging that governmental agencies control information to nefarious ends. Thus, narrative-motifs expose commonalities between multiple conspiracy theories even when they refer to different events or circumstances. In the process, these representations help us understand how users talk about conspiracy theories and offer us a means to interpret what they talk about. Our approach enables a population-scale study of conspiracy theories in alternative news and social media with implications for understanding their adoption and combating their spread
- Need to upload to ArXiv (try multiple tex files) – done!
- If I’m charging my 400 hours today, then start putting together text prediction. I’d like to try the Google prediction series to see what happens. Otherwise, there are two things I’d like to try with LSTMs, since they take 2 coordinates as inputs
- Use a 2D embedding space
- Use NLP to get a parts-of-speech (PoS) analysis of the text so that there can be a (PoS, Word) coordinate.
- Evaluate the 2 approaches on their ability to converge?
- Coordinating with Antonio about workshops. It’s the 2019 version of this: International Workshop on Massively Multi-Agent Systems (MMAS2018) in conjunction with IJCAI/ECAI/AAMAS/ICML 2018
Listening to We Can’t Talk Anymore? Understanding the Structural Roots of Partisan Polarization and the Decline of Democratic Discourse in 21st Century America. Very Tajfel
- David Peritz
- Political polarization, accompanied by negative partisanship, are striking features of the current political landscape. Perhaps these trends were originally confined to politicians and the media, but we recently reached the point where the majority of Americans report they would consider it more objectionable if their children married across party lines than if they married someone of another faith. Where did this polarization come from? And what it is doing to American democracy, which is housed in institutions that were framed to encourage open deliberation, compromise and consensus formation? In this talk, Professor David Peritz will examine some of the deeper forces in the American economy, the public sphere and media, political institutions, and even moral psychology that best seem to account for the recent rise in popular polarization.
Sent out a Doodle to nail down the time for the PhD review
Went looking for something that talks about the cognitive load for TIT-FOR-TAT in the Iterated Prisoner’s Dilemma and can’t find anything. Did find this though, that is kind of interesting: New tack wins prisoner’s dilemma. It’s a collective intelligence approach:
- Teams could submit multiple strategies, or players, and the Southampton team submitted 60 programs. These, Jennings explained, were all slight variations on a theme and were designed to execute a known series of five to 10 moves by which they could recognize each other. Once two Southampton players recognized each other, they were designed to immediately assume “master and slave” roles – one would sacrifice itself so the other could win repeatedly.
- Nick Jennings
- Professor Jennings is an internationally-recognized authority in the areas of artificial intelligence, autonomous systems, cybersecurity and agent-based computing. His research covers both the science and the engineering of intelligent systems. He has undertaken fundamental research on automated bargaining, mechanism design, trust and reputation, coalition formation, human-agent collectives and crowd sourcing. He has also pioneered the application of multi-agent technology; developing real-world systems in domains such as business process management, smart energy systems, sensor networks, disaster response, telecommunications, citizen science and defence.
- Sarvapali D. (Gopal) Ramchurn
- I am a Professor of Artificial Intelligence in the Agents, Interaction, and Complexity Group (AIC), in the department of Electronics and Computer Science, at the University of Southampton and Chief Scientist for North Star, an AI startup. I am also the director of the newly created Centre for Machine Intelligence. I am interested in the development of autonomous agents and multi-agent systems and their application to Cyber Physical Systems (CPS) such as smart energy systems, the Internet of Things (IoT), and disaster response. My research combines a number of techniques from Machine learning, AI, Game theory, and HCI.
7:00 – 4:30 ASRC MKT
- SASO Travel request
- SASO Hotel – done! Aaaaand I booked for August rather than September. Sent a note to try and fix using their form. If nothing by COB try email.
- Potential DME repair?
- Starting Deep Learning with Keras. Done with chapter one
- Two seedbank lstm text examples:
- Generate Shakespeare using tf.keras
- This notebook demonstrates how to generate text using an RNN with tf.keras and eager execution.This notebook is an end-to-end example. When you run it, it will download a dataset of Shakespeare’s writing. The notebook will then train a model, and use it to generate sample output.
- This notebook will let you input a file containing the text you want your generator to mimic, train your model, see the results, and save it for future use all in one page.
TF Dev Sumit
Highlights blog post from the TF product manager
- Connecterra tracking cows
- Google is an AI – first company. All products are being influenced. TF is the dogfood that everyone is eating at google.
- Last year has been focussed on making TF easy to use
- 11 million downloads
- tf.keras – full implementation.
- Premade estimators
- three line training from reading to model? What data formats?
- Swift and tensorflow.js
- Real-world data and time-to-accuracy
- Fast version is the pretty version
- TensorflowLite is 300% speedup in inference? Just on mobile(?)
- Training speedup is about 300% – 400% anually
- Cloud TPUs are available in V2. 180 TF computation
- ResNet-50 on Cloud TPU in < 15
- Grand Engineering challenges as a list of ML goals
- Engineer the tools for scientific discovery
- AutoML – Hyperparameter tuning
- Less expertise (What about data cleaning?)
- Neural architecture search
- Cloud Automl for computer vision (for now – more later)
- Retinal data is being improved as the data labeling improves. The trained human trains the system proportionally
- Completely new, novel scientific discoveries – machine scan explore horizons in different ways from humans
- Single shot detector
Derrek Murray @mrry (tf.data)
- Core TF team
- tf.data –
- Fast, Flexible, and Easy to use
- ETL for TF
- Dataset tf.SparseTensor
- Dataset.from_generator – generates graphs from numpy arrays
- for batch in dataset: train_model(batch)
- 1.8 will read in CSV
- Figures out types from column names
Alexandre Passos (Eager Execution)
- Eager Execution
- Automatic differentiation
- Differentiation of graphs and code <- what does this mean?
- Quick iterations without building graphs
- Deep inspection of running models
- Dynamic models with complex control flows
- immediately run the tf code that can then be conditional
- w = tfe.variables([[1.0]])
- tape to record actions, so it’s possible to evaluate a variety of approaches as functions
- eager supports debugging!!!
- And profilable…
- Google collaboratory for Jupyter
- Customizing gradient, clipping to keep from exploding, etc
- tf variables are just python objects.
- Object oriented savings of TF models Kind of like pickle, in that associated variables are saved as well
- Supports component reuse?
- Single GPU is competitive in speed
- Interacting with graphs: Call into graphs Also call into eager from a graph
- Use tf.keras.layers, tf.keras.Model, tf.contribs.summary, tfe.metrics, and object-based saving
- Recursive RNNs work well in this
- Live demo goo.gl/eRpP8j
- getting started guide tensorflow.org/programmers_guide/eager
- example models goo.gl/RTHJa5
Daniel Smilkov (@dsmilkov) Nikhl Thorat (@nsthorat)
- In-Browser ML (No drivers, no installs)
- Browsers have access to sensors
- Data stays on the client (preprocessing stage)
- Allows inference and training entirely in the browser
- Author models directly in the browser
- import pre-trained models for inference
- re-train imported models (with private data)
- Layers API, (Eager) Ops API
- Can port keras or TF morel
- Can continue to train a model that is downloaded from the website
- This is really nice for accessibility
- Mailing list: goo.gl/drqpT5
- Performance optimization
- Need to be able to increase performance exponentially to be able to train better
- tf.data is the way to load data
- Tensorboard profiling tools
- Trace viewer within Tensorboard
- Map functions seem to take a long time?
- dataset.map(Parser_fn, num_parallel_calls = 64)) <- multithreading
- Software pipelining
- Distributed datasets are becoming critical. They will not fit on a single instance
- Accelerators work in a variety of ways, so optimizing is hardware dependent For example, lower precision can be much faster
- bfloat16 brain floating point format. Better for vanishing and exploding gradients
- Systolic processors load the hardware matrix while it’s multiplying, since you start at the upper left corner…
- Hardware is becoming harder and harder to do apples-to apples. You need to measure end-to-end on your own workloads. As a proxy, Stanford’s DAWNBench
- Two frameworks XLA nd Graph
Mustafa Ispir (tf.estimator, high level modules for experiments and scaling)
- estimators fill in the model, based on Google experiences
- define as an ml problem
- pre made estimators
- reasonable defaults
- feature columns – bucketing, embedding, etc
- estimator = model_to_estimator
- image = hum.image_embedding_column(…)
- supports scaling
- export to production
- Feature columns (from csv, etc) intro, goo.gl/nMEPBy
- Estimators documentation, custom estimators
- Wide-n-deep (goo.gl/l1cL3N from 2017)
- Estimators and Keras (goo.gl/ito9LE Effective TensorFlow for Non-Experts)
- distributed tensorflow
- estimator is TFs highest level of abstraction in the API google recommends using the highest level of abstraction you can be effective in
- Justine debugging with Tensorflow Debugger
- plugins are how you add features
- embedding projector with interactive label editing
Sarah Sirajuddin, Andrew Selle (TensorFlow Lite) On-device ML
- TF Lite interpreter is only 75 kilobytes!
- Would be useful as a biometric anonymizer for trustworthy anonymous citizen journalism. Maybe even adversarial recognition
- Introduction to TensorFlow Lite → https://goo.gl/8GsJVL
- Take a look at this article “Using TensorFlow Lite on Android” → https://goo.gl/J1ZDqm
Vijay Vasudevan AutoML @spezzer
- Theory lags practice in valuable discipline
- Iteration using human input
- Design your code to be tunable at all levels
- Submit your idea to an idea bank
- Nuclear Fusion
- TF for math, not ML
- Would this be useful for genetic algorithms as well?
- Open source TF community
- Developers mailing list firstname.lastname@example.org
- SIGs SIGBuild, other coming up
- SIG Tensorboard <- this
- Improved usability of TF
- 2 approaches, Graph and Eager
- Compiler analysis?
- Swift language support as a better option than Python?
- Richard Wei
- Did not actually see the compilation process with error messages?
TensorFlow Hub Andrew Gasparovic and Jeremiah Harmsen
- Version control for ML
- Reusable module within the hub. Less than a model, but shareable
- Retrainable and backpropagateable
- Re-use the architecture and trained weights (And save, many, many, many hours in training)
- module = hub.Module(…., trainable = true)
- Pretrained and ready to use for classification
- Packages the graph and the data
- Universal Sentence Encodings semantic similarity, etc. Very little training data
- Lower the learning rate so that you don’t ruin the existing rates
- modules are immutable
- Colab notebooks
- use #tfhub when modules are completed
- Try out the end-to-end example on GitHub → https://goo.gl/4DBvX7
TF Extensions Clemens Mewald and Raz Mathias
- TFX is developed to support lifecycle from data gathering to production
- Transform: Develop training model and serving model during development
- Model takes a raw data model as the request. The transform is being done in the graph
- RESTful API
- Model Analysis:
- ml-fairness.com – ROC curve for every group of users
Project Magenta (Sherol Chen)
- Suharsh Sivakumar – Google
- Billy Lamberta (documentation?) Google
- Ashay Agrawal Google
- Rajesh Anantharaman Cray
- Amanda Casari Concur Labs
- Gary Engler Elemental Path
- Keith J Bennett (email@example.com – ask about rover decision transcripts)
- Sandeep N. Gupta (firstname.lastname@example.org – ask about integration of latent variables into TF usage as a way of understanding the space better)
- Charlie Costello (email@example.com – human robot interaction communities)
- Kevin A. Shaw (firstname.lastname@example.org data from elderly to infer condition)