Phil 10.4.2022

Collective intelligence for deep learning: A survey of recent developments

  • In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled practitioners to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Collective behavior, commonly observed in nature, tends to produce systems that are robust, adaptable, and have less rigid assumptions about the environment configuration. Collective intelligence, as a field, studies the group intelligence that emerges from the interactions of many individuals. Within this field, ideas such as self-organization, emergent behavior, swarm optimization, and cellular automata were developed to model and explain complex systems. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research’s involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its capabilities. We hope this review can serve as a bridge between the complex systems and deep learning communities.


  • Chat with Dave tonight? – Need to send links to papers, OpenAI, Stable Diffusion thread, Paul Sharre’s books, etc


  • Fill out reimbursement forms – done
  • Travel to Chirp – tried in Concur. Hopeless mess
  • More RCSNN2
  • 9:15 standup
  • Experiment logger meeting – done
  • Reached out to Dr. Giddings on format of white paper

GPT Agents

  • Change table_user to have user ID as unique, primary key and see if update into works right – done
  • Add checkboxes for optional user attributes (requires downloading users for tweets)
  • No meeting is scheduled, so write up a status report