IUI 2024 Notes

Information visualization and Visual Analytics

• Assessing User Trust in Active Learning Systems: Insights from Query Policy and Uncertainty Visualization – Active learning, where the machine asks for help if uncertain. This is human ESDT. Numeric labels seem to help the most over other graphical visualizations. Like a picture with a xx% confidence seems best.

25% confident

• iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries – rates model rating of essays. Word replacement with synonyms allows the model to detect the salience of particular words in the grades. It would be interesting to try this with tests like GPTZero to see what they are looking for. Yes, they do have an API, so this would be straightforward?
• SlopeSeeker: A Search Tool for Exploring a Dataset of Quantifiable Trends. A good point here for our conversational interfaces. We may need to train (or prompt-tune?) models to be able to discriminate behaviors from the simulation in ways that can be articulated by the LLM
• Visual Analytics of Co-Occurrences to Discover Subspaces in Structured Data – a tool for exploring really large datasets. Like the Blackrock problem? It sort of resembles the way that I set up spreadsheets some times? And in fact, I think you could process the data and make it spreadsheet compatible.
• AutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning. Looks like a useful tool for building and evaluating models. I’d be curious how it would work with the JSC data. Worth watching the video, if its available. Shown using AutoSKLearn. Not sure about parallelization like raytune. Code is online.

Applications of Language Models

• Empirical Evidence on Conversational Control of GUI in Semantic Automation – conversational interface to access db. JSON-2-JSON transformer.
• FigurA11y: AI Assistance for Writing Scientific Alt Text – this looks like a really nice tool to generate alt-text and the code exists on GitHub. Not sure how easy to build, but worth trying. It would be interesting to see how the text could be incorporated in the body of the text as well, particularly in cases where the image needs to be pulled for length.

Generative AI: Theory and Applications