惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 三生石上(FineUI控件)
月光博客
月光博客
S
SegmentFault 最新的问题
有赞技术团队
有赞技术团队
Stack Overflow Blog
Stack Overflow Blog
Engineering at Meta
Engineering at Meta
T
The Blog of Author Tim Ferriss
The GitHub Blog
The GitHub Blog
小众软件
小众软件
Hugging Face - Blog
Hugging Face - Blog
IT之家
IT之家
宝玉的分享
宝玉的分享
A
About on SuperTechFans
Vercel News
Vercel News
P
Proofpoint News Feed
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
博客园 - 【当耐特】
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
V
Visual Studio Blog
Jina AI
Jina AI
Y
Y Combinator Blog
T
Tailwind CSS Blog
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
Last Week in AI
Last Week in AI

cs.HC updates on arXiv.org

Quantitative Movement Testing: Measuring Patient Movements from a Single Smartphone Video Vision-Language Models Suppress Female Representations Under Ambiguous Input The New Social Image: How AI Competency and AI Proactivity Influence Self- and Peer-Perceptions in the Workplace TUX: Measuring Human--AI Tacit Understanding LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents Label Over Logic? How Source Cues Bias Human Fallacy Judgments More Than LLMs Inform, Coach, Relate, Listen: Auditing LLM Caregiving Support Roles How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions MetaRanker: Human-in-the-loop Active Ranking for Metalens Image Quality Analyzing Persona Effects in Generated Explanations from Multimodal LLM Agents in Urban Perception First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering The Timing Dependencies of Trust: Speed, Accuracy, and cBCI Neuro-Decoupling in Human-AI Teams Bayesian Distributional Models of Executive Functioning Visual Matters: Connecting Aesthetic Appeal and Production Quality of Photos, Infographics and Data Visualizations to Credibility of Social Media Posts Data-driven Head Motion Generation through Natural Gaze-Head Coordination Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why Perceptually Lossless Tactile Texture Synthesis with Compact Spectral Envelope Models MambaGaze: Bidirectional Mamba with Explicit Missing Data Modeling for Cognitive Load Assessment from Eye-Gaze Tracking Data CogAdapt: Transferring Clinical ECG Foundation Models to Wearable Cognitive Load Assessment via Lead Adaptation Augmented Analytics and Decision Quality: The Role of Trust among Non-Technical BI Users Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build PaintCopilot: Modeling Painting as Autonomous Artistic Continuation Personality Engineering with AI Agents: A New Methodology for Negotiation Research PULSE: Agentic Investigation with Passive Sensing for Proactive Intervention in Cancer Survivorship Access Timing as Scaffolding: A Reinforcement Learning Approach to GenAI in Education Conversations in Space: Structuring Non-Linear LLM Interactions on a Canvas MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysis nASR: An End-to-End Trainable Neural Layer for Channel-Level EEG Artifact Subspace Reconstruction in Real-Time BCI
MusicJam: Visualizing Music Insights via Generated Narrat...
Chuer Chen, Nan Cao, Jiani Hou, Yi Guo, Yulei Zhang, Yang Shi · 2023-08-22 · via cs.HC updates on arXiv.org

Visualizing the insights of the invisible music is able to bring listeners an enjoyable and immersive listening experience, and therefore has attracted much attention in the field of information visualization. Over the past decades, various music visualization techniques have been introduced. However, most of them are manually designed by following the visual encoding rules, thus shown in form of a graphical visual representation whose visual encoding schema is usually taking effort to understand. Recently, some researchers use figures or illustrations to represent music moods, lyrics, and musical features, which are more intuitive and attractive. However, in these techniques, the figures are usually pre-selected or statically generated, so they cannot precisely convey insights of different pieces of music. To address this issue, in this paper, we introduce MusicJam, a music visualization system that is able to generate narrative illustrations to represent the insight of the input music. The system leverages a novel generation model designed based on GPT-2 to generate meaningful lyrics given the input music and then employs the stable diffusion model to transform the lyrics into coherent illustrations. Finally, the generated results are synchronized and rendered as an MP4 video accompanied by the input music. We evaluated the proposed lyric generation model by comparing it to the baseline models and conducted a user study to estimate the quality of the generated illustrations and the final music videos. The results showed the power of our technique.