Songyan Teng

songyanstteng@hotmail.com

I am a PhD Candidate in Human-Computer Interaction with interests and experience in smartphone sensing, natural language processing (NLP), and large language models (LLMs). My work focuses on gaining deeper insights into human behaviour by capturing and analysing smartphone usage patterns. By leveraging techniques such as LLMs, machine learning, and statistical analysis, I aim to develop advanced methods for processing smartphone data, ultimately enhancing our understanding of the complex relationship between people and their devices. I am supervised by Prof. Vassilis Kostakos and Dr. Simon D'Alfonso.

Alongside my research, I love programming and creating new tools for people to use. I am also a music enthusiast who enjoys playing and making music, as well as working with music and multimedia technology!

View My CV

Education

Doctor of Philosophy

The University of Melbourne
Human-Computer Interaction
Thesis: Enhancing Smartphone Sensing Through Screen Text
September 2022 - Present

Bachelor of Science (Honours)

The University of Auckland
Computer Science (Awarded with First Class Honours)
Dissertation: Evaluating the Effectiveness of Responsive Visualisation within a Breathing Application
January 2019 - June 2022

Publications

  • Songyan Teng, Simon D'Alfonso, and Vassilis Kostakos. Understanding User Behavior in the Wild Using Smartphones. In Handbook of Human Computer Interaction. Springer, December 2024.

  • Johnny Chan and Songyan Teng. The Design and Implementation of PAGE: Personalised Assessment Generative Engine. In Pacific Journal of Technology Enhanced Learning, volume 6, page 33-46. Auckland University of Technology (AUT) Library, December 2024.

  • Songyan Teng, Tianyi Zhang, Simon D'Alfonso, and Vassilis Kostakos. Predicting Affective States from Screen Text Sentiment. In companion of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp Companion '24. ACM, October 2024.

  • Tianyi Zhang, Songyan Teng, Simon D'Alfonso, and Vassilis Kostakos. Leveraging LLMs to Predict Affective States via Smartphone Sensor Features. In companion of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp Companion '24. ACM, October 2024.

  • Songyan Teng, Simon D'Alfonso, and Vassilis Kostakos. A Tool for Capturing Smartphone Screen Text. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24. ACM, May 2024.

Research and Technical Experience

Advanced Technology Group Intern

Dolby
  • Generating personalised multimodal experiences using smartphone sensor data
  • Developing and deploying multimodal large language models (MLLMs) for adaptive multimedia generation
  • Exploring how to effectively transform textual data into corresponding video, audio, and light outputs
  • Python, Large Language Models, Digital Signal Processing
May 2025 - Present

Research Assistant and Software Developer (Emotion Regulation through Music Generation)

The University of Melbourne
  • Developing a Spotify-based web application to generate playlists based on user moods
  • Researching how custom playlists can shift users between affective states
  • Node.js, MySQL
September 2024 - June 2025

Research Assistant (Smartphone Sensing and Wellbeing)

The University of Melbourne
  • Investigated how smartphone usage can serve as a predictor of health and wellbeing
  • Implemented AI and machine learning techniques to analyse data correlations and identify patterns
  • Python, Large Language Models
April 2024 - July 2024

Research Assistant and Software Developer (Exploring Generative AI in Education)

The University of Auckland
  • Built a tool to automatically create versioned assessments
  • Researched methods to generate dataset variations using synthetic data
  • Coordinated team members to achieve project outcomes
  • Node.js, Large Language Models
February 2023 - December 2024

Software Engineer Intern

Serato
  • Created various web applications to support product delivery and manage transactions
  • Implemented a web-based music visualiser application
  • Vue.js, PHP, Audio Processing
November 2021 - February 2022

Course Development Project Manager

The University of Auckland
  • Led the creation of the University of Auckland course: Introductory Business Analytics
  • Researched and organised content in developing an introductory course for all students within the Business Analytics department
  • SQL
July 2021 - December 2021

Software Developer

SYQEL
  • Created responsive music visualisers according to audio input
  • Researched and implemented efficient methods for constructing music visualisers
  • Worked on an application allowing users to submit and share their own visualisers with other users
  • React, Three.js
January 2021 - May 2021

Research Assistant (Cognitive Engagement and Debugging)

The University of Auckland
  • Conducted user studies to collect eye-tracking, EEG and GSR data for analysing program debugging behaviour
  • Created data-analysis tools to deduce patterns and identify behaviours beneficial to the debugging process
  • Communicated research results via oral presentations and written reports
  • Python, SPSS, User Studies
November 2020 - March 2021

Research Projects and Code

MoodDJ

  • Web application that generates Spotify playlists to help users transition from their current mood to a desired emotional state
  • Node.js, MySQL

StressLess Shell

  • Prototype of a device that can identify and relieve user stress through mobile phone activity
  • Machine Learning, Data Analysis, Arduino

ACM UbiComp Conference Website

  • Website of the ACM UbiComp 2024 Conference
  • HTML, CSS, WordPress

ACM MobileHCI Conference Website

  • Website of the ACM MobileHCI 2024 Conference
  • PHP, CSS

AWARE Sensing Software

  • Smartphone sensing framework developed for Android, consisting of multiple sensor technologies
  • Java, Android Studio

CIS Doctoral Colloquium Website

  • Website of the CIS 2023 Doctoral Colloquium
  • HTML, CSS, Squiz Matrix

Personalised Assessment Generative Engine (PAGE)

  • Engine for generating randomised datasets and assessment questions within Canvas LMS
  • Node.js

Breathing Visualiser

  • Real-time visualisation of breathing through a microphone
  • HTML, CSS, JavaScript, Audio Processing

Teaching Experience

Head Tutor

The University of Melbourne
  • Semester One, 2025
    • COMP10003: Media Computation
    • COMP90059: Introduction to Programming
    • INFO20005: User Interface Development
  • Semester Two, 2024
    • COMP90018: Mobile Computing Systems Programming
    • COMP90059: Introduction to Programming
    • INFO10003: Fundamental of Interaction Design
  • Semester One, 2024
    • COMP10003: Media Computation
    • COMP90059: Introduction to Programming
  • Semester Two, 2023
    • COMP90018: Mobile Computing Systems Programming
    • INFO10003: Fundamental of Interaction Design
  • Semester One, 2023
    • COMP10003: Media Computation
February 2023 - Present

Course Coordinator

The University of Auckland
  • Semester One, 2023
    • BUSAN 201: Data Management
  • Semester Two, 2022
    • BUSAN 201: Data Management
  • Semester One, 2022
    • BUSAN 201: Data Management
    • BUSAN 300: Data Wrangling
  • Semester One, 2021
    • BUSAN 300: Data Wrangling
February 2021 - July 2023

Graduate Teaching Assistant

The University of Auckland
  • Semester One, 2022
    • INFOMGMT 192: Information Tools for Business
  • Semester Two, 2021
    • INFOMGMT 192: Information Tools for Business
  • Semester One, 2021
    • INFOSYS 222: Database Systems
February 2021 - July 2022

Leadership and Volunteering

Web Chair

ACM UbiComp Conference
  • Created, maintained, and updated the website to present information about the conference
  • Met with the committee to plan conference events and scheduling
2024

Web Chair

ACM MobileHCI Conference
  • Created, maintained, and updated the website to present information about the conference
  • Met with the committee to plan conference events and scheduling
2024

Seminar Coordinator

Human-Computer Interaction Group, The University of Melbourne
  • Organised weekly HCI-themed seminars
  • Communicated about seminar requirements with internal and external speakers
  • Provided technical assistance and facilitated seminar sessions
  • Promoted seminars via social channels
2023 - 2024

Communications Officer

Computing and Information Systems Graduate Research Student Group, The University of Melbourne
  • Welcomed new graduate research students to the faculty
  • Met with the committee to plan student events and scheduling
  • Sent announcements about student events
2023 - 2024

Student Volunteer

CHI 2023, MobileHCI 2024, UbiComp 2024
  • Assisted with in-person organisational activities at academic conferences
  • Planned conference events for large attendee groups
2023 - 2024

Website and Registration Coordinator

Computing and Information Systems Doctoral Colloquium, The University of Melbourne
  • Created, maintained, and updated the website to present information about the colloquium
  • Met with the committee to plan colloquium events and scheduling
  • Promoted the colloquium via social channels
2023

Skills

Programming Languages & Tools
  • Python, Java, C#,
  • HTML, CSS, JavaScript, PHP, React, Vue.js, Node.js, Three.js
  • SQL, MongoDB, Database Design
  • Large Language Models, Artificial Intelligence, Natural Language Processing, Git
Research
  • Designing and Conducting User Studies
  • Data Collection (In-the-Wild Studies, Lab Studies, Interviews)
  • Data (Quantitative, Qualitative, Mixed Methods) Analysis

Awards & Scholarships

Google Australia Research Day Travel Scholarship

Google
Invitation to attend Google Australia's inaugural Research Day.
May 2024

Research Training Program Scholarship

The University of Melbourne
Awarded to graduate researchers based on previous academic merit
September 2022

Senior Scholar Award

The University of Auckland
Awarded to graduates who received the highest overall grades in their undergraduate degree
May 2022

Postgraduate Honours Scholarship

The University of Auckland
Awarded to postgraduate students who achieved top grades in their undergraduate degree
July 2021