PROMPT ENGINEERING REIMAGINED FOR BETTER USER EXPERIENCE
TEAM
Dr. Setor Zilevu (Instructor)
Helen Fang
Yanrong Feng
ROLE
UX Researcher
UX Designer
TOOLS
Figma
Midjourney
PROJECT OVERVIEW
This project is based on on OpenAI's Democratic Input to AI grant program. The policy statement worked on for this project was to explore ways in which AI models can balance and reduce the possibilities of generating homogenous or diverse outputs in order to make AI more inclusive.
THE PROBLEM
The initial prompt question provided by OpenAI
When generative models create images for underspecified prompts like “a CEO”, “a doctor”, or “a nurse”, they have the potential to produce either diverse or homogeneous outputs. How should AI models balance these possibilities? What factors should be prioritized when deciding the depiction of people in such cases?
LITERATURE REVIEW
KEY INSIGHT 1
KEY INSIGHT 2
KEY INSIGHT 3
PROJECT OBJECTIVE
based on the insights provided in the literature review
Given this context, the objective of this project is to explore ways to create an unbiased version of the text-to-image AI model, and to mitigate representational harms, and promote the representation of diverse cultures and visual identities. In addition to this, the aim is to empower users with greater control over the AI generated images.
How can the integration of content mindfulness and prompt refinement in text-to-image AI systems empower digital content creators to produce inclusive and representative visual content, while mitigating the perpetuation of biases and stereotypes?
PROJECT HYPOTHESES
Hypothesis 1: Prompt refinement and guardrail approach to help solve the problem of underspecified prompts.
Hypothesis 2: Designing the output UI to showcase diversity by create a place for user customization based on location, preferences and many more criteria.
USER RESEARCH
USER INTERVIEWS
6 Users
Discussion Guide
SURVEYS
23 Participants
100% - noticed biases and stereotypes
70% - 'often' or 'always' tweak their prompts
43% - spend 5 to 30 minutes fine-tuning prompts
13% - spend 30 minutes to an hour fine-tuning prompts
Survey Questions
INSIGHTS
HIGH APPRECIATION FOR SPEED AND EFFICIENCY
Most users value the system’s quick and efficient generation of results
PROMPT ADJUSTMENT IS A KEY STEP
The fact that all users engage in prompt adjustments highlights its importance in the user journey.
NEED FOR BETTER PROMPT GUIDANCE
Many participants seek for clearer instructions for creating effective prompts
Common frustration in achieving the desired image
LOW-FID PROTOTYPE
3 VERSIONS OF PROTOTYPES BASED ON DATA ANALYSIS
USABILITY TESTING
Protocol
Task 1: Choose a Prompt
Participants are asked to choose one of the three given underspecified prompts.
Task 2: Generate a result in Midjourney
Generate in Midjourney. They are encouraged to comment on their opinions regarding the results.
Task 3: Prompt Refinement with our Prototypes
Use our prototypes to refine prompts and rate their experience on a scale of 1 to 5.
Task 4: Regeneration in Midjourney and Feedback
Share thoughts on the suggested prompts and prototypes.
INDIVIDUAL INSIGHTS
view details
Prototype 1A) and 1B)
INSIGHT 1
Preference for the V2 over V1 as it aligns with their previous experiences.
INSIGHT 2
The drag and drop feature of the text blocks changing the sentence structure may affect the purpose of the prompt.
INSIGHT 3
Prototype 2
INSIGHT 1
The users might be overwhelmed with so many options in the detailed dropdowns. The sentence structures might be restricted based on this.
INSIGHT 2
AI is usually monochrome and therefore seems like the colors are fake and might cause one to not take the output seriously.
INSIGHT 3
Prototype 3
INSIGHT 1
Users liked the suggested prompt options and the template format, finding them inspiring and efficient.
INSIGHT 2
Users questioned the prompt generation limits and the reasoning behind suggestions.
INSIGHT 3
OVERALL INSIGHTS
THE ULTIMATE GOAL
To help increase user control when generating prompts for AI generated images
HIGH-FI PROTOTYPE
CONTENT MINDFULNESS
As users start typing, relevant categories will be highlighted and checked. This indicates that the content they input aligns with these content mindfulness parameters.
PROMPT REFINEMENT
If the user isn’t happy with the prompt or the image, a “Regenerate” button is provided to regenerate more results either on their current prompt or a new option.
HISTORY
History allows users to see the previous generated images. By clicking on one of the images, it opens up the metadata of the prompt.
FINAL DEMO
IMPACT
CONTENT CREATION & MINDFULNESS
DIVERSE CONTENT
UNDER SPECIFIED PROMPTS
RAISING AWARENESS
RESPONSIBLE / THOUGHTFUL USE
SUCCESS METRICS
LESS BIASED RESULTS
CLICK RATE
TIME-SAVING
SATISFACTION RATES
EFFECTIVENESS
CRITICAL THINKING ON HUMAN-IN-THE-LOOP
Currently, there is not much content mindfulness solution for text to image AI in the market. Therefore, our goal is to help users craft inclusive and efficient prompts on the fly.
USER CENTRIC FOCUS
In order to fully absorb the research question, we took a much longer process narrowing down our research question and form hypotheses. The prototype reflects our commitment to creating a user-centered and ethical text-to-image AI system.