pip/docs/html/ux-research-design/research-results.md

14 KiB

/2020-research-outputs/about-our-users
/2020-research-outputs/ci-cd
/2020-research-outputs/improving-pip-documentation
/2020-research-outputs/mental-models
/2020-research-outputs/override-conflicting-dependencies
/2020-research-outputs/personas
/2020-research-outputs/pip-force-reinstall
/2020-research-outputs/pip-logo
/2020-research-outputs/pip-search
/2020-research-outputs/pip-upgrade-conflict
/2020-research-outputs/prioritizing-features
/2020-research-outputs/users-and-security

UX Research Results

2020 Research Synthesis

Over the course of 2020, the pip team worked on improving pip's user experience, developing a better understanding of pip's UX challenges and opportunities, with a particular focus on pip's new dependency resolver. The Simply Secure team focused on 4 key areas:

Some key outcomes from the 2020 work are:

  • This documentation & resource section!
  • A pip UX research panel (Sign up here!)
  • New and expanded GitHub issues
  • UX improvements in 2020
    • UX work supporting the dependency resolver
    • Improved error messaging
    • Supporting Documentation
  • UX Training for the Pypa + pip maintainers

This work was made possible through the pip donor funded roadmap.

Research Methods

Outreach

We recruited participants for a user research panel that we could contact when we wanted to run surveys and interviews about pip. In total 472 people signed up to the panel, although some unsubscribed during the research period.

At the end of the 2020 research, we asked users to opt-in to a long-term panel, where they can be contacted for future UX studies. Should the pip team wish to continue to build this panel, we recommend translating the sign-up form into multiple languages and better leveraging local communities and outreach groups (e.g. PyLadies) to increase the diversity of the participants.

User Interviews

In total, we interviewed 48 pip users, recruited from the user panel, and through social media channels.

During the interviews, we asked users about:

  • How they use Python
  • How long they have been using pip
  • Whether or not they use a virtual environment
  • If and how they address security issues associated with pip
  • Which pip commands they regularly use
  • How they install packages with pip
  • Their experience using pip list, pip show and pip freeze
  • Their experience using pip wheel
  • Whether or not they use other package managers, and how pip compares to their experience with these other tools
  • What the pip team could do to improve pip
  • Problems they have experienced while using pip, and how they solved these problems
  • Their perception and use of the pip documentation
  • What other technical documentation they value, and how the pip docs could take inspiration from these
  • What other resources the pip team could provide to help pip users solve their problems

Surveys

We published 10 surveys to gather feedback about pip's users and their preferences:

Title Purpose Results
Pip research panel survey Recruit pip users to participate in user research, user tests and participate in future surveys. See associated blog post for more information. 472 full sign-ups
Feedback for testing the new pip resolver Understand use cases where the new resolver fails due to dependency conflicts. See associated blog post for more information. 459 responses via the feedback form, approx. 8 issues transferred to issue tracker
How should pip handle conflicts with already installed packages when updating other packages? Determine if the way that pip handles package upgrades is in-line with user's expectations/needs. See related blog post and GitHub issue for more information. See write up, including recommendations
Learning about our users Learn about pip's users, including:
  • their usage of Python and pip
  • why and how they started using Python
  • if they are living with any disabilities, and if so what effect (if any) this has on their usage of Python and pip
  • if they use assistive technologies when using Python and pip and how this work for them
  • where they get support when you have issues with pip
See write up
Buy a pip feature Establish which features are most important to pip's users See write up
Should pip install conflicting dependencies? Establish whether pip should provide an override that allows users to install packages with conflicting dependencies See write up
How should pip force reinstall work? Establish whether or not pip force reinstall should continue to behave the way it currently does, if the functionality should be changed, or if the option should be removed See write up
Feedback on pip search To establish whether or not to remove or redesign pip search. See this GitHub issue for more information. See write up
Design brief for pip's logo To gather information and inspiration from the community to form the basis of a design brief for pip's logo. See design brief
Feedback on pip's docs To gather feedback on pip's docs, supplementing feedback gathered in user interviews See write up

Research Results

Below is a compiled list of all research outputs and recommendations made by the pip UX team based on the research conducted in 2020.

Title Category Description
About our users Who uses pip High-level summary of who uses pip. Includes recommendations for supporting languages other than English, supporting users with disabilities, and improving pip's output
Pip personas Who uses pip Defines and explores three Python user personas
Mental models of pip Who uses pip Explores users' general knowledge of package management, what pip is, and what pip does during an install process.
Behaviours and attitudes towards code security and integrity Who uses pip Explores pip users behaviour and attitudes towards security and makes recommendations on how to improve pips security experience
Usage of pip in automated and interactive environments Who uses pip Assessment of use of pip in automated environments (i.e. continuous integration, continuous deployment) vs manual input from the command line
Improving pip's documentation Documentation Summarises how pip users get pip help, and make recommendations on how to improve pip's documentation
Pip's identity: In search of a logo Pip community Summarises community ideas for a new pip logo as input for a design brief
Prioritizing features (buy a feature) How pip works Summarises which features are most important to pip's users
Pip Search How pip works Summarises current use of pip search and makes recommendations on how to move forward with pip search, given that PyPI XMLRPC search has been disabled
Pip Force reinstall How pip works Looks at at current use of `pip --force-reinstall` and whether the current behavior matches users expectations
Dependency conflict resolution when upgrading packages 2020 dependency resolver Recommends whether pip should take into account packages that are already installed when a user asks pip to upgrade a package
Providing an override to install packages with conflicting dependencies 2020 dependency resolver Recommends weather or not to provide an override for users to install packages with conflicts (the new pip resolver blocks this behaviour by default)

Read More