Pytanie – W jaki sposób zbadać skuteczność pracy specjalisty ds. użyteczności na etacie?

W polskich serwisach “pracowych” powoli zaczynają pojawiać się propozycje dla specjalistów ds. użyteczności (zwykle bez dookreślenia, jakie mają być główne zadania i rola takiego specjalisty). W moim odczuciu można zauważyć dwie główne grupy pracodawców:

• software house/portale webowe nastawione na długie projekty, tworzące jeden/dwa produkty
• agencje interaktywne i webowe realizujące wiele projektów jednocześnie

Bardziej niż miejsce w którym ma się znajdować dział użyteczności (więcej na blogu Adama Plony we wpisie Rola projektantów w strukturze korporacji) interesujące jest, w jaki sposób można mierzyć i oceniać zatrudnionego specjalisty do spraw użyteczności. Ciężko jest znaleźć jakąś miarę, a jeśli nawet uda się ją zastosować, to jest ona niepełna. No bo co oceniać?
Ilości wyprodukowanych prototypów? Gradienty na mockupach?  Elegancję architektury informacji? Wyniki badań (mówiące że wszystko jest do poprawy :)? Zadowolenie klienta?

Część odpowiedzi na to pytanie chociaż z nieco innej strony, znalazłem w prezentacji Katriny Alcorn How to Manage a UX Team:

How do you know there’s a problem?

1. Have regular 1-on-1s with each team member
2. Check in with clients and business partners
3. Establish good relationships with other disciplines
4. PMs, especially, are your canaries in the coal mine

Common performance issues

1. Great ideas, but poor presentation
2. Dificulty collaborating with project team
3. Poor time management; leads to sloppy work
4. Unmotivated, thinking is lazy

Idąc dalej tym tropem, znalazłem wpis “Performance Management” na blogu Managing UX teams:

Here are some tips to help you set up a performance management process for your team:

1. Be clear about what’s expected
An obvious statement, but how often do we fudge our way through this?  Publish a ladder of levels for UX practitioners, with performance expectations for each level.  AIGA gives you a summary of the various roles, but you’ll need to add detail that’s specific for your org.  Meet (at least) quarterly and set objectives and key results for the quarter based on these expectations and on what your business needs to achieve.

2 . Factor in your team member’s career goals
Certainly, work has to get done.  But once you’ve set team member goals and expectations that support the business, you’ll need to factor in what the UX practitioner wants to get out of his/her experience with your org.  Add these goals to the mix of quarterly objectives and key results. (…)

3. Get peer and client feedback
It may sound obvious, but when evaluating your team members, collecting data from those who work closely with them is the only way to get rich feedback.  Ask your team members’ peers about their strengths and areas of opportunity.  Be transparent and share everything you get with your team member (and don’t forget to let their peers know that you’ll be sharing the feedback with them).

4. Use a consistent scoring system
Break down expectations into categories based on the predefined expectations (like “quality of work”, “ability to lead projects”, “complexity of projects”).  Taking peer and client feedback, as well as your own observations, score your team members on each category.  Once you’ve done this for everyone at one level, compare scores to ensure that they’re well calibrated.

5. Track results over time
Maintain scores and your notes on team member performance over time in one document.  This will allow you to monitor trajectory and to catch any areas of opportunity that were previously addressed but are creeping back.

6. Delivering the news
Deliver feedback quarterly.  Very important: Write your feedback prior to meeting with your team member, and do so in the context of the expectations and goals that were set.  Having a trajectory of scores will help color your commentary e.g. you’re continuing to improve in this area, but have leveled off in this other area.

W załączniku wspomnianego wpisu znajduje się plik XLS Performance management tool, który może być dobrym punktem wyjścia.

Najciekawsze znalezisko (na tym samym blogu) to “Margaret’s View of Performance Assessment” autorstwa Margaret G. Stewart (mstewart@google.com) – któtki dokument dobrze opisujący elementy podlegające ocenie:

Inputs

• Job ladder descriptions
• Peer reviews
• Feedback from key stakeholders that you meet/interact with (PMs, Eng)
• My own observations at team meetings, product team meetings, Design Review meetings
• Our one on ones
• Your weekly reports
• Reviewing your quarterly goals and, at the end of the quarter, your scored goals to assess completion
• Emails I receive from you and others talking about recent accomplishments/launches/etc

Design work

• Quality of work delivered during a quarter – ie, clever, clean, innovative designs
• Efficiently and effectively applying our company’s design principles to your work
• Ability to thinking conceptually about high level user experience problems/opportunities
• Ability to deliver finely tuned tactical solutions as needed
• Ability to work fast under tight deadlines
• Size and complexity of design problems in your projects
• Getting experiments and product launches approved
• Contributions to UX infrastructure projects (ex: serving on Style Guide committee)

Presentation Skills

• General preparedness and presentation style in Design Review
• Knowing your audience and delivering the right kind of data/prototypes to sell ideas

Productivity

• Manages multiple projects and activities without losing track or missing deadlines
• Significant progress made on planned tasks (ie, how did you do on your quarterly goals?)
• Quantity of work delivered during a quarter

Collaboration

• Working relationships with PM, engineers, UXers w/ you
• Being pro-active in sharing your work in design sessions, team meetings, etc
• Taking a keen interest in the work of our research colleagues
• Participating in field studies and any resulting brainstorms
• Using our limited research resources wisely (ie, don’t insist usability testing be done for something if there is no time to adopt findings, etc)
• Taking advantage of specialists’ help (VisDe, UX Writing)

Developing yourself and your skills

• Setting and working towards long term professional goals and associated skill development
• Choosing conference and training opportunities thoughtfully with an eye to professional development

Team work/ Attitude / Communication / Negotiation, etc

• Helping with process improvements (ex: drafting Consumer Noogler materials)
• General positive/productive attitude towards work and colleagues
• Pinch hitting for others
• Giving recognition/feedback to colleagues who deserve it (peer bonuses, emails cc’ing their manager, etc)
• General willingness to do what needs to be done
• Ability to respectfully push back when you don’t agree
• Ability to assert/stand up for yourself and your ideas with tough stakeholders, but not so often that you are perceived as being difficult to work with
• Responding well to feedback, in all its forms
• Listening to and learning from your colleagues
• Proactively seeking out feedback through informal (ie, periodic check-ins with partners, etc) and formal means (ie, optional perf)
• Formal/informal report-outs from conferences and training
• Doing required admin tasks (OKR writing, perf, snippets) with minimum of prodding and bellyaching – no one loves doing it, but it’s the right thing to do!
• Proactively telling me you have capacity or have too much work (helps me do a better job of load balancing)
• Proactively keeping me in the loop (sending me an email to tell me that a launch date has changed and why, or letting me know that there may be a resource issue with a certain initiative)
• Adapting to change: teams change and so do the needs of products and users; showing an ability to stay flexible and open to new ideas and perspectives is an important part of professionalism and leadership

Leadership

• Mentoring new team members
• Mentoring less experienced designers
• Managing/helping run team meetings (weekly, monthly, offsite, etc)

Wnioski? Ostatni wspominany dokument jest chyba najpełniejszą odpowiedzią na zadane w temacie poniższego tekstu pytanie. Mimo to, całe zagadnienie wydaje dość mało udokumentowane. Bardzo mnie też ciekawi, jak obecnie wygląda ocenianie specjalistów użyteczności w Polsce? Czy mierzone jest coś więcej niż tylko ilość mockupów?