Transcription

WENDIE: Hello.  This is awesome.  Welcome everybody, did you have a good lunch?

All did you like the food?  I loved it.

The door is closed we are ready to start, are we ready to start?  What is?  …

I am in front of the screen?  Oh my god! Thank god, it is my first time as an MC so I am Wendie I think that is enough for now.  I will be MCing the whole afternoon, I will introduce the speakers and helping out with the Q&A, I try not to be in front of the screen or inside the light.  I am going to keep it really short.  There has been some practical information I needed to share with you today.  We just swallowed our lunch but some information about dinner, it is going to be later than previously announced so it is going to be at 7:30.

Forgot the location but, it is still time to find that out.  Today, we are going to start this session with Tammie Lister, it is on the screen.  Know your users.  I talked to Tammie find out some things, I talked to her yesterday.  I she was one of the first women speakers I saw at WordCamp 7 years ago.  We are the same age, we have both a very, um, exciting lives and she is going to tell you about why it is important to know your users big, big applause for Tammie.

TAMMIE LISTER: So this is phrase, almost seems like a mantra for most designers.  I am sure that non-designers say it of:  You are not the user.

Why is that said?

Often when that phrase is said people think, well it is actually creating for users a bad thing?  How bad can that be?  You are creating a product, passionate about a product, why is it so bad to do that?  The thing is it gives a reminder, a note to truly create the right experience, you have to get out of your head space.  You have to get beyond yourself and see other people that are going to use your product.

Perhaps a better way of seeing this would be you are not the only user.

But if you are not, how do you actually create for the users for all the users and how do you go about doing that?  So that is where the why of user research comes into it.  A story that is told in ancient cultures who aren’t visually looking, touching and describing a part of the elephant.  Some would touch the head and perhaps said that is the basket or maybe touch the body and that is a granary, or maybe touch a leg and it is a pillar, nobody would be able to get what they were touching would be a elephant, they are touching a small part and then trying to describe it.

Without knowing the full picture, how can you create something?  Truth comes from seeing that full picture and the truths only come when you do user research.  Without that your frame, design, code at the start code, probably not in the same room as the dart spinning, it might be spinning or circular, you are not going the do it.  You are never going to succeed with what you are trying to do.

By doing research, you open up this better way of designing more appropriate design.  To me, this Erika Hall, if there is one book that you have to read on user research, and at the end of my talk there is a list of resources I would like the share with you.  I would like to say in her book, Just Enough Research, she says this leads to more design solutions than merely asking how you feel or tweaking the current design based on analytics.  Beyond your hunches or the surface information.

Whenever the subject of research comes up it is like a carnival of but’s happen.  It a breaks the flow, it just rains all the excuses and reasons why you shouldn’t do user research.

Assuming it costs a lot is a fair assumption, there is a lot of media articles say that, user research costs a lot, you shouldn’t do it.  There is a notion beyond money that the time cost is huge, barrier, prevents this process and the flow of creating which is incredibly important that momentum of creation.

The thing is, research probably is on a small scale happening right now within the product you are creating or with your clients.  It just isn’t being communicated.  It is one of the missing pieces of research, communication of the research, all very well on doing it.  But if you are not having the communication of it.

Customer support is like daily user research.  People are giving responses, people are saying what you know, in the interactions.  It is really important to give space and communicate those opportunities of user research.

Listening is the foundation skill of this and listening, it is kind of free.  It is something that you can get better at and it is a skill you can improve.

The notion of just doing research is something really important and that is what I would like to get you in thinking about today, do it, test the water, see it as part of that real flexible fluid, easy flow, it doesn’t have to be this heavy burden that oh, now we have got to do user research in this phase or now we have to do this.  It should be fluid and free.

There is also this fallacy that one type of research really is the way.  People would say, oh user research should be done this way and this process should be done.  That is not true.

I would like to dispel that.  Adaption is really the key to speed and to prevent blocking the process.  It is going to be different for each person doing it for the situation doing it.  And we are getting into how that happens in a bit later.

Because I think that is one thing that is said against user research, this immovable mountainous task and it is a bit like doing your homework, you have got to the it.  Ugh I have to do user research, it has that weight.  So by being lighter on that …

If it is a business you can’t say, yes, these are my users how simply are you going the optimise to make the best solutions, maybe you have a weird divination process I don’t know about but I don’t think that is the case.  Doing the right amount; the right type of research; that is simply a smart business decision.

This is something so very important that you get from knowing your users.  Your work responds rapidly to the user demand as a product grows as well.

Actually doing the right research is economical and that is a table flip I would like you to make.

We are faced with the but’s about doing research, to response here I think is really apt.  John says we remind everyone to say that really waste time and money, all you need to do is to build a service that nobody wants and can use.  That is pretty apt I think.

Time, money, energy and reputation all these things can actually break a product and if you are not doing user research, you are opening up to making all the things happen.  You are making it inevitable that all the things happen.  So hopefully, I have convinced you that community research is a good idea.  How you go about doing it?

I would like to move into doing that a little bit.  First of all, I have a question, who has ever used revisions in at all in WordPress?  Put your hand up?  Cool, excellent.

Who here who put their hand up for revisions, who here has actually reverted a revision?  Cool, awesome.

So, I just did some user research.  I asked a question.  Then I documented it.  It is really as simple or as complicated as the goal that you are going to do it.  It can really be frictionless, I now know in this room in this particular time the amount of people that had their hand up.  I have the documentation of that.  That is, that is useful information if you look at it in the right perspective as well.  That is the key, you have to use judgment with research as well.  You don’t need  — Scrooge McDuff’s, user research, you can do it.

It is not an exact science, it is something you have to learn as a skill.

There isn’t a clear right or wrong way with research.  Going back to that adaption you really need to be flexible with your approach.  It is a craft.  You know, learn to adapt and I think that is something that anyone can learn as well.  It is a craft that anyone can participate in.  It isn’t a process, it isn’t something that should be done one off.  Whenever someone treats research like a dot at the end of the sentence, it kind of makes me feel that is really the wrong approach, it shouldn’t be a check mark that you do it should be iterative and part of your process.

Research is effective when you have this plan and when it is part of the cycle and as you grow user research, as long as the  — grows it is adapting and you do change and you do different type of research as well.

I spoke a little bit about the kind of no right or wrong way.  However there is a different approach.  I would like to show a bad approach to maybe doing a survey.  I have heard people say this, surveys are required for this product because everybody does them.  Surveys are a great idea.

This is bad, there is no goal, they are saying that because surveys are the new hotness we should really do them.  That is not going to get you anything.  What you need to do is to have a goal and make it testable.  So something like this, I created a sign up flow, I want to see if users are able to have an easier experience, it is testable, work out how you are going to test and get the results and see whether that is actually happened.

That is really, really important to have that focus on your user research.  To be truly effective, user research has to be part of a culture and I have seen this time and time again, user research is allocated to a particular person, that is a limited approach and not something that you will be able to scale or not.  Or not something that is used the research the most.  It is important that it is something that not just a single role does but because when you do and when you live and experience research it is really important and empathic connection happens.  Important not just for designers but important for developers and anyone involved at any stage within the product to experience that have the feeling because there is this thing that happens in our brains we make a connection when we have that empathy and it is very important to do that when you are making that product and to have that connection and remember that experience.

By working on a tool kit you can make research easier.  It helps others in the team and it makes things easier to do as well.  So, if you have a set of guidelines that people can follow when they do user research, they can pick it up and they can do it.

Maybe there is an outline for user testing or there is this way.  It is something we are working with automatic and it is a great way of doing.  It ensures that you get valid consistent research and also as researchers then can feedback into the tool kit the experience of the people doing the research.  So everyone gets the quick start into research.

One bad approach that I have seen people do is do one of type of research and call that research done.  That is, I am just going to pick the one type and that is all the research I will do for this product.  P doing multiple sources actually gets you the entire picture.  If you think about a painting it will be incredibly minimalistic and rare painting if there was one brush stroke.  Normally several, normally a build up of a picture, that is what you do by getting the different research, you get different signs and then mould this into a picture.

Who is here heard of quantitative and qualitative data?  This is a part about combining those different types of research.  So quantitative data tells you something that people is doing.  Quantitative tells you why, quantitive is surveys, analytics anything that can be answered by numbers.  Qualitative, something like usability testing you get a lot more feelings.

By doing the combinations you get a more powerful research so, there is lots of different types you can do, there is card sorting, interviews, post typing, personas usability test, so many different things.  I am not suggesting you do them all but I am saying is, you can look at the project, you can find the right tools for the task; the project and you and then find these really powerful combinations of research to get the results that you want for that goal that you are testing.

The way that my mind works I often try link things to other things, the thing I think about with research, I think a little bit of a TV forensic program.  Bear with me on this.  You have the forensic scientist enter the room, looking for that one tiny piece of tiny little crumb of information, because that crumb of information will put the criminal away and or solve the murder or all those kinds of things.  That is what you are trying to do with user research, find the crumbs of information that will make the product work for the users.

Going back to the cycle, you have to think about this research, you begin with the needs, then you look at the data resources and validate them and share them.  I said earlier, communication was a large part of getting, getting that message out you have got all the awesome user research, but if you don’t deliver it in a way that people understand, then it is just some data here that is not being empathised by anyone.  Depending on the situation, you then need to rinse and repeat.  So a very quick process flow might look something like this, write down an assumption, define a hypothesis, how to test this?  Going back to, you should always have a goal in mind when you are doing user research, not just oh well, this amazing AB testing I read in article media, now I need to do AB testing.

Make the change to the product.  Then measure the outcome of the change.

This is like, this is frictionous and you can add this to any part of the process, okay, tools down, do all the user research for the next 4 months.  That is not the best way, the way is to fit in with your project and do this.

So when it comes up a lot of times speaking about user research, sample size.  So it turns out around about 5 is a good number.  But the problem with this is, 5 for good number for usability testing again it is like mm, maybe?  The thing is if you get 5 of exactly the same results, then no, doing the same thing again is not a good idea because you are going to get the same results.  You know for the groups you are testing, you get the same results, you don’t have to do more than that.  If you get varying results of course a larger sample size, if you are doing simple tests and usability test you will need to have different sample sizes as well.  So about having a bit of intelligence and noted taking things rigidly, it is about the right goal and testing positions as well.

Don’t get me wrong in saying this.  I hope I will be clear, not all research is good research.  It is about quality not quantity just because you have done a lot of research, doesn’t mean you know your users.  It is how you do it and how you then communicate it.  Bad research turns out really easy to do, I have done bad research myself and I am sure a lot of people have as well.  Think of it as a skill, to study, refine and constantly learn, respond to users as well and as you do that you level up.  I learned so much from other peoples user research as well.  Because then I can learn the good and the bad ways of doing it.

Now saying about communicating the actual research one of the big things you want to communicate is the stories because your users are experiencing the products, so these stories are what will become the foundation of how you do changes.

Donna Lichaw I said on the communication side of changes I highly recommend her book “the user’s journey” because she looks at using story arcs to accurately convey the story of users and I would like to pause and show the way she does this because to me this is a really easy and really understandable way of doing it and it’s a way to communicate it so not just someone who has done the research understands but everyone involved in the project understands as well.

And the reason stories work and they resonate and by filtering stories and surfacing them out is important to us; as humans, stories make us connected, back to Palaeolithic times, stories power the human brain.  And there are other ways of doing it and I’ll look at other stories after I look at the story arc.

A story arc for her would start with the expedition, have a rising action, a climax and falling action and going to closure.  Think of a lot of films.  One of the examples in her book is back to the future which us great way of doing that as well.  You can see as it goes up you can see how filmic it is as well but think of it in terms of a digital experience, maybe you have record an action, then you have a sign up, then sign up successful and problem solved because sign up can be dicey, problems involved in that, then you go down at closure.  So generally at the end of the closure you would be in a different but better situation than the exposition.

So if there was a home page here you’ve gone through a record of action to sign up at the end, you sign up so you’re now using the service. And it’s these visual representations that you can use to communicate those stories that you find in your research.

A common way is also to do story boarding or comic booking; it’s a great way a format universally understood to us as humans.  Visually you don’t have to be a great drawer.  Do the doodles to communicate the stories of the experiences your users are having and they become real when you do this.

At the time your users come out from the shadows they have these faces, they become people.  It’s a bit like developing a photograph out of film.  And each data point and each information crumb they add up.  And it’s really, really important though that you don’t just freeze personas in carbonite and never go back to them.  That’s not going to be good at all.  You need to see them as fluid and grow and as you discover more data see that you can iterate those as well.  And as your product grows see that you can iterate those also.

Communication is a great skill you need to make sure there is a format for everyone involved and make sure it’s understood otherwise it’s pointless going back to that.  If someone delivers all user research in a great big wall of text, that not going to be accessible, it’s not going to get people buying into it, it’s not going to get the stories, it’s doing the stories and users you did the research on an injustice.  Adapt the report to situation and audience.  Be playful, be open to new ways of communicating that new research.

I think this is really, really an important statement: user researcher’s fallacy.  My job is to learn about users.  Think we’ve all thought that at some point.  The truth is my job is to help my team learn about users, you’re a conduit for those stories, and that’s how you should be approaching this.

When delivering research you shouldn’t judge in the same place that you showed the results because that’s your bias, that is your interpretation of it.  Let people make those own connections.

Going back to other people doing user research, when someone does that there is this kind of brain chemistry that happens when they make that empathic connection with a user they’re going to remember that.  You’ll see someone developing code and say, well, George feels this particular way so, George, I’m going to do this experience because George is not going to hit this road block that he does.  Or Mary always has this road block when she goes to sign up, I am going to add this feature so Mary doesn’t have this.  And I have heard people talking about users in this way because they are engaged have that empathic connection.  It’s back to stories about relating, they’re real.  It’s not just face less avatars that some users researchers created that you didn’t have any buy into and you are just forced because personas are a thing you have to use.  That shouldn’t be the approach.  Everyone should live and breathe and feel the users.  And a huge part of that is enabling and enabling that to happen.

Something I really think is important that not enough people do in fact is snapshots like digestible user research.  Maybe it’s a dashboard or may be the people who you only support weekly or monthly do the top 5 support issues and relay them to developers and designers.  It’s that frontline easy snackable research that is free.  It’s already existing.  It’s just about communicating it and getting it out there.

And it keeps that constant temperature.  For the users it means people can go, okay, yeah, that’s what our users are experiencing currently as you change things as well and it avoids that burden, heavy weight and mental perception of weight of research.

And data is always going to be the smaller opinion. You need to acknowledge the limits of your sources and that is important with research.  All research as long as its got context if you do it correctly and you have a goal can really be understood but it’s about conveying that context of that research as well.

It doesn’t mean without perspective that even the smallest subset isn’t valuable.  Opinion is also key to mute yourself in your researches.  The users need to be heard not you that needs to be heard.  We want to be heard but in terms of user research, you should not be the voice that’s heard.

Interpreting is really dangerous so spreading out the problem and solution is really important.  I try one of the methods I try and do at the moment if I’m writing or trying to deliver those stories in the research, I will not comment at all.  I will may be add if I do as a post I would may be add that later or I would try and get to have a group to have those thoughts.  I really would not try and add my voice to that and I think it’s really important you don’t add your voice right there.  You may be see it as more of a discussion or brainstorming or here is the data, here’s what’s come back the stories that have come back, now let’s have a discussion as to those.

I hope what I have shown is that knowing your users is really, really important.  Because when you do, you can create for them and that is kind of what we should be doing.  You can make the best experience possible. And research is not just a one off process.  I want you to stop thinking about that think of it as part of the cycle, make it a habit, part of your culture.  Some of the testing happens for data will do for your code as well then do the same for users.  How you research and really know your users should adapt, it should be fluid.  That approach of a toolkit I think is one that everybody should have that mindset for.  And it isn’t, it really isn’t a privilege of the wealthy agency or the invested company.  It’s not just me standing here saying, oh everybody should use user research and if you don’t have the budget to do it you’re not going to be able to do it anyone can do user research and that’s important.

It doesn’t cost and it doesn’t take months to do if you have this adaptive fluid approach to it.

And knowing your users matter for everyone involved.  Shouldn’t be a single role or just something user researchers do or designers do.  It should be developers product managers – everyone should be engaged in doing this because it makes sense.  When it’s part of your P process your users become part of your process so of course you are going to be creating for them.

I wanted to leave some space, I wanted to either answer some questions but I also want to maybe hear your experiences of user research as well.

So thank you.  {Applause}.

WENDIE: So that was interesting at least for me.  I hope you found it as interesting as me.  Are there any questions at the moment?

TAMMIE LISTER: So if you don’t want to talk about user research I’m happy to talk about anything at all to do with design.  Any questions?

FROM THE FLOOR: Maybe you can share some user research practices.

TAMMIE LISTER: I’m also going to skip forward to this –

FROM THE FLOOR: Can you repeat the question?

TAMMIE LISTER: In a second but I’ll also say these are some resources I’ll recommend as well.  The slidex is going to have this but –

So I would like to convey some bad user testing experiences.  I’ll use my own because that’s probably a polite thing to do and I probably made all the mistakes I’ll ever make.  One of the mistakes I think I’ve made more recently is when I don’t – it’s very hard – you can know the context of your research and I think it’s really important to convey that context of that research to people to say this is maybe a small subset but this is an important small subset we’re doing, it’s just about the temperature and part of what I’m trying to do with this talk is show there are these different types of research, that I can believe in a leaner approach of research but if I’m not saying I’m doing that leaner approach of research, someone is going to go, well, it’s just like a few people.

I definitely think the goal thing is big if you don’t have a goal with your research and I’ve done it it’s just like I’m – I was freelance I went to Automattic and I would be like okay, we’re starting a project, we now need to do user research and I had that mindset of right now this is the user research budget and we do it at this point, why I would pick the beginning and only – now I know it’s better but it happens at the beginning, you do all these changes and don’t iterate.  It’s having that making it part of your process and when you don’t then you get the bad things.

So that is definitely some bad – I have made all the mistakes in everything so hopefully I can convey that so people don’t make as many mistakes as me.

Any other questions at all?  Or any other research?  Are people doing user research at all?

FROM THE FLOOR: Just off the top of your head if you can think of what were really surprising things you discovered in the research you’ve done lately that you could may be share which is interesting or any anecdotes?

TAMMIE LISTER: So I kind of have a bad habit of making an assumption about something that then I find out a user really doesn’t have that.  It’s almost like I have the ability to think the opposite of what a user is going to do.  I do that an awful lot.  So I have kind of assumed – time and time again I have assumed recently something is worse than it is and I would say oh no we can’t do this we need to make this feature easier for people and I have this feature content I assumed users were having a lot of problems with and I spoke to in automatic we call our support engineers happiness engineers and I spoke to someone and they were like people aren’t having a problem with it and I was like it’s a bad interface they must be but they were having problems with other areas and sometimes it’s how the area is delivered and I’m maybe thinking with my designer’s head and say, oh it looks horrible users must be having a problem with this.  Particularly in WordPress some subsets of users have learned a particular way to do something and if you radically change something just because of the new hotness or mental model all designers are subscribing to, the users might not be subscribing to that model because they’ve learned a particular way and learned to do a particular thing.  So suddenly changing everything, whilst we want new users and old users it doesn’t do new users existing users justice if I read an article and see a new hot way to do something and think oh this all out to do.

There was discussion yesterday about menu buttons and that was one way.  All designers used to think menu icon was the way to do it, it was the best thing in the world, turns out users didn’t find it great but the problem is we kept it there for a while a lot of times and then we realised we had to put the word menu and that helped but that confused the users that already learned the icon.

You have to be careful about understanding the mental models of the users and mindsets of users you’ve trained with your interfaces as interface matures.  That’s one of the biggest things as a challenge I find that on WordPress.com there is a lot of mindset trained to people.  We want new people but already got people using it and WordPress as a project has that absolutely.  There is a certain way, might be a really weird way to do things, but someone has been doing it that way for years and to suddenly change that is earth shattering for someone it’s like wo, what did you do?!  And I have really bad gut reactions so I try and may be put those out but definitely justify them.  I’m very lucky in Automattic that happiness engineers have an amazing temperature that I can just get that gauge and know and I think doing that and testing assumptions.  I have constantly been surprised at things and I think that is great and that to me remind me how much I should be doing user research as well.

FROM THE FLOOR: I was wondering in your research process at what point would you find it most appropriate to start running AB testing?

TAMMIE LISTER: That’s assuming every research has to have AB testing because I would say it doesn’t.  If your testing is AB testing then your goal would say at what point.  Does that make sense?  I think – sorry to be flippant but I don’t think every test needs AB testing.  I think AB testing is something I’ve heard a lot of people say yes I need to do it and surveys of AB testing are like yes I need to do it for everything.  It’s great to put it out there but there can be other forms, similar results to AB testing that you can do which aren’t AB testing so it’s a case of what is your goal, what does AB testing achieve of that goal?  Then do it.  Just having AB testing at like 80 per cent of your project just because a project manager decided that has to happen in every Agile project development I would not say that’s a good reason to do user research.  In your company and product AB testing has proven time and again to work all the time absolutely have it and have it at that point.  And some companies and some products it’s going to be 80 per cent – I am plucking a figure – at 80 per cent of our products cycle AB testing has proven so many things and it’s proven so many things of the similar thing you are testing, absolutely have it part of your process, but it’s that flexible mindset.  And we’re discovering more about humans.  There is a lot of the brain we don’t know about and as we know more about humans we can get better user research and better testing mechanisms as well.

FROM THE FLOOR: Hi, Tammie.  Do you ever disagree with your users and what would you do in that case?

TAMMIE LISTER: Absolutely because I am not my users and I am ridiculously trained in my own perception.  The way I do it is I tend to not {inaudible} myself and it sounds weird but I really can’t assume that I am a user, I can’t A assume – I am a particular user with a particular experience but I am a designer trained in different ways of doing it so I’m going to see things very differently from someone who is a blogger who is doing a particular thing.  I tend to always put the user before my voice.  I definitely filter.  There are a lot of times in research you can get very varied and some crazy ideas that my designer sense says probably not a good idea we make everything pink just because Vera really likes pink and in her response she has said everything should be pink because pink makes me happy.  That’s probably not a time I would listen to the user unless it is proven like a thousand users have said pink is like the way we would respond to this.  So, it’s having that little bit of common sense but generally the users win over me.  Any other questions?

WENDIE: Last question.

TAMMIE LISTER: I’m not repeating them.  If they’ve got a microphone I don’t need to.

FROM THE FLOOR: This is probably petty terminology some of the way you talk about research seems to me what I call testing so can you just explain is testing research, is research testing are they separate things with an overlap?

TAMMIE LISTER: To me when you talk about usability testing that’s research and I think different people will have different opinion, it’s potentially very subjective.  My opinion and why I’m saying research is – I feel that that is part – you are getting a feedback from the user.  So to me that’s user research.  People may not agree with that but to me we need to be simple in our terms.  We get hooked up in the community designs in particular on certain terms need to mean certain things and it’s very binary but to someone who is learning this if you say user research and you open it and show the tools and show user research – I am not talking about code tests here.  Another person may argue that should be but that’s not what I’m talking about.  I see things like usability testing, AB testing all those things they absolutely are part of user research.  For me the line is am I getting feedback from my users that is something I can then use to my product?  That quantitative qualitative data if I’m using that it’s user research.

WENDIE: All right, time is up.  Big applause for Tammie, please.  {Applause} One thing are you talking about this WordCamp Brighton?

TAMMIE LISTER: No –

WENDIE: But you are one of the organisers because you told me so I know it is in August so it is a long time until then but if you are interested it’s going to be great.

TAMMIE LISTER: Yes it’s going to be 2 days, we’re going to have workshops, probably some JavaScript and UX work shops and a day of talks and a contribution day.  Going to be 200 people, quite small, talks will be open so we’re going to have a call for talks in Brighton is kind of amazing to be in August.

WENDIE: Feel free to sign up as a volunteer, speaker and visitor.  If you want to stay here that fine but we’re changing rooms so if you want to go to another room that’s fine also.

TAMMIE LISTER: Thank you.