Reflection Theme 5

This week we've been discussing design research. This is one of my favorite research topics. That is because it offers such a wide range of options and methods, and (almost) always result in something practical that is easy to account for and easy to explain. I had Ylva Fernaeus as a teacher in another course (Interaction Design) and learned a lot about this kind of research during this. I also learned a lot during my job this summer, at Antrop here in Stockholm. What I think is an advantage of design research is that projects tend to be focused and easily evaluated. Especially if you work with design breifs, where you first design something for maybe a few days/a week and then show it to the customer, get feedback and do the same thing again. It's an iterative process, that lets you avoid many of the problems like high cost for a major change that you need to implement late etc. Working briefly and then low-fi prototype is an effective way of getting a lot of work done, without having to change anything major later.

What I found most interesting this week was the low-fi prototypes. I had encountered this way of designing before, but I really want to stress how effective I think it is. Both as a way to work, and also as a way of making sure a (for example, but not limited to) potential customer is satisfied. During this summer at my job we worked a lot with design, and we decided early together with our customer that we should have weekly meetings where we discussed how the work went. And this later turned out to be really important, since we could work focused for a week, then meet the customer and show our work and explain our ideas, get feedback, and then make the necessary (and as time went by; minor) changes. We saved many hours working like this for sure, and it is something I really recommend when working with design and research for a customer or other part.

As this is my last reflection I would just like to thank everybody for making this course swell. I really enjoyed being a part of group C, and I feel that my ability to write a good thesis has improved, although by how much is to early to tell.

Theme 5

Why could it be necessary to develop a proof of concept prototype?

Sometimes it can be necessary to make sure that you are on the right track when designing a system etcetera. A good way to find out if you are indeed on the right track is developing a proof of concept prototype. This lets you show your ideas to those they might concern, and get valuable input regarding possible changes, things that is worth getting rid of and things that must be included. Why it’s so valuable to get in this face is of course the time/cost factor. The cost of change in a design research project can get really high if you want to make changes at the end of the project. This can be both time-consuming and expensive. The earlier you make the necessary changes, the easier they will be to make, and this also guarantee that you use your time efficiently.

What are characteristics and limitations of prototypes?

To quote Wikipedia on this one: “A prototype is an early sample or model built to test a concept or process or to act as a thing to be replicated or learned from.” It’s a good explanation, and it would be a futile attempt if I attempted to come up with a better one. What I can say though, is that these days prototypes are not the lone kid on the block. Several other methods are used to test your ideas. Mock-ups are one: where you create a simple very narrow website for example, just to test your key features. This can also be called prototyping of course, but when I (and certainly not just me) hear the word prototype I tend to think about something tangible and physical.

Prototypes have several limitations of course. Their very existence is a limitation. A prototype is never a finished product or system, which sometimes makes their evaluation a bit difficult, especially if you include members of the targeted user group. For example things like design could be a problematic thing to prototype, since you cannot work on it to long (its just a prototype after all) but then again the persons you show it to might not like the unfinished or rough sketches that you bring to the evaluation. Tricky.

The article I’ve selected for this Theme is “Alleviating communication challenges in film scoring: an interaction design approach” by J. Phalip, M Morphett & E. Edmonds. I believe that this article uses the theory as guidelines to what needs doing. They identify a problem, divide it into different parts and outline their solution to the problem by doing this.

The explorative study performed in the article makes use of several different methods of data collection: questionnaires, video and sound-recorded interviews, observations in the subjects work environment and oral and email discussions. All these methods have their individual cons and pros, but combined (mixed!) like this they really result in a lot of qualitative data and a good understanding of the problem that is to be alleviated.

I again see that you can successfully combine different qualitative methods so that they together lets you understand the views and struggles of the users or target group. I liked that the authors chose to both interview subjects and observe them. I guess that it is effective in the way that you get both the subjects picture and your won to compare with. That is basically all the new things I learned.

Reflection Theme 4

This week we have been discussing the interesting topic that is qualitative methods. The topic of choice is very broad, and it’s certainly open to debate as exactly which methods should be included here. For me, qualitative methods hold a special place, since I’ve had quite a lot of experience of using these methods. Partly in school during courses such as Evaluation Methods and Interaction Design, but even more during my summer job at Antrop. I’ve tried several different methods during this time, and noticed that if you do it right, qualitative methods can be very rewarding. For example, during Interaction Design, me and a fellow student were developing a system and we had constructed a basic mock-up that showed our general design ideas and concepts. This we then showed to a person of the target audience, well versed in the subject. We spent two hours with him showing him our mockup and discussing it back and forth and received tons of information and ideas for improvement. So when it comes to things like design research and evaluation, qualitative methods are powerful indeed.

This stated, it’s important to know the limits of these methods. One thing you always have to think about is that the result of these cannot and should not be used to point at general behavior or attitudes etcetera. The numbers of participants are almost always too few to draw these kinds of general conclusions. So it is generally inadvisable to use these methods to find out something that can be stated as statistical facts. This is of course nothing new, but still important to ponder.

What you can do however, is use these methods in a mix. If you for instance combine some quantitative method with for instance, interviews, you can get a real good insight into the subject. You find out who does what, and also you get an insight into why this is. I feel a bit sad about that the power of this mix is used quite sparingly (often due to economical and/or other factors). Why I feel sad is because a lot of decisions are made singularly on statistical grounds, without the knowledge of what is the cause of the problem and vice versa. This is a huge waste of resources in my opinion, and hopefully we will see different approaches in the future. That’s all for this week’s reflections!

//Love Larsson

 

 

 

Theme 4

For this theme I selected the paper ”Resolving Ambiguity of Scope in Remote Collaboration: A study in Film Scoring” (Phalip, Jean & Edmonds, 2008) published the Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat. This paper is about the problems that can arise when composers of film scores and directors/film-makers tries to communicate with each other remotely. This communication can result in a lot of problems. For one, simple misunderstandings of words might happen (words like “Spacey” “Erie” etc.), there can be a mismatch of scope or something else. The examples of such things happening are many. So the authors of this article undertakes the task of creating a collective workspace that composers and film-makers can use together to get a better understanding of the other parts ideas.

The paper selected is the second of three: here they have come up with a simple design plan for the workspace. This design plan they then decide to evaluate using a qualitative method. This qualitative method consists of the authors meeting three different persons: two composers and one director. They meet them alone, so that the test subjects don’t affect each other with their views. These subjects are shown the design plan, and for two to three hours, the subjects and two observers discuss their opinions and whether the design is good or not. This discussion can consist of everything from design issues, to anecdotes, to suggestions for improvement or incorporation of other elements. When all three subjects have been interviewed, the result is discussed by the authors and steps are taken to improve the design.

The benefits of this qualitative study are of course the abundant material and insights collected. Things like that are almost impossible to come across when performing quantitative studies. More so, with the large amount of time for each interview, the authors really has time to go deep in the design with the test subject, and really get into what is good and not. The weakness of their choice of method is of course that with only three test subjects, some parts (and probable flaws) are probably overlooked. It’s probably safe to assume that not all film-makers and composers work in the same way. Considering that, it could be a source of error and misinformation to just interview three persons.  

I didn’t learn that much new things from this paper, as we already have discussed some of the basics of qualitative methods in this and other courses. But still it was refreshing and intriguing to see these methods being used “professionally”.

I can’t really point out some main methodological problems in this paper. The choice of method seems quite clear and well accounted for. It generated a lot of material that the authors sought after, and also provided additional input and information that was not even asked for. With all this said there is still the slight problem with the low number of test subjects, but the results are still valid, and larger studies with more subjects will surely be carried out later (or by this time, probably has).

That is all for me this time, hope some of it is worth reading. The first of these three papers can be found here in case of interest:

http://dl.acm.org.focus.lib.kth.se/citation.cfm?id=1517744.1517817&coll=DL&dl=GUIDE&CFID=142077322&CFTOKEN=87353866

Reflection Theme 3

This week I’ve been reflecting on quantitative methods versus qualitative methods and their mixture. My experience of these methods comes down from some work I’ve done in school during different projects, maybe especially during my bachelor thesis and my work at Designkontoret this summer. In my experience, and this I tried to state earlier, the combination of these methods probably is the best way to go about your research. I really like qualitative methods because you can learn so much that is otherwise overlooked in quantitative research methods. Some of the most fundamental and important insights of my studies I’ve gained through qualitative studies, like interviews and other forms of more personal approaches.

This being said, I still think it’s important to include some elements of quantitative methods into your work. This is because that it’s easy to draw the wrong conclusions if you just ask 10-15 people about their habits, insights and ideas. You often don’t get thee whole picture through that method alone. The ability to enforce you qualitative findings with quantitative data is important if you want your material to be as indisputable as possible.

I can see why there is such a debate among researchers when it comes to the choice between these two methods. It isn’t an easy nut to crack. The ones in favour of qualitative methods get accused of not coming up with something that you can draw conclusions from. The ones in favour of quantitative get accused of not going deep enough into the subject, and therefore missing the point. And the ones that try to mix these methods get accused of both things.

For me alone it’s impossible to tell who is right or wrong. Maybe no one is? But the more logical answer would be that all of these people are wrong at times. I think that the question of right or wrong is most easily answered if one looks at the particular study conducted and its key points. How do you answer the key questions of you study? Which method do you use? A method is only good if it helps you prove (or disprove) your theory. And although every choice of method is and should be open to critique, (imagine the opposite!) if you have though enough about the choice and can argument well for the use of your theory, then you have done as good a choice as it is possible to make regarding theory. That is my conclusion.

Theme 3

In the first article, ”Mixed Research and Online Learning: Strategies for Improvement” (Lownthal & Leech, 2009) the authors main focus is the use of different research methods when investigating how the results of online learning differs from traditional face-to-face learning. The main concept of the article is that a combination of different research methods (sometimes called mixed methods) produces the most rigorous and indisputable results. This the article attempts to explain by pulling in various sources and studies, and from them forming both it’s general thesis, and also presents some steps one must undertake if the use of mixed methods shall prove efficient and successful.

The presentation of the arguments and sources follow a quite logical path. This is due to the articles good separation into different chapters, so the reader is able to follow along the text without being flooded with to many sources at the same time. Discussing these sources, that might be the only thing that lowers the general grade for the article. It’s quite obvious from the text that the authors want to form a solid base with related theories and data from other sources and other authors. This does create a problem though: the text gets quite thick with citations, causing it lose some of its momentum, and by losing that, losing its ability to lay out its arguments with clarity. This does not occur on every page, but I found it to be quite disturbing from time to time. It’s a relatively short article (14 pages) but the references take up over five pages. That is a lot of sources and citations. And it’s of course important to remember, and this we’ve been through earlier in the course, the fact that you have a lot of sources does not automatically make your theory a lot better.

The second article: “Emotional Presence, learning and the online learning environment” (Cleveland-Innes & Campell, 2009) I found to be better. The theory is well explained for the reader, so that it becomes very clear what the authors intend with this research, and how they are going to achieve it. The strong theoretical basis then turns into a formulation of the method. The method of choice in the paper is a quantitative method that involves a lot of students from different schools in the U.S. Because of their good explanation of the theory, their method feels quite adequate when introduced and its use feels very motivated.

The data they collect through their study is quite adequate for their purpose. However it’s of course a lot to take in for the reader as it’s simply displayed in huge tables. It takes time and effort to really get into the material so that conclusions might be drawn. They of course assist with this by drawing their own conclusions. I found these to be quite what I expected; for me learning always has been an emotional process. If I come across something that I find interesting and exciting, my ability to take in information and go deep in the subject is really enhanced, and vice versa. This is overall something I believe is overlooked in the learning of today: The power of emotional engagement. If you as a teacher or lecturer can awaken that engagement in a student, that student is very well set for the future of the course. But so many times the opportunity to do something like that as a teacher is often missed or the know-how isn’t there. And that is to me both a shame, and a threat to learning itself. Well, now I just reflected a bit though it wasn’t the real task. Maybe I’m emotionally engaged?

Reflection on Theme 2 and the week that has passed.

This week I have been reflecting upon the question about what theory is, and what it is not. I cannot say that I have come to a full conclusion about what theory is though, despite my (and all the others attending the seminar) best efforts. This is because that theory is and means different things for different people. On one hand it can be said about theory that it is built on information; that theory has to be confirmed or established by observations or experiments. This could all be well and true of course, but there is another side to the coin. Several in fact. The word theory is tossed around so much, both in the world of researchers and everyday life. Everyone has used the phrase “I have a theory” more than one time. And in the scientific world some theories cannot in fact be confirmed or established by observations or experiments (like the vortex-theory discussed at the seminar). Still these “special” theories are called theories, which adds to the confusion. Suddenly the line between theory and hypothesis is becoming quite blurry. That lack of consensus and clarity is confusing, not only for me as a inexperienced student, but I suppose for the scientific community at large. And it's unclear if the situation is going to improve. The question about what theory is and isn't has been discussed back and forth ad nauseam, to no avail. It still stands unanswered. So I have taken on a different approach. Maybe we do not need a singular term for all that theory could be. Maybe it's better to have different groups of sub-theory? Would that be preferable? Maybe a whole new word for it? Or some of it? With those questions, I leave this topic, hoping that everyone has time to finish theme 3! :)