Friday, February 15, 2013

Some thoughts on design and user experience

What makes good UI design? This question has probably been asked and answered a thousand times already.

What makes good user experience?
Same as above...

As a user I have some high level requirements that are valid for mainly all user interfaces I'm exposed to. They only slightly differ for interfaces I know and use regularly and those that I use for the first time. The most important one in both cases is:

I do not want to have to bother about the user interface! I want to be able to start right off. No waste of time. What I want is intuitive usability.

For new devices or new software this mainly means that I don't need to read the documentation to perform the basic tasks it has been designed for. For things I use regularly this extends to finding more advanced options following my logic. This logic I typically built up by experience with similar tools: I expect to find the  Print option somewhere in the File menu no matter what software I use. This could be called "common agreement" or "standard" even if is not explicitly formulated anywhere. Or my logic is built by letting the task guide me. This is hard to describe. Maybe the best way is to put it this way: When it works it feels natural and effortless.

This sounds straight forward and reasonably easy. There are university courses and blogs on the topic. There is research, teaching and the relevant techniques are applied in industry on a daily basis. And yet the question remains: Why is it still going wrong so often?

I don't know for sure, but I have spent a lot of time thinking about why it goes wrong. I was never very deep into research on user interfaces. So all my thoughts here have to be viewed as just that: my thoughts - not backed up by elaborate tests, questionnaires or any other scientific method.

I come to believe that one of the problems in UI design often is a difference in how designers and users approach the topic. As I said above in my role as a user I want to get things done. But when I switch to be a designer or even engineer, for some reason I loose the focus on the task the user is interested in. In the engineers  role, I want to make all the functionality accessible and I tend to overdo it in terms of different functions put in on software or tool. I want to make things reusable, modular, sort and group things, but often I'm not guided by the task, but by technical similarities - engineering stuff you could say. In my experience sometimes the root cause for this is that the engineer or developer is simply not familiar enough with the users task to deliver good design. And sometimes the engineers just get carried away, designing a Swiss Army Knife - I call this over-engineering. In short you could say,  when it goes wrong the situation is like this:
  • User experience is top-down: You have a task -> select an appliance -> find suitable function -> use the function -> task completed!
  • Design is bottom-up: You have a bunch of functions -> you want to put as many as possible in the tool -> you need to present all of them to the user 
 No reference to the user's task in the design at all. OK, that's exaggerated ;-)
As most of the time, there is no pure black-and-white.

One very strange thing is the following observation: Users - me and you - tend to know very quickly when a tool has a bad UI design. (I'm from the Ubicomp area, so don't wonder why I use terms like "tool" or "appliance" a lot. In most cases you can replace it simply with "software". If you find this lame, read "The Design of Everyday Things" by Don Norman.) But we can also very quickly identify good design. Then, why is it so hard to design a very good interface and user experience?
  1. Design does not equal implementation. Still the design is often not addressed explicitly, but done as part of the implementation.
  2. The use is always goal driven. You want to "get something done". That's why you spent time using an appliance.  Design is often pragmatic - driven by other priorities like cost, technology constrains, time, resources in general.
  3. Sometimes systems become so complex that the goals or tasks are not obvious to the implementer but only to an experienced user. In this case you can end up with very funny results if the design is done implicitly by the implementer.....
  4. The users do not always know what is best for them. This might sound strange or even arrogant. 
This last point is very interesting as it seems to contradict partly what I said before, but in fact it does not. When you ask a user what an interface should look like, you take her out of the "task" context. She will think engineer like without the technical background knowledge of the actual engineer. You will end up with hundreds of requirements covering special cases that occur once in a lifetime. The user tends to ask for Swiss Army Knife - just in case.

So do I have a ultimate solution? No. But I can draw my personal conclusion:
  1. UI Design should be a separate development task that is explicitly addressed. (Also in terms of time and budget)
  2. The user should be integrated in the design process and contribute the expert knowledge on the task at hand. Not more!
  3. The actual design should follow the appliance idea and take into account ergonomics, cognition and efficiency.

Tuesday, June 12, 2012

New gadget: Synology DS412+

I'm finally giving up running my Ubuntu server. I used the server as file server and and media server running MediaTomb with JavaScript support. In addition I hosted my sitebar and a local wiki. The server is now replaced with a new shiny gadget from Synology: DS412+ a NAS with all the features I need and low energy consumption. I can even host my wiki and the sitebar on it. I'm running it with 4x2TB disks in a RAID configuration with a total of 6TB available storage.

This means there will probably be no updates to the 'MediaTomb with JS support on Ubuntu' topic from my side. Sorry for that ;-)

When I have play with the new gadget a little more I will probably let you know more details. For now I can already say that the media server of the DS412+ is a lot more cooperative when used with the DLNA client of my Samsung BD player than MediaTombe ever was.

Friday, December 9, 2011

HowTo: Compile MediaTomb with JavaScript support on Ubuntu 11.10 Oneiric Ocelot

This is a follow-up post to my original article on compiling MediaTomb with enabled JavaScript support for Ubuntu 10.04. Since Ubuntu 11.10 (Oneiric Ocelot) is now available, it was time for an update.

Since some time in Ubuntu MediaTomb is compiled without JavaScript support. JavaScript support is needed for example to handle/import playlists. This HowTo gives you a walk-through to compiling and installing MediaTomb with JScript support under Ubuntu 11.10. For this HowTo, I assume you start from a freshly installed Ubuntu.

You might ask why the version of MediaTomb provided with Ubuntu is compiled without JS support. To make long stories short: The problem is with SpiderMonkey, the Mozzila JavaScript engine. MediaTomb needs a header called jsapi.h to compile. As Ubuntu comes with packages providing these headers (libmozjs185-1.0, libmozjs185-dev) I gave it a try, but didn't manage to get is running this way.


But I managed to build MediaTomb on my Ubuntu Oneiric Ocelot from source with enabled JS support. Here is a step-by-step walk-trough of what worked for me:
  1. change to root or use sudo (I will assume you use sudo)
  2. sudo apt-get build-dep mediatomb
  3. sudo apt-get source mediatomb
  4. sudo vim mediatomb-0.12.1/debian/rules
  5. change --disable-libjs to --enable-libjs
  6. Now you should update the changelog file in the same directory
  7. get libmozjs2d and libmozjs-dev from the debian squeeze distibution: libmozjs2d_1.9.1.16-10_i386.deb and libmozjs-dev_1.9.1.16-10_i386.deb
  8. If you are running 64-bit Ubuntu (amd64), you should get the corresponding amd64 libraries instead: libmozjs2d_1.9.1.16-10_amd64.deb and libmozjs-dev_1.9.1.16-10_amd64.deb
  9. sudo dpkg -i libmozjs2d_1.9.1.16-10_i386
  10. If you have missing dependencies:  sudo apt-get install -f
  11. sudo dpkg -i libmozjs-dev_1.9.1.16-10_i386
  12. If you have missing dependenciessudo apt-get install -f
  13. Now you can cd to mediatomb-0.12.1 and try a sudo ./configure. You should see a line in the configuration summary that reads
    libjs                 : yes
  14. Now its time to install additional libs you might what to have for the build. Check the configuration summary. When you are done let's go!
  15. If you are not already there: cd mediatomb-0.12.1
  16. sudo dpkg-buildpackage -rfakeroot -us -uc (omit the -rfakeroot if you are working as root)
That's it! If all went well you should now have three new .deb files in the directory you did the apt-get in. You can install them all in one go by dpkg -i mediatomb*.deb. Again, if you have missing dependenciessudo apt-get install -f

Wednesday, August 17, 2011

Removing Duplicate Event Entries in Google Calendar with DupEventRemover

This is really annoying! Since I started heavily relying on my Google Calendar, I have run multiple times into the trap of events getting duplicated when synchronizing other devices and calendars with Google Calendar.
For private use I have fully switched to Google Calendar and Google Contacts (yes, I know, now they have all my personal data...). At home I use Lightning to access my calendar in Thunderbird and gContactSync for synchronizing the address book. At work I also use Thunderbird in the same configuration wherever I can. In addition I have to use Lotus Notes sometimes as this is our official corporate system and for booking meeting rooms et cetera there is no way to by-pass it. For synchronizing Notes with Google I use AweSync. And to have access to all my data on the move I also sync my Android phone with Google.
Now from time to time AweSync looses the connection to the Notes server or Google or both and stops syncing. For some reason I was unable to track down sometimes the synchronization context gets lost then. Events in Lotus are then re-synced to Google and there get duplicated.
Nothing that does not happen everyday with hundreds for different configurations. Therefore I was quite sure to find an easy solution to this problem like the "Find and merge duplicates" option in Google Contacts or some small tool solving the issue. But an Internet search showed only some commercial tools and a few links to Apple Scripts.
Technically speaking there are two options for cleaning up Google Calendar:
  1. Accessing the calendar online via Google's API (that's what the commercial tools do).
  2. Exporting the calendar in iCalendar format to an *.ics file, eliminate the duplicates in the local file and reimport the cleaned-up calendar (that's what the Apple Scripts do; at least the clean-up part).
Now I don't have an Apple and I don't plan to get one. In addition I was not willing to pay for an full iCal tool suite when I only have a quite straight forward parsing job to do to get rid of the duplicated events in my calendar.

So there is only one option left (apart from living with an increasing number of duplicate events....): DIY

I decided to go for the off-line approach: Export iCal, work on the file locally and reimport the result. First I went looking for an easy to use API to handle the iCal format from Java: I found iCal4j by Ben Fortuna. Thanks to Ben for that fine piece of work! Next I did some coding.... and here we go, my calendar is clean and tidy again!

Now this post could end here, letting you know my problem is solved, but not going further into making a solution available. As this would be crap and not really very nerdy, I decided to provide my source code here. I do not have time at the moment to develop this into a pre-compiled easy-to-use software with a fancy front-end. Nevertheless I like to share what I have done at this early stage as there seem to be no other tools out there.

The easiest way to get the code working is to import it to Eclipse and add iCal4j to the build path.

 // ===========================  
 // DupEventRemover - License  
 // ===========================  
 //  
 // Copyright (c) 2011, Tobias Zimmer  
 // All rights reserved.  
 //  
 // Redistribution and use in source and binary forms, with or without  
 // modification, are permitted provided that the following conditions are met:  
 //  * Redistributions of source code must retain the above copyright  
 //   notice, this list of conditions and the following disclaimer.  
 //  * Redistributions in binary form must reproduce the above copyright  
 //   notice, this list of conditions and the following disclaimer in the  
 //   documentation and/or other materials provided with the distribution.  
 //  * Neither the name of Tobias Zimmer nor the  
 //   names of any other contributors may be used to endorse or promote products  
 //   derived from this software without specific prior written permission.  
 //  
 // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND  
 // ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED  
 // WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE  
 // DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDERS AND CONTRIBUTORS BE LIABLE FOR   
 // ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES  
 // (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;  
 // LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND  
 // ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT  
 // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS  
 // SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.  
   
   
 package de.cwtz.dupeventremover;  
   
 import java.io.FileInputStream;  
 import java.io.FileNotFoundException;  
 import java.io.FileOutputStream;  
 import java.io.IOException;  
 import java.util.Iterator;  
   
 import net.fortuna.ical4j.data.CalendarBuilder;  
 import net.fortuna.ical4j.data.CalendarOutputter;  
 import net.fortuna.ical4j.data.ParserException;  
 import net.fortuna.ical4j.model.Calendar;  
 import net.fortuna.ical4j.model.Component;  
 import net.fortuna.ical4j.model.ComponentList;  
 import net.fortuna.ical4j.model.ValidationException;  
 import net.fortuna.ical4j.model.component.VEvent;  
 import net.fortuna.ical4j.util.CompatibilityHints;  
   
 public class DupEventRemover {  
   
      /**  
       * @param args  
       */  
   
      public static void main(String[] args) {  
           // TODO Auto-generated method stub  
   
           CompatibilityHints.setHintEnabled(  
                     CompatibilityHints.KEY_RELAXED_VALIDATION, true);  
   
           // Reading the file and creating the calendar  
           CalendarBuilder builder = new CalendarBuilder();  
           Calendar cal = null;  
           Calendar[] calOut = null;  
           String inputFile = null;  
           String outputFile = null;  
           String extensionFile = ".ics";  
             
           // Max number of events in one calendar file  
           int googleLimit = 2500;  
             
           //System.out.println(args.length);  
   
           if (args.length > 0) {  
                if (args[0].equals("--help")) {  
                     System.out.println("" +  
                               "Reads an ICal calendar file, removes duplicate event entries and writes the result to one or more new files.\n" +  
                               "\n" +  
                               "DupEventRemover [source | --help] [destination]\n" +  
                               "\n" +  
                               " source \t Specifies the file to read calendar data from. Make sure you give the full name including the file extension.\n" +  
                               " destination \t Specifies the file(s) to write the new calendar to. Omit the file extension. It will be added automatically\n" +  
                               " --help \t Displays this help.\n" +  
                               "\n" +  
                               "If you don't specify 'source' and 'destination', DupEventRemover will by default look for a file\n" +  
                               "named 'my.ics' in it's current directory and write the new calendar to 'my_total_new_x.ics', where x is the number of the file.\n" +  
                               "If you don't specify destination, but only source, the default destination will be used.");  
                     System.exit(0);  
                } else {  
                     inputFile = args[0];  
                }  
           } else {  
                inputFile = "my.ics";  
           }  
   
           if (args.length > 1) {  
                outputFile = args[1];  
           } else {  
                outputFile = "my_total_new_";  
           }  
   
           System.out.println("Reading calendar file...");  
   
           try {  
                cal = builder.build(new FileInputStream(inputFile));  
           } catch (IOException e) {  
                System.out.println(e.getMessage());  
                System.out.println("Try typing 'DupEventRemover --help' for help.");  
                // e.printStackTrace();  
                System.exit(1);  
           } catch (ParserException e) {  
                System.out.println(e.getMessage());  
                // e.printStackTrace();  
                System.exit(1);  
           }  
   
           System.out.println("Start processing. Please wait...");  
   
           int nProcessed = 0;  
           int nDeleted = 0;  
   
           // For each VEVENT in the ICS  
           for (Object o : cal.getComponents("VEVENT")) {  
                Component c = (Component) o;  
                VEvent e = (VEvent) c;  
   
                for (Iterator i = cal.getComponents(Component.VEVENT).iterator(); i  
                          .hasNext();) {  
                     VEvent event = (VEvent) i.next();  
   
                     if ((event.getSummary() != null) && (e.getSummary() != null)  
                               && (event.getStartDate() != null)  
                               && (e.getStartDate() != null)  
                               && (event.getEndDate() != null)  
                               && (e.getEndDate() != null)) {  
                          if ((event.getUid() != e.getUid())  
                                    && (event.getSummary().getValue().equals(e  
                                              .getSummary().getValue()))  
                                    && (event.getStartDate().getValue().equals(e  
                                              .getStartDate().getValue()))  
                                    && (event.getEndDate().getValue().equals(e  
                                              .getEndDate().getValue()))) {  
                               nDeleted++;  
                               cal.getComponents().remove(c);  
                               break;  
                          }  
                     } else {  
                          if (e.getSummary() == null) {  
                               nDeleted++;  
                               cal.getComponents().remove(c);  
                               break;  
                          } else {  
                               // debug:  
                               // System.out.println((VEvent) event);  
                          }  
                     }  
                }  
   
                nProcessed++;  
   
                if ((nProcessed % 100) == 0)  
                     System.out.print(".");  
           }  
           System.out.println("");  
           System.out.println("Number of records processed: " + nProcessed);  
           System.out.println("Number of records deleted: " + nDeleted);  
             
           calOut = new Calendar[(int)Math.floor((nProcessed - nDeleted)/googleLimit)+1];  
           for (int i = 0; i < calOut.length; i++){  
                calOut[i] = new Calendar(cal.getProperties(),new ComponentList());  
           }  
             
           int limitCounter = 0;  
           int calendarCounter = 0;  
           for (Iterator i = cal.getComponents(Component.VEVENT).iterator(); i.hasNext();) {  
                 VEvent event = (VEvent) i.next();  
                   
                 calOut[calendarCounter].getComponents().add(event);  
                 limitCounter++;  
                   
                 if (limitCounter >= googleLimit){  
                      limitCounter = 0;  
                      calendarCounter++;  
                 }        
           }  
             
           // write new calendar file(s)  
           for (int i = 0; i < calOut.length; i++){  
                FileOutputStream fout = null;  
                try {  
                     String outFile = outputFile + i + extensionFile;  
                     fout = new FileOutputStream(outFile);  
                } catch (FileNotFoundException e) {  
                     // TODO Auto-generated catch block  
                     e.printStackTrace();  
                }  
        
                CalendarOutputter outputter = new CalendarOutputter();  
                try {  
                     outputter.output(calOut[i], fout);  
                } catch (IOException e) {  
                     // TODO Auto-generated catch block  
                     e.printStackTrace();  
                } catch (ValidationException e) {  
                     // TODO Auto-generated catch block  
                     e.printStackTrace();  
                }  
           }  
      }  
 }  
   

You see it's extremely rudimentary and even hack-style. What it does is, it goes through an input iCal file finds all duplicates by searching for same date and time and description with different IDs and deletes the found events. It also deletes events with an empty description. Then it writes the newly created calendar to a series of files each containing a fixed maximum number of events (at the moment 2500).  

This limit I have introduced as there is a daily quota to the number of events you can create by uploading calendar files to your Google Calendar. The actual quota is not documented to my knowledge. It said on the Internet it would be 5000, but testing this I ran into difficulties sometimes already with 4500. 2500 seems to be well on the save side, but might be inefficient to upload bigger calendars (mine is about 10k events, so it took me 4 days to upload). 

Some words of caution: You mess around with your calendar at your own risk. Make backup copies and store them in save places. As the license of the code indicates this is provided 'as is', I'm not taking any liability neither for the fitness of the code nor for the correctness of any other information in this post.

Monday, July 18, 2011

HowTo: Compile MediaTomb with JavaScript support on Ubuntu 11.04

--- UPDATE ---
Since Ubuntu 11.10 (Oneiric Ocelot) is now available, it was time for an update:
HowTo: Compile MediaTomb with JavaScript support on Ubuntu 11.10 Oneiric Ocelot
--- UPDATE END ---

Since some time in Ubuntu MediaTomb is compiled without JavaScript support. JavaScript support is needed for example to handle/import playlists. So I spent some time on the net looking for a howto. The best I found was this one targeted to Ubuntu 10.04.

To make long stories short: The problem is with SpiderMonkey, the Mozzila JavaScrip engine. MediaTomb needs a header called jsapi.h to compile. As Natty comes with packages providing these headers (libmozjs185-1.0, libmozjs185-dev, xulrunner-2.0-mozjs) I gave it a try, but I didn't manage. I also followed Gabriel's hints on how to build and install SpiderMonkey from source, but did still not succeed. 


After a little more of research I finally managed to build MediaTomb on my Ubuntu Natty. Here is a step-by-step walk trough of what worked for me:
  1. change to root or use sudo
  2. apt-get build-dep mediatomb
  3. apt-get source mediatomb
  4. vim mediatomb-0.12.1/debian/rules
  5. change --disable-libjs to --enable-libjs
  6. Now you should update the changelog file in the same directory
  7. get libmozjs2d and libmozjs-dev from the debian sid distibution: libmozjs2d_1.9.1.16-6_i386.deb and libmozjs-dev_1.9.1.16-6_i386.deb
  8. Now you can cd to mediatomb-0.12.1 and try a ./configure. You should see a line in the configuration summary that reads
    libjs                 : yes
  9. Now its time to install additional libs you might what to have for the build. Check the configuration summary. When you are done let's go!
  10. If you are not already there: cd mediatomb-0.12.1
  11. dpkg-buildpackage -rfakeroot -us -uc
That's it! If all went well you should now have three new .deb files in the directory you did the apt-get in. You can install them all in one go by dpkg -i mediatomb*.deb

My thanks go to Gabriel Burca for his solution for Ubuntu 10.04 that was the basis for my Natty solution.

--- UPDATE ---
Triggered by Viv's comment, I tried this also on a 64-bit Ubuntu 11.04 (amd64). 
The only difference is in step 7. Here you will have to use the amd64 libraries: libmozjs2d_1.9.1.16-6_amd64.deb and libmozjs-dev_1.9.1.16-6_amd64.deb
After  that there is one additional step:
7.1. apt-get install libjs-prototype
--- UPDATE END ---

First entry!

Ok, I finally decided to start a blog.  I was looking quite some time for a way to take notes that might also be interesting for others, and have them accessible on the go. So, thanks to my friend Albrecht Schmidt I will try with a blog to keep my public notes and thoughts on computing, computer science and related topics.