Thursday, 21 June 2012

New KDE Copy Dialog: First Preview

I'd like to shed some light on my GSoC project. I'm working (with a help of Björn Balazs) on new cool user friendly dialog for copying/moving files. This is a continuation of my previous project to make errors handling asynchronous from files copying process. You can read about it in my previous posts: first, second.

To make a long story short, now if there is a problem with file(s) you are trying to copy, it will no longer block the copying process. Problematic file is classified and just goes to the coresponding tab, where user can find and resolve the problem.

Take a look (I suggest turning 720p and fullscreen) at first preview:

See it on youtube.
Or download via dropbox: file_transfer_dialog_v3.webm

Keep in mind that it is not yet ready of course ;)

This is not a replacement for widgets in notification area:




Normally, the dialog will be opened only if user interaction is required. But for non-KDE systems it will replace the old dialog. For instance this is how the old dialog looks in Mac OS X:

Old KDE Copy Dialog on Mac OS X

We are trying to keep new dialog as simple as possibe. So don't worry: if there are no errors the dialog will be very compact :)

Tuesday, 20 December 2011

Golang: goroutines performance

Intro


In this post I'll try to measure goroutines performance. Goroutines are something like lightweight threads. It is a built-in Go's primitive providing multitasking (together with channels).

Documentation tells us:

It is practical to create hundreds of thousands of goroutines in the same address space.

So the point of this post is to check it and to figure out how the performance suffers from using such a big number of concurrently running functions.

Memory


The size of a newly minted goroutine is not documented. It is said to be about few kilobytes. Tests on different machines help to clarify this number to 4 — 4,5Kb. So 5Gb is more than enough to run 1 million goroutines.

Performance


Let us figure out how much speed we lose when running function in a goroutine. As you probably know it is very easy — just add go keyword before the function call:

go testFunc()

Goroutines are multiplexed on threads. By default, if there is no GOMAXPROCS environment variable, program uses only one thread. To take advantage of all CPU cores you need to specify their number. For example:

export GOMAXPROCS=2

This value is used in the runtime. So there is no need to recompile program every time you change this value.

As far as I can presume, the time is spent mostly on goroutines creation, switching among them, and also sometimes on moving goroutines into others threads and on communications of goroutines from different threads. To avoid last cases, let us start from using only one thread.

All actions were performed on my nettop:

  • Atom D525 Dual Core 1.8 GHz
  • 4Gb DDR3
  • Go r60.3
  • Arch Linux x86_64

Methodology


Here is a test functions generator:

func genTest (n int) func (res chan <- interface {}) {
        return func(res chan <- interface {}) {
                for i := 0; i < n; i++ {
                        math.Sqrt(13)
                }
                res <- true
        }
}

And this is how we get a set of functions calculating sqrt(13) 1, 10, 100, 1000 and 5000 times respectively:

testFuncs := [] func (chan <- interface {}) { genTest(1), genTest(10), genTest(100), genTest(1000), genTest(5000) }

I'm running each function X times in loop and then in X goroutines. Then results are compared. Sure, garbage collection should be kept in mind. To reduce its influence I explicitly call runtime.GC() after all goroutines finish and only then note the finish time. Of course, for the accuracy each test is performed many times. Total run time took about 16 hours.

One thread


export GOMAXPROCS=1

gorounes performance 1_1

The graph shows that the function whose running time is approximately equal to the computation of the sqrt() works in goroutine about 4 times slower.

Let us consider 4 remaining functions more in detail:

gorounes performance 1_2

You can see, that even 700 thousand simultaneously running goroutines don't reduce performance more than by 80%. And now the most awesome. Starting from sqrt()x1000, the overhead is less than ~2%. 5000 times — only 1%. And it looks like this number does not depend on the number of goroutines! So the single limitation is memory.

Resume:

If the run time of independent code is more than calculating square root 10 times, and you want to run it concurrently, do not hesitate to run it in goroutine. Although, if you can easily collect 10 or even 100 such code parts together, the loss of performance will be only 20% or 2% respectively.

Several threads


Now let us consider situation when we want to use several processor cores. In my case there are 2 ones:

export GOMAXPROCS=2

Running our testing program again:

gorounes performance 2_1

Here you can see that despite the fact that the number of cores has doubled, the run time of the first two functions increased! It is probably because moving them to another thread is more expensive than just executing :) Currently scheduler can not resolve such situations, but Go authors assure it will be done in the future.

gorounes performance 2_2

As you can see last two functions use both cores to the fullest. On my nettop their execution time is ~45µs and ~230µs respectively.

Conclusion


Evan despite the language youth and temporary scheduler implementation, goroutines performance feels amazing. Especially when combined with Go's simplicity. I'm very impressed. Many thanks to Go Dev team!

I can also propose to think twice before running goroutines whose lifetime is less than 1µs, and not to hesitate if its lifetime is more than 1ms :)

Tuesday, 23 August 2011

GSoC Results: Interaction Dialog

GSoC 2011 is coming to end. And I would like to tell you about what is done by this time. I was working on advanced error handling during file transfer. Now I'm introducing new non-blocking interaction dialog. It will be used to handle interaction requests.


As you can see this screen is very different from what I was describing in my previous post. This is true. I really had a lot to alter. Now the part interacting with user is taken outside the JobView (which is in the tray). Interaction dialog appears as soon as first conflict occurs. You no longer need to worry that the errors will go unnoticed.

Interaction requests may be different types, so they are collected up to different tabs. At the moment there are two interaction types: "File already exist" and "File could not be read". Each of these types provides it's own way of conflict resolution. Also you can select multiple files and apply common action (such as retry or skip) to them.

The way of resolving conflicts with already existing files is displayed on the screen in the beginning of this post. And this is what users will see if they try to copy file without appropriate permission:

In the future there will be added the widget that lets you change the permissions directly from this dialog. But now you need to do it manually and than just press Retry button.

Also all interaction requests can be found in one place:


Working with KDE community was a great pleasure. I'd like to thank my mentor David Faure, guys from kde-usability mailing who have helped me to clarify the look of dialog, and everyone who commented my previous post.

Few words about the plans for the future. I'm planning to continue working on this dialog: adding handling folders-specific errors, permissions widget, maybe something else... I would be very grateful to your suggestions!

Tuesday, 19 July 2011

GSoC: Errors Handling During File Transfer

This summer I'm taking part in Google Summer of Code 2011 again. And again I'm happy to work with the KDE community. Recently I've finished the first part of my project: Improvements in KIO File Transfer. Now it's time to report about my results, show some screencasts and also ask for some advices related to UX. :)

I was working on intelligent errors handling during the file transfer process. The essence is that sometimes users need to transfer large amounts of data, but sure they don't wont to monitor the transfer all the time.

Previously in case of any error transfer was blocked and requested user interaction. This is not a right approach to the matter, because it wastes user's time and makes him/her nervous. I divided these errors into two parts: one of them really need user actions while others just informs about some files that can not be transferred.

Now if interaction is required (for example file already exists and we need to choose among skip, rename, overwrite), information about it goes to the special widget (don't be afraid, it's ugly now, but I hope some of usability guys will help me to make it shiny :) placed in JobView. And copying don't stops! When the user have free time, he can complete the interaction. Though just take a look at this video:



The conflicts discussed later were resolvable, but there are some cases when we just can't get access to a file. So there is no sense in distracting user. Just informing about problems during the transfer process would be sufficient. After transfer finished, a notification is created. It contains information about happened errors. In current implementation it is "Errors" button, which opens in text editor file with a list of problematic files. This would be useful if you want to apply any console command such as chmod to them quickly. Anyway, the better to see once:



And now about things that should be improved. And I hope someone help me.
  1. Icons. view-conversation-balloon edit-clear-list I used view-conversation-balloon for displaying files requiring interaction and edit-clear-list for displaying errors. Sure they need to be replaced by somethings more consistent.
  2. Lists of errors and interaction requests should look different. Now they seems inconsistent. :)

P.S. I am very grateful to all users of KDE, who sent me a letters with the wishes of good luck. I'll try not to disappoint you:)

Monday, 23 August 2010

GSoC: My Results

Today final results of GSoC 2010 would be announced. I just want to say, it was awesome! I successfully finished my project and it is a time to describe the progress. As may be you remember I'm working on mind mapping in KOffice.

For first, already done things:
  • More interactive movement. Now when you move shape by TreeTool it searches for the closest tree and proposes place where it would be situated.
  • Text on shapes. All tree roots are text containers now.
  • Saving/Loading and Copy/Paste. TreeShape can be saved to ODF. But since I used unsupported tag, other office suites will not recognize trees.
  • New tree structres. I added two new structures corresponding to XMind's Tree (left) and Tree (right)
  • Widget. When you use TreeTool it is possible to change tree structure, connectors type and root shape with special widget.

Unfortunally right now plugin is not ready for end users. Trees looks ugly and should be polished. Also there is a an unresolved conflict with Flake memory model which sometimes results in crashes.

But scince I stay a KOffice developer I'll continue working on this project. So this is my plan:
  • to resolve conflict with memory
  • to change saving format to make trees visible in Open Office
  • to make trees more shiny :)
  • also the code need refactoring

If someone would like to test plugin, this may be done by installing KOffice from svn.

Thursday, 15 July 2010

GSoC 2010: Mind Mapping in KOffice

Hi guys. I'm GSoC student Cyril Oblikov. And this is the first post about my project - Mind Mapping in KOffice. Mid-term evaluations deadline coming soon, and I'd like to describe what is done for this moment.

A key element of mind map is tree which layouts shapes with text. I decided tree may be usefull not only for mind mapping, but for some other things. So I started working on TreeShape plugin. Plugin makes it possible to layout any KOffice shapes in tree structure and manage it.
As tree can work with any KOffice shape, I prefer using Ellipses or Rectangles (this simple shapes are well suited for debugging).


Let us see, what tree plugin can do right now:
  • There are four types of layouting shapes: OrgUp, OrgDown, OrgLeft and OrgRight. Also layouts can be combined.
  • Keys 1,2,3,4 change type of tree's layout.
  • You can add new children (by pressing <Tab>) and neighbours (by pressing <Enter>).
  • Trees can be moved from one parent to another. Just select tree and release mouse over the new parent. As you can see this way is not very comfortable, so currently I'm working on more interactive movement.
  • It is possible to undo/redo children moving, adding and deleting.

    And this is in turn in to-do list:
    • Some new layout types similar to XMind ones.
    • Saving/Loading trees.
    • Indicators - icons attached to shapes which display such usefull information as readyness, priority, etc.
    • Comfortable docker widget to control trees.