Thursday, September 18, 2008

Dependencies

You might have heard me complaining about that issue already, but it is still an issue.
Every once in a while I come around checking the dependencies of a KDE package and then I find out that they are 1) either available, 2) available but not buildable (a lot of foss software on windows is hacked together, a.k.a. patched heavily) or 3) not even available. Availability means available for both of our compiler flavors (we currently use mingw 3.4.5 and Microsoft Visual Studio 2003/2005/2008). If the library builds and if it is not to much messed up we normally fix this case by finding a CMake patch for it. The latest library that took this way is exiv2 and I am glad to say that this patch will go upstreams for version 0.18. This is the good side.


Another dependency I have come across was mysql. It is used in several places throughout KDE, on of its use cases is as an embedded database for amarok. As a nice packager I made myself on the way to find this stuff. If you look at the MySQL page and you search for embedded, you will find "Thinking about using MySQL as an Embedded Database? Contact us online." with a link to the sales department. Ok, no money for that over here, so maybe they provide it somewhere else. Searching through the download section I found the rpms for red hat for the embedded db but the windows packages don't contain anything resembling the embedded library. And after searching a bit more I found the confirmation: Mysql doesn't provide the embedded library.
Hm, since it would benefit multiple packages, I thought, why not try to compile it myself. Ok, I downloaded the package and on the Build Instructions page they spoke about cmake which made me happy at that point. Even though I need to build the stuff myself, I just need to run
cmake && nmake && nmake install
and then I have my library. But it should get even more painful:
Before running cmake you need to run a javascript configure file.
Then you get a batch script which copies the CMakeCache.txt and then runs cmake for you - nothing more.
Then you have a Visual Studio project file.
Going through all available targets I couldn't find libmysqld.
There was no libmysqld. Ok, searching through CMakeLists.txt files gave me - nothing.
Their CMake build system doesn't contain that library. Only the autotools stuff as I found out - so autotools for Linux, a crippled CMake for Windows.


Is it really so hard to find somebody which knows enough cmake (or can read enough to understand the cmake help) to get a decent CMake buildsystem for mysql? Or is this simply a way to sell even if mysql is GPL?
I am pretty sure I will fix this in the future, but this is the point where I definitely will not contribute to GPL licensed code without a paycheck.


What this means:
- Gwenview will be hopefully enabled in the next windows packages for trunk and 4.1
- Amarok's mysql collection plugin will have to wait.
- All other users of mysql embedded will simply have to use the client with a server.

7 comments:

Anonymous said...

Hi all,

I understand from this post that not all the uses of MySQL in KDE are embedded, right?

May I go out on a limb and posit that adding a dependency to a whole darned database is a spectacularly bad idea, and should be made optional regardless of the platform? I'm well aware of the performance issues of, say, SQLite when there are many concurrent requests, but is it that much of an issue for a desktop?

This is a honest question, please don't take it wrong! Maybe I vastly underestimate the complexity and concurrent numbers of the requests that are needed by whatever bits of KDE perform them? Do we have any data on that?

Seriously, it was already hard to keep arguing rationally against the 'KDE is bloated' trolls; I shiver at the thought of the moment when they figure out some bits of KDE depend on a database server. Hopefully I'm not overreacting there.

At any rate, thank you for the good work! It's awesome that KDE is going to work natively on Windows, and that wouldn't happen without your work.

Anonymous said...

Are there any technical reasons why KDE developers tend to avoid the QtSql module? From an outside pov it looks like unnecessary duplication of effort.

Ian Monroe said...

I actually asked the internals list if there was any efforts to use mysql with cmake on Linux. This sparked a large thread, where it was revealed that there was actually a mysql guy who had ported to cmake on Linux and had dropped the JS crap on Windows (you have to remember that the cmake port is quite recent, no reason to call them cmake morons).

I've looked at the cmake files to understand how the embedded server is built (since I can't read automake very well). So there is clearly intention that the embedded server work on Windows, I guess you didn't look hard enough for it? I mean, its right where you'd expect it to be, in the CMakeLists.txt file of libmysqld.

@Anonymous: The QtSql module provides very little abstraction. About the only thing it abstracts it the initial connecting to the database, which is the easiest part anyways. So its handy if you want to use some of the GUI widgets which take advantage of it and pretty pointless otherwise.

@s. its not that big. 7 megs + some support files. Well technically it is a "whole darned database", I think you overstate the awesome power of a database. They are pretty cool no doubt, but seriously its not that big of a deal.

@taurnil thats for configure, not cmake silly.

Anonymous said...

@Ian:

1) The problem is the memory consumed, not the disk storage, c'mon. Database engines require lots of memory to perform adequately on non-trivial datasets, all the more so if there is a lot of concurrency.

2) This doesn't answer the question of just how much concurrency KDE requires in its requests, that a file-based solution such as SQLite would provably not cut it.

3) This doesn't, either, answer the matter of making the dependency optional. Please kindly leave it to me to decide what constitutes a 'big deal' based on how much memory I have. Thanks.

I hope I'm not coming across as too aggressive here. My mind is just blown that we are having this discussion at all.

Anonymous said...

@Ian: I still don't get why basing something directly on MySQL libs is more abstract than using QtSql.

SaroEngels said...

@taurnil: just with configure (not on windows).
@s: of course it is optional; but the problem arises if a user comes, looks and wants his one feature supported. So basically in the end we will have to support every feature - just "because it works on other platforms too".
@ian:
Which version are we talking about? I looked into mysql 5.0.67 and there is no CMakeLists.txt in libmysld directory there. But I will try some newer version too now, maybe I have more luck there.
Nevertheless the quality of those cmake files I read is not equal to the one in KDE - to be polite.
One last point stays: If I want to have libmysqld in my project, I need to build mysql myself. This means extra work for something that wouldn't be to hard for them.

Anonymous said...

I knew the Metin2 gold so I always try my best to earn them more and more to make myself strong. I have never played the game before, at the beginning I did not know what is so I went to kill the monsters with the Metin2 yang that I earned with myself in the game. I will duty bound to a friend to help brush the Cheap metin2 yang together with my friends. I spend a good relationship is then fly to tears. If my levels are very high, I can go to Buy metin2 gold more and more and I will not depend on my friends to help me to earn them. I get some Cheap metin2 gold as the gifts to encourage me.