Memory Management UI – Trial #2.

I’m going on my third north Indian trip tomorrow, spiritual purposes only. So I’m off the net for next 2 days. I happened to hurriedly do this:
http://lists.kde.org/?l=kstars-devel&r=1&b=200807&w=2

I’d want comments on that UI, because I have no idea about UI design.

2.5 million stars in trunk!

Yay! All the machinery for loading 1e6 stars has been moved to trunk and the 2.5-million-star Tycho-2 catalog has moved to the ‘Download Data’ (Get Hot New Stuff) feature. I plan to put up a screencast on how to use this and the conjunction tool from KStars sometime, along with a whole lot of other features.

To get the Tycho-2 deep catalog, just click on File->Download Data and click ‘Install’ next to ‘Tycho-2 Star Catalog’.

KStars loads 41560 stars in 225ms!!!

“We all know Linux is great…it does infinite loops in 5 seconds.”
–Linus Torvalds

KStars is now no longer far behind, thanks to James Bowlin. James seems to know every bit of computer architecture. One interesting thing that I learnt from him, after he repeatedly tried to put it into my head, is that malloc()s are like hard disk seeks. You had better do a BIG malloc() at once than make several small malloc()s. He knew for sure, that much of the time KStars spent in loading stars in the StarComponent::readData() method went into the constructors!

So he suggested something that I initially thought was un-implementable: Forget C++ constructors. Let’s do our own memory management. Create a template StarObject with some default data initialized in it. Whenever you want to create a new StarObject, just memcpy() it to make a new one!

Now, the first question I asked is what about those memory allocations that the constructor does? For instance, StarObject::StarObject() creates a new QString to store the spectral type. James said we’d take care of that by allocing them in a StarObject::init() function that sets up our freshly copied StarObject. I still had a lot of doubts in mind.

I decided to jump in and see what happens. Went and made some changes in StarObject – implemented a StarObject::init() that did the job of a constructor, and after a few segfaults, changed StarObject::SpType from QString, to QString *, so that I could alloc it freely at my will. Now, this could create some bugs and memory leaks in KStars if not used properly, but every programmer will have the sense to look at my “WARNING:” comments. I couldn’t believe that it actually worked! And I was really amazed to see the whooping 6x increase in speed, with the readData() method taking 6 microseconds per star instead of the earlier 42!!!

I had some funny idea of how C++ might store the class, but it turned out to be different. I still don’t know how it does that – it looks fuzzier after I implemented this.

I’m pretty sure that things will break (and segfault) when KStars learns to remember the user-added information links and observation log in the Detail dialog. We will probably fix this in the trunk before the KDE4.1 release, but that means that I’ll have to make some more changes before it safely merges into the branch.

I’m now trying another suggestion that James made, about loading all star name data into memory, to avoid the disk head seeking between two different files. Now that the fread() that grabs the data from the binary files takes a significant fraction of the load time, this might be an important factor. Earlier, it would’ve simply gotten buffered by the time StarObject::StarObject() would finish. This requires me to add a header to the star name file describing the number of records, so that’s going to change everything right from the data file generating program. I can’t wait to see how much of an efficiency boost I’ll get. It might not be much, though.

First steps to acheiving faster loading of stars in KStars

So far, this is what I’ve done:
1. Written a PHP script to load star data from our flat file into a MySQL database. Tested it on our current 125980 star database.
2. Understand some of the fundamental ideas involved in this project.

What needs to be done:
1. Put HTM trixel data also into the MySQL database
2. Read all of James’ emails describing the whole roadmap to success.

PS: I wrote this post only for my satisfaction, to give me the feeling of having done something towards GSoC.