Linux

I was looking for ways to enhance database performance on the quite busy database server of the very busy forum site. Currently I have to maintain forum that have to handle more than 1000 people online concurrently and have average posts about 5k posts per day. Therefore, it ever reached around 5M posts last year before my clients decided to split to new forums and left old forums read-only. And now ... It already reach 700k of posts for only half year with 1.5GB of database size (So ... WHAT A HECK THEY POSTING ?).

When database size increase dramatically ... And if it becoming large and nearly reach amount of RAM that your server has. Your database will face serious performance problems because your memory now not enough to hold all disk cache of database files on disk.

It happened last year when database size of this forums reach around 10GB when the server has only 8GB RAM. Even this database is on very fast 4 disks, 15k RPM RAID 10, Battery Backed LSI MegaRAID SAS HBA. I/O Utilization usually reach 100% at the busy time. And my web server start to respond Error 500 because of database overload.

And now SSD is out and may be a choice of what I want ... High mixed, random, read/write IOPS. But one thing to consider, Endurance, how much data can writes to SSD.

As you may know, SSD have limited time of writes compared traditional magnetic harddisk. If you decided to put SSD inside server ... You need to measure how much data gonna write on it to ensure your SSD will live long enough.

From X25-M SSD datasheet:

3.5.4 Minimum Useful Life
A typical client usage of 20 GB writes per day is assumed. Should the host system
attempt to exceed 20 GB writes per day by a large margin for an extended period, the
drive will enable the endurance management feature to adjust write performance. By
efficiently managing performance, this feature enables the device to have, at a
minimum, a five year useful life. Under normal operation conditions, the drive will not
invoke this feature.

You may notice 2 things here. First X25-M SSD endurance is 5 years for average writes of 20GB per day. Second, if you put data more than 20GB per day, it will activate endurance management mechanism that results in lower performance.

Now it time to measure disk writes on my server. It can done by basic command in Linux ... 'iostat' (If your server not have iostat ... simply install sysstat package).

popcorn2 64h # uptime
 03:31:16 up 23 days,  6:47,  2 users,  load average: 0.35, 0.15, 0.31
popcorn2 64h # iostat -m /dev/sdb
Linux 2.6.27-gentoo-r5 (popcorn2)       03/30/09

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           9.30    0.00    1.66    0.49    0.00   88.55

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
sdb              15.81         0.02         0.07      37865     144223

With basic calculation ... This result around 6.26GB per day.

So I can use X25-M SSD on this system for database, or even faster version X25-E.

If you decided to use SSD for your database server ... make sure you check how much data gonna write to your SSD before putting some important data on it. And do not forgot to BACKUP. SSD is quite more expensive than 15K RPM disks of the same capacity with much lower write endurance. ($400 80G SSD vs $250 74G 15K RPM)

Posted by SF-Alpha

I follow state of iSCSI and FCoE on linux since a few years ago. But ... It seems slow and Wrong Way !

Altough kernel SCSI maintainer like the move everyting to userspace idea. But it not good for performance. I know that SCST already exists and mature but not going to mainline kernel. Why ? because kernel maintainer doesn't like this way and more likely happy to have STGT instead (which use much userspace things).

I tried both but STGT performance is simply BAD!. Like iSCSI, moving to more complicated layers to handle such storage low-lattency things is BAD!. So, by the same reason, iSCSI not so popular on performance storage because of too much overhead added to processing and it increasing lattency, although throughput seems ok (1GBPS with only 2-3% cpu load even on software initiator) but SCST proves to have lower lattency very close to pure HW implemantation. but STGT is more likely to have delay on userspace processing.

Something wrong ? Although STGT is more good, but SCST is mature enough to get it to mainline kernel for now. It should be, at least, for everyone to test on the mainlike kernel both SCST and STGT to see what is good for real life usage.

Posted by SF-Alpha

If your database disks have read-ahead enable, It good to turn off especially database that require very very high random-read on busy servers.

Posted by SF-Alpha

Recently filesystem activity besides ext4 in Linux kernel is Tux3. Which, in technicial, is really B-Tree extents based filesystem like ext4 or reiserfs. Despite of it not too complete and not stable now, Tux3 is journal fileystem with log and atomic commit. New atomic commit approch is to write metadata update to logical log (memory) and then data blocks to physical storage follow by commit block. Then update metadata (from logical to physical) afterwards, unlike NTFS and other journaled fileystem. Transaction on Tux3 are considered completed after log is written and will not replay any metadata but will load log in place every mount to check filesystem consistency. That simplify maintain data consistency when disk is hot remove and make sure log recovery code is well tested. Considered situation when atomic commit is not complete (data blocks not contains commit block), after mounted, log not need to replay because corrupted data is written somewhere not the same block as previously reside in.

Seems SSD and flash based disks really benefits from Tux3 implementation which not require much performance of contigous blocks or fragments data but reduce write levering that will turn disk in to short life.

Posted by SF-Alpha

Gentoo recently drops old release schedule and focus on new Weekly or Monthly automatic release build.

And more, There is (may be) no more Gentoo Catalyst for release build for the Gentoo, and that means, Catalyst project is gone.

Because of new release model and attempt of Daniel Robbins (former Gentoo leader and founder), he decides to bring Gentoo and his own Gentoo Portage Overlay project called Funtoo to the next steps. By providing something that let people build distribution and customization based on Gentoo and Funtoo more easier and have larger user based. Daniel have create new automatic build mechanism called Metro (Wtih the release 1.1 currently released).

Gentoo developer now working closely with Dainel to get this thing working for gentoo release as well, but not much activity on this. Gentoo councils now seems to focus on based system (such as EAPI, Baselayout) than the release engineering or building things.

Let's try Funtoo. Just For Fun.

P.S. You should to watch videos on Daniel's blog. Much fun !

Posted by SF-Alpha

Article about techniques to reduce writes on flash based disks.

http://www.cyrius.com/debian/nslu2/linux-on-flash.html

I found this article useful for me (EeePC901 Flash model) and other who run Linux on flash.

And again, portage seems to do much about filesystem read/writes ? I will figure out how to reduce it !!!

Posted by SF-Alpha

I found this bug when trying building stage4 image for my EeePC from my AMD64 PC.

Seems like genkenrel not pass valid ARCH on make menuconfig (this bug #247246)

I reported this for a while and nobody seems to work on it. What those bug wranglers doing guys.

Although I can get fixed myself. I think this is major bugs that could by fixes soon.

BTW: I got Gnome+XFCE stage4 for my EeePC (with compiz-fusion optionally enabled).

Posted by SF-Alpha

So I grab some patch from Gentoo bugzilla and it works. But ... It really does have some problem because (some bad) design of Portage and also Gentoo Catalyst itself. I found that catalyst do good on cross compile and make anything consistent. But portage cannot solve circular dependencies on building stage4, and more, it does not update use flags on stage3 packages (@system) for stage4

This is another bug that I go to fix it myself by adding only --deep --newuse to the emerge command in build stage4 target script.

Continue reading "Gentoo Catalyst is way too complex (and buggy)"

Posted by SF-Alpha

Again for my EeePC 901. eeepc-acpi ebuild. In fact, there is nothing, just a skeleton for linux_mod eclass.

But hey.... 2.6.27 does not need it.

Posted by SF-Alpha

For EeePC 901 / EeePC 1000 Users and others.

I have done Gentoo ebuild for Ralink RT2860 Linux Wireless Driver (net-wireless/ralink-rt2860-1.8.0.0), you can grab my ebuild here. This may need firmware from Ralink to work.

Note: I have updated from 1.8.0.0 to 1.8.0.0-r1, Enable WPA Supplicant support by default and add networkmanager use flags to support linux wext and NetworkManager

This is official Ralink's GPL drivers.

For the drivers of official kernel tree (RT2x00), currently rt2x00 project does not release any drivers for RT2860 at all (It done some in git, but not consider usable because it incomplete). I think it will be out for test around 2.6.30. Drivers from rt2x00 are based on mac80211 stack and much better in linux userland support.

If you have RT61, RT73, RT24xx and RT25xx, suggested  to use rt2x00 drivers from offical source tree, available from 2.6.24 and later.

Posted by SF-Alpha

I will start moving Gentoo mirrors from mirrors.64hosting.com. In fact, Currently, we using backup server for Gentoo mirrors because our primary mirror server was down because hardware failure a month ago and I did not replaced it yet.

Continue reading "Transfer Gentoo Mirror to Gentoo.in.th"

Posted by SF-Alpha